Staff Software Engineer, Model Infrastructure

Highlights

• Lead the design of systems powering every AI request at Harvey.
• Build intelligent model routing based on reliability, latency and cost.
• Improve observability with health dashboards, usage analytics and cost reporting.

What they offer

• Equity
• Bonus

Ideal candidate

Seasoned distributed systems leader who thrives owning critical platforms and mentoring others. You care about reliability, efficiency, and enabling product teams with simple, scalable AI infrastructure.

Requirements

• 7+ years of software engineering experience building large-scale distributed systems.
• Experience designing and operating highly available production services.
• Strong programming skills in Go, Java, Python, Rust, or C++.
• Deep understanding of distributed systems, cloud infrastructure, networking, and observability.
• Experience leading technical projects across multiple engineering teams.
• Ability to balance long-term architecture with pragmatic execution.
• Strong communication and collaboration skills.
• Passion for building foundational platforms that enable other engineering teams.