Software Engineer, Model Routing & Inference

Highlights

• Build the inference platform powering every AI interaction in the product.
• Own the full inference path for agent sessions, chat and tab completion.
• Design intelligent cross-provider failover to prevent user-visible outages.

Ideal candidate

Truth-seeking, passionate engineer who thrives in small talent-dense teams, loves shipping code and debating ideas, and enjoys weighing latency, cost, and reliability tradeoffs at scale.

Requirements

• Deep experience building high-throughput, low-latency distributed systems, especially in inference serving, traffic routing, or real-time data pipelines
• Comfortable reasoning about cost/performance tradeoffs at scale (GPU utilization, provider economics, capacity planning)
• Strong software engineering fundamentals and enjoy shipping production systems that handle millions of requests
• Make good calls in the gray area weighing reliability, cost, latency, and user experience