Engineering Manager, Inference Router

Highlights

• Lead the intelligent control plane for the fastest generative AI inference platform.
• Own global request routing that decides where every inference request runs.
• Drive lowest time-to-first-token at massive scale for real-time AI apps.

Ideal candidate

A hands-on technical leader who thrives owning low-latency, large-scale systems and growing high-velocity infrastructure teams. You care deeply about operational excellence, mentorship, and turning routing innovation into real customer latency wins.

Requirements

• Proven experience building, managing and growing distributed systems or infrastructure engineering teams
• Background across distributed systems, Kubernetes-based infrastructure and ML systems
• Experience with global routing, load balancing and low-latency scheduling under bursty workloads
• Familiarity with prefill/decode disaggregation and prefix-aware cache-aware routing for high reuse
• Strong grounding in observability, SLOs and incident response for high-traffic production systems