Senior Software Engineer, ML API – AI Inference

Highlights

• Build the ML API layer powering disaggregated AI inference systems.
• Deliver consistent streaming, tool use, and multimodal inference behavior.
• Own features end to end from design through production operation.

Ideal candidate

Hands-on senior engineer who loves API design and distributed ML systems and owning features end to end. You care about reliability, clear semantics, and translating model capabilities into great developer experiences.

Requirements

• Experience with multi-model or multi-tenant serving, including routing, admission control, quotas, rate limiting, or capacity-aware scheduling
• Experience developing performance-sensitive or highly concurrent services in Go, C++, Rust, or a similar language
• Familiarity with GPU inference, AMD Instinct accelerators, ROCm, CUDA, collective communication, or multi-node model serving
• Familiarity with reduced-precision inference and quantization formats such as BF16, FP8, FP4, INT8, or INT4
• Experience building SDKs, developer tools, configuration systems, or self-service ML workflows, or collaborating directly with customers and open-source communities