Staff Software Engineer, Inference API

Highlights

• Build production inference APIs powering ultra-fast disaggregated AI serving.
• Deliver unified request semantics across GPU prefill and Wafer-Scale decode.
• Enable new foundation models with streaming, structured outputs and tool calling.

What they offer

• Build breakthrough AI platform beyond GPU constraints
• Publish and open-source cutting-edge AI research
• Work on one of the fastest AI supercomputers
• Job stability with startup vitality and non-corporate culture

Ideal candidate

A hands-on ML systems engineer who loves turning fast-evolving model capabilities into durable production APIs and thrives at the intersection of distributed systems and user-facing design.

Requirements

• Understanding of model-specific tokenization, chat templates, generation configuration, logits processing, stopping criteria, tool calling, structured generation, and constrained decoding
• Experience with disaggregated prefill/decode architectures, KV-cache transfer, prefix caching, chunked prefill, memory-aware admission control, or request scheduling
• Experience designing, building, or operating production APIs and services for machine learning, large language models, or other data-intensive applications
• Familiarity with reduced-precision inference and quantization formats such as BF16, FP8, FP4, INT8, or INT4