Performance Engineer, Inference

Highlights

• Own the production serving path for large distributed models end to end.
• Own the latency and throughput numbers the company plans against.
• Serve multilingual and multimodal models across a large multi-node fleet.

Ideal candidate

A systems-minded performance owner with production inference experience who obsesses over latency, throughput and cost, and thrives owning SLOs and collaborating across teams.

Requirements

• 5+ years in ML systems, with 2+ years on inference serving at production scale
• Experience serving 100B+ parameter models across multi-node parallelism in production
• Source-level fluency in SGLang, vLLM, Dynamo, or TensorRT-LLM with scheduler or KV modifications
• Operated and extended disaggregated prefill-decode with distributed KV/cache transfer and routing
• Built and trained speculative decoders with distillation and acceptance-rate tuning
• Deep understanding of KV cache internals, TP/PP/EP, NCCL, and multi-tenant isolation
• Profiling fluency and C++ and CUDA at read-and-modify level with on-call SLO ownership