Production Engineer, LLM Serving

Highlights

• Own reliability and performance of production LLM serving on custom AI silicon.
• Drive latency, throughput and efficiency gains through workload-driven measurement.
• Lead incident response and build automation to reduce operational toil.

Ideal candidate

A systems-minded engineer with 3+ years running production services who loves measurement, debugging distributed performance, and collaborating across teams to make AI inference fast and reliable.

Requirements

• Bachelor's degree in Computer Science or related field, or equivalent practical experience, with 3+ years developing and operating production services
• Strong programming skills in Rust, C++, Python, or Go, with ability to read, instrument, and improve production code
• Hands-on experience operating services in Kubernetes with solid Linux, containers, networking, concurrency and distributed systems knowledge
• Understanding of LLM serving concepts including prefill/decode, batching, KV-cache management and request scheduling
• Experience profiling production services and analyzing telemetry to improve tail latency, throughput and efficiency
• Experience building observability with metrics, logs and traces to diagnose incidents and drive preventive improvements
• Ability to communicate quantitative findings and collaborate across teams to deliver measurable production improvements