Software Engineer, GPU Inference

Highlights

• Build disaggregated AI inference pairing GPU prefill with ultra-fast decode.
• Drive time to first token, throughput, and tail latency at rack scale.
• Make a new AI serving path reliable, correct, observable, and fast.

Ideal candidate

A hands-on systems owner who thrives debugging across hardware to APIs and cares deeply about production reliability, speed, and correctness for large-scale AI inference.

Requirements

• 5+ years of software engineering experience, including ownership of complex production systems
• Experience building, operating, or optimizing production inference systems for large language models, multimodal models, or similarly demanding GPU workloads
• Strong programming ability in C++ and Python, including multithreading, concurrency, memory management, and performance-sensitive software
• Experience optimizing Mixture-of-Experts or multimodal models
• Knowledge of GPU kernel optimization, operator fusion, graph capture, attention kernels, GEMM tuning, and communication/computation overlap
• Experience with reduced-precision inference and quantization formats such as BF16, FP8, FP4, INT8, or INT4, including validation of numerical and model-quality effects
• Experience building numerical-comparison, determinism, model-validation, or performance-regression test systems
• Experience collaborating directly with accelerator vendors, framework maintainers, or open-source communities