Software Engineer, Model Runtime

Highlights

• Build LLM inference runtime executing frontier models on custom AI silicon at scale.
• Design scheduling, batching and distributed execution across chips, hosts and racks.
• Co-design runtime with silicon teams to shape future AI accelerator generations.

What they offer

• equity

Ideal candidate

Systems-oriented engineer who thrives across hardware and software boundaries and cares about production reliability at scale. You reason quantitatively about performance and enjoy driving ambiguous, cross-functional problems to closure.

Requirements

• Strong systems programming experience in C++, Rust, Python, or comparable performance-oriented environments.
• Built or optimized runtimes, distributed systems, compilers, kernels, model-serving infrastructure, or adjacent systems software.
• Understand modern LLM inference, including prefill and decode behavior, batching, KV-cache tradeoffs, and model parallelism.
• Can reason quantitatively about latency, throughput, compute intensity, memory bandwidth, communication, and utilization.
• Comfortable profiling and debugging performance across multiple layers of a hardware-software stack.
• Can design clean abstractions while retaining low-level control needed to extract performance from specialized hardware.
• Work effectively across model, systems, compiler, kernel, and hardware teams to drive ambiguous technical problems to closure.
• Care about production quality, including correctness, observability, reliability, maintainability, and graceful behavior at scale.