Software Engineer - Inference Performance

Highlights

• Make demanding AI workloads run faster across inference engine and runtime.
• Turn latency wins into cost savings with higher tokens per GPU-hour.
• Bring up new model architectures on new hardware within release week.

What they offer

• Meaningful equity (U.S. only)
• 100% coverage of medical, dental, and vision insurance for employee and dependents
• Flexible PTO including Winter Break closure
• Paid parental leave
• Fertility and family-building stipend through Carrot (U.S. only)
• Company-facilitated 401(k)

Ideal candidate

First-principles performance thinker who thrives in fast-paced startups and obsesses over making demanding LLM workloads faster and more efficient in production.

Requirements

• Bachelor's, Master's, or Ph.D. degree in Computer Science, Engineering, Mathematics, or related field
• Experience with one or more general-purpose programming languages, such as Python or C++
• Familiarity with LLM optimization techniques (e.g., quantization, speculative decoding, continuous batching)
• Strong familiarity with ML libraries, especially PyTorch, TensorRT, or TensorRT-LLM
• Demonstrated interest and experience in LLMs
• Deep understanding of GPU architecture