Staff Software Engineer - GenAI inference

Highlights

• Lead architecture of inference engine powering Foundation Model API.
• Bridge research advances and production for large-scale LLM serving.
• Drive routing, batching and memory scaling across distributed nodes.

What they offer

• Annual performance bonus
• Equity

Ideal candidate

Performance-obsessed systems leader with 6+ years owning complex infrastructure who thrives bridging research and production. You care about throughput, reliability, and guiding teams through influence to ship scalable GenAI systems.

Requirements

• BS/MS/PhD in Computer Science, or a related field
• Strong software engineering background (6+ years or equivalent) in performance-critical systems
• Proven track record of owning complex system components and driving architectural decisions end-to-end
• Deep understanding of ML inference internals: attention, MLPs, recurrent modules, quantization, sparse operations
• Hands-on experience with CUDA, GPU programming, and key libraries (cuBLAS, cuDNN, NCCL)
• Strong background in distributed systems design, including RPC frameworks, queuing, batching, sharding
• Demonstrated ability to uncover and solve performance bottlenecks across layers
• Experience building instrumentation, tracing, and profiling tools for ML models
• Ability to lead through influence and translate novel model ideas into production systems
• Excellent communication and leadership skills, with proactive ownership mindset