Staff Software Engineer, Inference Performance Optimization, GenAI

Highlights

• Push boundaries of large-scale AI inference to make it faster and cheaper.
• Analyze full inference stack to find bottlenecks and drive system improvements.
• Build tools tracking compute spend to guide latency and cost tradeoffs.

What they offer

• 20% bonus target
• equity
• benefits

Ideal candidate

Systems-driven engineer with deep performance intuition who thrives profiling complex distributed AI workloads and turning first-principles insights into faster, cheaper inference at scale.

Requirements

• Bachelor's degree in Computer Science, Computer Engineering, Electrical Engineering, Applied Mathematics, or related technical field, or equivalent practical experience
• 8 years of experience in software development
• Experience in Python and C++, including navigating, debugging, and modifying serving codebases
• Experience with AI model execution constraints, throughput-latency tradeoffs, memory bandwidth limitations, and modern serving architectures
• Experience with real-world LLM inference serving environments or contributions to open-source inference frameworks (e.g., vLLM, TensorRT-LLM, SGLang, Dynamo)
• Experience profiling workloads using standard ML profilers (e.g., PyTorch profiler) and internal trace analysis tools
• Experience with observability and reliability for large distributed systems
• Familiarity with GPU/TPU/accelerator performance concepts and implications for inference serving performance