ML Model Serving Engineer

Highlights

• Turbocharge serving for LLM, speech, and vision models powering new consumer products
• Make models faster without sacrificing quality to enable lifelike voice computers.
• Help define a new consumer product category for lifelike voice agents.

What they offer

• 401(k) max employer match: 3.5% of compensation
• 100% employer-paid health, vision, and dental benefits for you and your dependents
• Unlimited PTO and sick time
• Flexible spending account with employer matching up to $1,650/year (medical FSA)
• Guardian Employee Assistance Program (EAP)
• Opportunity to share in the company's success with competitive stock options

Ideal candidate

Performance-obsessed systems thinker who loves squeezing speed from large models and shipping reliable, high-throughput services for real consumer products. You stay current on serving tricks and lead complex projects end to end.

Requirements

• Expert in differentiable array computing framework, preferably PyTorch
• Expert in optimizing ML models for high-throughput, low-latency reliable serving
• Significant systems programming experience with high-performance server systems
• Significant performance engineering experience including bottleneck analysis and profiling
• Up to date on latest techniques for model serving optimization
• Familiarity with high-performance LLM serving, VLLM and SGLang deployment and internals
• Experience with public cloud platform such as GCP, AWS, or Azure
• Experience deploying and scaling inference workloads using Kubernetes, Ray, etc.
• Track record of leading complex multi-month projects without assistance
• Excited to learn new things and work in a multitude of roles