Senior ML Infrastructure Engineer

Highlights

• Own the production inference stack serving models on RDU accelerators.
• Take speculative decoding and long-context serving from prototype to production.
• Ship high-throughput, low-latency services via SambaStack and SambaCloud.

What they offer

• Equity
• 95% employee medical premium coverage
• 77% dependent medical premium coverage
• HSA with employer contribution
• Dental and vision insurance
• Short/long term disability insurance
• Life and AD&D insurance
• FSA options
• Headspace subscription
• Gympass+ membership
• One Medical membership
• Employee Assistance Program

Ideal candidate

Systems-oriented engineer with experience running large-scale distributed services who loves turning inference ideas into reliable, low-latency production systems and collaborating across ML, hardware, and product teams.

Requirements

• B.S. in Computer Science, Electrical Engineering, or related field
• 5+ years building and operating large-scale distributed systems, ideally in ML serving
• Strong fundamentals in algorithms, data structures, concurrency, and systems design
• Experience with production services with strict latency, throughput, and availability requirements
• Working knowledge of LLM inference techniques and serving stacks like vLLM, TensorRT-LLM, or SGLang
• Proficiency in Python
• Experience collaborating across teams on system-level solutions