Staff Software Engineer, Inference Infrastructure

Highlights

• Build next-gen AI platform serving frontier LLMs via easy APIs.
• Deploy optimized NLP models for low-latency, high-throughput serving.
• Create customized model deployments meeting enterprise customer needs.

What they offer

• Weekly lunch stipend of $75/£75 or equivalent
• Full health and dental benefits, including a separate budget for mental health
• RRSP matching, 401K, Pension Scheme
• 100% Parental Leave top-up for up to 6 months, for either parent
• Annual enrichment benefits for arts & culture, fitness/wellness, quality time and workspace
• Education & learning stipend for conferences, courses, and coaching
• 6 weeks of paid vacation (30 working days)
• Budget for traveling to other offices if you are remote, plus an annual company offsite
• $500 home office stipend
• Daily lunch program, snacks, and community events for those in the office
• Co-working benefit for those not near an office

Ideal candidate

Infrastructure engineer with large-scale production experience who is energized by high-performance ML systems, collaboration, and tackling evolving complex challenges with grit.

Requirements

• 5+ years of engineering experience running production infrastructure at a large scale
• Experience designing large, highly available distributed systems with Kubernetes, and GPU workloads on those clusters
• Experience with Kubernetes dev and production coding and support
• Experience with GCP, Azure, AWS, OCI, multi-cloud on-prem / hybrid serving
• Experience in designing, deploying, supporting, and troubleshooting in complex Linux-based computing environments
• Experience in compute/storage/network resource and cost management
• Excellent collaboration and troubleshooting skills to build mission-critical systems, and ensure smooth operations and efficient teamwork
• The grit and adaptability to solve complex technical challenges that evolve day to day
• Familiarity with computational characteristics of accelerators (GPUs, TPUs, and/or custom accelerators), especially how they influence latency and throughput of inference
• Strong understanding or working experience with distributed systems
• Experience in Golang, C++ or other languages designed for high-performance scalable servers