Site Reliability Engineer, Inference Infrastructure

Highlights

• Build self-service systems powering Cohere's LLM API platform.
• Deploy optimized NLP models for low-latency high-throughput serving.
• Create customized model deployments directly with enterprise customers.

What they offer

• Weekly $75 lunch stipend
• Full health and dental benefits including mental health budget
• RRSP matching, 401K and pension scheme
• 100% parental leave top-up for up to 6 months
• 6 weeks paid vacation
• Annual enrichment, education and wellness stipends
• $500 home office stipend and co-working benefit

Ideal candidate

Seasoned infrastructure engineer energized by reliable, high-performance ML systems who thrives on automation, ownership, and collaborating across teams to solve evolving production challenges.

Requirements

• 5+ years of engineering experience running production infrastructure at a large scale
• Experience designing large, highly available distributed systems with Kubernetes, and GPU workloads on those clusters
• Experience with Kubernetes dev and production coding and support
• Experience with GCP, Azure, AWS, OCI, multi-cloud on-prem / hybrid serving
• Experience designing, deploying, supporting, and troubleshooting in complex Linux-based computing environments
• Experience in compute/storage/network resource and cost management
• Excellent collaboration and troubleshooting skills to build mission-critical systems
• Familiarity with computational characteristics of accelerators (GPUs, TPUs, and/or custom accelerators)
• Strong understanding or working experience with distributed systems
• Experience in Golang, C++ or other languages designed for high-performance scalable servers