Member of Technical Staff - Platform
Vals AI · San Francisco
Applied AIonsite$130k–$185k
At a glance
Highlights
- Own the platform that runs benchmarks evaluating LLMs at scale.
- Build distributed systems to run evaluations across models and machines at scale.
- Help establish the standard for measuring real-world LLM task performance.
What they offer
- Meaningful ownership
- Relocation and transportation support
- Health/dental insurance coverage
- Lunch and dinner provided, free snacks/coffee/drinks
- 401K plan
- Unlimited PTO
- $1,500 housing stipend (within one-mile radius)
Ideal candidate
You are a fast-learning, high-ownership builder who thrives in ambiguity and ships quickly. You care about rigorous LLM evaluation and want autonomy to shape systems in a talent-dense, in-person startup.
Requirements
- Strong engineering fundamentals with track record of building significant-scope projects
- Significant professional experience in Python
- System design knowledge including VMs, containerization, load balancers, databases
- Experience working with LLM APIs and concepts like temperature, tokenization, reasoning
- Experience with development sprints, Git workflows and pull request reviews
- Strong written and verbal communication and feedback skills
- Comfort breaking fuzzy problems into clear, actionable steps
- Tenacity to develop and iterate quickly
- remote policy
- In-person
Lab profile
About Vals AI
Builds expert-driven benchmarks and platform to evaluate LLMs for safety and enterprise use.
What they build
- LLM evaluation platform
- AI safety benchmarks
- Cybersecurity benchmarks
Known for
- LLM evaluation at scale
- AI safety benchmarking
- Cybersecurity benchmarking
- Expert-led scenario design
Posted · Last seen