Machine Learning Scientist - ML Research
Arena (formerly LMArena) · SF Bay Area
Infrastructureonsite
At a glance
Highlights
- Design experiments revealing what makes AI models useful and trustworthy.
- Analyze millions of human votes to uncover real-world model performance insights.
- Build new evaluation methods that go beyond traditional benchmarks and leaderboards.
What they offer
- Comprehensive health and wellness benefits, including medical, dental, vision
- Opportunity to work on cutting-edge AI with a small, mission-driven team
- Culture that values transparency, trust, and community impact
Ideal candidate
Curious, rigorous researcher with a PhD mindset who loves open-ended evaluation questions and turning real human preferences into transparent, impactful science.
Requirements
- PhD or equivalent research experience in ML, NLP, Statistics or related field
- Hands-on experience training large-scale models including reward and preference models
- Experience fine-tuning LLMs with RLHF, DPO and contrastive learning
- Strong foundation in ML and statistics and novel training and evaluation design
- Strong understanding of LLMs and modern architectures like Transformers and diffusion
- Proficiency in Python and PyTorch, JAX, or TensorFlow
- Demonstrated ability to design and analyze experiments with statistical rigor
- Experience publishing research or open-source work in ML, NLP or AI evaluation
- Comfortable with real-world usage data and metrics beyond standard benchmarks
Lab profile
About Arena (formerly LMArena)
Platform for evaluating AI models in the real world via crowdsourced preferences.
What they build
- Arena real-world AI evaluation platform
Known for
- Real-world model evaluation
- Large-scale crowdsourced human preferences
- Transparent benchmarking
Key facts
- Founded by UC Berkeley SkyLab researchers
- Tens of millions of monthly users
Posted · Last seen