Skip to content
← Arena (formerly LMArena)

Machine Learning Scientist - ML Research

Arena (formerly LMArena) · SF Bay Area

Infrastructureonsite
Apply on the lab’s site ↗

At a glance

Highlights

  • Design experiments revealing what makes AI models useful and trustworthy.
  • Analyze millions of human votes to uncover real-world model performance insights.
  • Build new evaluation methods that go beyond traditional benchmarks and leaderboards.

What they offer

  • Comprehensive health and wellness benefits, including medical, dental, vision
  • Opportunity to work on cutting-edge AI with a small, mission-driven team
  • Culture that values transparency, trust, and community impact

Ideal candidate

Curious, rigorous researcher with a PhD mindset who loves open-ended evaluation questions and turning real human preferences into transparent, impactful science.

Requirements

  • PhD or equivalent research experience in ML, NLP, Statistics or related field
  • Hands-on experience training large-scale models including reward and preference models
  • Experience fine-tuning LLMs with RLHF, DPO and contrastive learning
  • Strong foundation in ML and statistics and novel training and evaluation design
  • Strong understanding of LLMs and modern architectures like Transformers and diffusion
  • Proficiency in Python and PyTorch, JAX, or TensorFlow
  • Demonstrated ability to design and analyze experiments with statistical rigor
  • Experience publishing research or open-source work in ML, NLP or AI evaluation
  • Comfortable with real-world usage data and metrics beyond standard benchmarks

Lab profile

About Arena (formerly LMArena)

Platform for evaluating AI models in the real world via crowdsourced preferences.

What they build

  • Arena real-world AI evaluation platform

Known for

  • Real-world model evaluation
  • Large-scale crowdsourced human preferences
  • Transparent benchmarking

Key facts

  • Founded by UC Berkeley SkyLab researchers
  • Tens of millions of monthly users
More about Arena (formerly LMArena) →
Apply on the lab’s site ↗

Posted · Last seen