AI Engineer: Evals
PhysicsX · London
Frontier labonsite
At a glance
Highlights
- Build the evaluation discipline that enables trustworthy agents in production.
- Create and publish open benchmarks for agents doing real engineering work.
- Connect offline evaluation, regression testing and production telemetry.
Ideal candidate
A hands-on engineer who has shipped agent evals in production and loves turning ambiguous goals into trusted measurements. You coach teams, work closely with domain experts, and care about rigor without slowing delivery.
Requirements
- Built and operated evaluation systems for agents or LLM-powered products in production
- Turn ambiguous product goals into datasets, evaluators, experiments and decision criteria
- Apply statistical judgement to failure modes of probabilistic systems
- Choose appropriately between deterministic checks, expert judgement, human review and LLM-as-judge
- Build usable frameworks and production-quality evaluation tooling
- Enable multiple teams to adopt evaluation practices through coaching and standards
- Partner with customers and domain experts to encode correctness without losing nuance
- Foundation in multi-step agent evaluation, observability, dataset and experiment design, and CI gates for non-deterministic systems
Lab profile
About PhysicsX
Physics AI company building simulation software to accelerate industrial hardware innovation.
What they build
- Simulation software stack for industrial engineering
Known for
- Physics AI
- Aerospace & defence
- CFD and manufacturing engineering
Posted · Last seen