Skip to content
← Profound

Member of Technical Staff, Evals and Post-Training

Profound · New York, New York · San Francisco, California

Applied AIonsite$230k–$325k
Apply on the lab’s site ↗

At a glance

Highlights

  • Build eval systems that measure how well AI performs marketing tasks.
  • Turn eval results and expert feedback into training signal for models.
  • Wire evals into CI, launch reviews, and model selection decisions.

What they offer

  • equity

Ideal candidate

A builder with 2+ years in measurement and evals who loves separating signal from noise and has strong opinions on agent behavior. You thrive in fast startups and care about defining quality for AI marketers.

Requirements

  • 2+ years building evaluation or measurement systems, including AI evals, experimentation, ranking, or search quality
  • Strong Python and data skills with track record shipping production systems
  • Deep familiarity with LLM evaluation including LLM-as-judge, benchmark design, agent harnesses, and calibration
  • Sound experimental design with instinct to separate signal from noise in non-deterministic systems
  • Familiarity with modern post-training methods including SFT, preference optimization, and RL
  • Strong opinions about how models and agents should behave and keeps up with current research
  • Comfortable operating in fast-paced startup environment with rapid iteration cycles

Lab profile

About Profound

AI marketing platform for research, content, and campaigns with visibility into AI answers.

What they build

  • AI marketing platform
  • AI Marketer agent
  • AI visibility analytics and reporting

Known for

  • Used by 15% of the Fortune 500
  • Customers include Comcast, Estee Lauder and Walmart
  • Built on 2B+ real user prompts
More about Profound →
Apply on the lab’s site ↗

Posted · Last seen