Applied Science Manager, Artificial General Intelligence , Frontier AI Assets: Assessments

Highlights

• Lead quality strategy safeguarding data workflows for Amazon Nova models.
• Build LLM-as-a-Judge systems and evaluation rubrics for automated QA.
• Drive auditing innovation to power state-of-the-art LLM products.

Ideal candidate

Innovative science leader passionate about data quality and LLM evaluation who thrives mentoring teams and driving rigorous auditing standards to improve customer experiences.

Requirements

• Background in machine learning, statistics, quality assurance and auditing methodologies
• Experience developing automated evaluation systems including LLM-as-a-Judge
• Experience leading, hiring and developing a team of Applied Scientists
• Ability to design auditing strategies, SOPs, quality metrics and sampling methodologies
• Ability to communicate with senior leadership and cross-functional technical teams