GDM Staff Software Engineer, Agent Data Quality, DeepMind

• Construct rigorous quantitative benchmarks and automated evaluation frameworks (including LLM-as-a-judge) to measure agent capabilities in reasoning, planning, and tool use.
• Develop end-to-end data flywheel on agent evalset curation, and automate and productionize the flywheel with agentic data pipelines.
• Develop a dedicated platform for "Self-Service Judges," allowing researchers to rapidly prototype and test new LLM-based classifiers and annotations against historical trajectory data.
• Establish a formal ecosystem for metric lifecycle management, including a sandbox for validating new metrics against a library of high-confidence calibration experiments.
• Productionize automated insights tools to identify the top movers of metric deltas across complex slices such as task type and user cohort.

Minimum qualifications:

• Bachelor's degree in Computer Science or Engineering, or equivalent practical experience.
• Experience with Large Language Models (LLMs), NLP, or Generative AI.

Preferred qualifications:

• MBA or Master's degree in Computer Science.

• Bachelor's degree in Computer Science or Engineering, or equivalent practical experience.
• Experience with Large Language Models (LLMs), NLP, or Generative AI.