Research Engineer, Serving Quality, DeepMind

• Design and develop metrics that measure model performance, and detect anomaly events on serving stack (such looping, leakage and so on).
• Discover novel serving or modeling quality issues, as well as anomaly events through large-scale real user data mining.
• Work cross-functionally on logging, serving, and management to quantify the improvements.
• Contribute to reporting, analytical infrastructure, model testing framework including replay, evals and dashboarding.
• Provide insights to improve our frontier models serving quality and improve user experiences. 

Minimum qualifications:

• Bachelor's degree in Computer Science, Mathematics, Statistics, Machine Learning, a related field, or equivalent practical experience.
• Experience with Python or C++.
• Experience with machine learning and statistics.

Preferred qualifications:

• Experience in software engineering and working on large-scale ML projects.
• Experience working on projects from proof-of-concept through to implementation.
• Experience in experiment analysis.
• Experience with TensorFlow or similar ML frameworks (e.g. JAX).
• Experience conducting applied research to improve the quality and training/serving efficiency of large transformer-based models.
• Familiarity with LE and Rasta.

• Bachelor's degree in Computer Science, Mathematics, Statistics, Machine Learning, a related field, or equivalent practical experience.
• Experience with Python or C++.
• Experience with machine learning and statistics.