Novel AI Lead Methodologist

• Drive the methodological frontier of model evaluation. Partner with DeepMind and Data Science, developing novel, data-driven methodologies for structured and unstructured testing of emerging AI products. Move beyond standard benchmarks, designing sophisticated experimental frameworks, uncovering latent model behaviors and capabilities.
• Define testing and safety standards, working with cross-functional colleagues to ensure they are met. Perform analyses and drive insights to develop model-level and product-level safety mitigations.
• Lead and influence cross-functional teams to implement safety initiatives. Advise executive leadership on complex safety issues.
• Represent Google's AI safety efforts in external forums and collaborations, contributing to industry-wide best practices. Mentor analysts, fostering a culture of excellence, acting as a subject matter expert on adversarial techniques.
• Work with sensitive content or situations and may be exposed to graphic, controversial or upsetting topics or content.

Minimum qualifications:

• Bachelor's degree or equivalent practical experience.
  
• 10 years of experience in AI testing or research, data analytics, data science, or a related field.

Preferred qualifications:

• Master's degree or PhD in relevant field.
• 5 years of experience in data analysis for AI Testing with experience in SQL or Python.
• Experience building or partnering with engineering teams to build prototypes for AI testing.
• Experience in designing and conducting experiments or quantitative research, preferably in a technology or AI context.
• Experience in AI systems, machine learning, and their potential risks.
• Strong technical competency with a data-driven investigative approach to solve complex tests, including demonstrable proficiency in data manipulation, analysis, and automation using languages like Python and SQL.

• Bachelor's degree or equivalent practical experience.
  
• 10 years of experience in AI testing or research, data analytics, data science, or a related field.