logo
AI Resume Tailoring Sign in to use this AI AI Cover Letter Sign in to use this AI

Job Description

  • Architect, build, and operate large-scale environments that allow developers to test complex agent behaviors, tool calling, multi-turn workflows, and end-to-end workflows in isolated, reproducible sandboxes.
  • Deliver reproducible evaluation infrastructure and automated test harnesses capable of validating against enteprise visibility, security, compliance requirements.
  • Design and scale the platform infrastructure that enables agentic systems to validate online and audit offline and alert based on behavioral patterns.
  • Build the metrics engines and dashboards to track evaluation coverage, semantic correctness, quality metrics, and regression signals over time. Standardize the use of metrics for production health.
  • Integrate evaluation runs into deployment pipelines, creating release gates to prevent semantic and structural regressions before agent deployments reach production.

Minimum qualifications:

  • Bachelor's degree or equivalent practical experience.
  • 8 years of experience in software development.
  • 7 years of experience leading technical project strategy, ML design, and optimizing industry-scale ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
  • 5 years of experience with design and architecture, and testing, launching software products.
  • 2 years of experience with GenAI techniques (e.g., LLMs, Multi-Modal, Large Vision Models) or with GenAI-related concepts (language modeling, computer vision).

Preferred qualifications:

  • 10 years of experience leading the design and implementation of scalable distributed systems, ML platforms, or observability infrastructure, ideally in the context of large-scale agent platforms.
  • Experience leading engineering teams and collaborating with product managers and researchers to translate scientific concepts into reliable developer tools.
  • Strong understanding of LLM execution mechanics, tool-calling APIs, and model-level failure analysis.
  • Fluency in distributed tracing, semantic monitoring, and telemetry collection at scale.
  • Proven track record of shipping complex enterprise-grade developer platforms, virtualization/sandbox environments, or production-grade test harnesses.
  • Bachelor's degree or equivalent practical experience.
  • 8 years of experience in software development.
  • 7 years of experience leading technical project strategy, ML design, and optimizing industry-scale ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
  • 5 years of experience with design and architecture, and testing, launching software products.
  • 2 years of experience with GenAI techniques (e.g., LLMs, Multi-Modal, Large Vision Models) or with GenAI-related concepts (language modeling, computer vision).
Do you match this job?

Sign in and we will show how your role, experience, salary and location line up against what this employer asked for.

Check my match
Google
AI Research Lab · 500+ Members · Mountain View, CA, United States

Google builds internet, software, cloud, and AI products used by consumers, developers, and organizations. Its portfolio includes Search, YouTube, Android, Chrome, Maps, Gmail, Workspace, Google Cloud, advertising platforms, devices, and Gemini AI products. The company develops large-scale computing infrastructure and research that power information retrieval, communication, productivity, media, navigation, and machine learning. Google is the largest operating business within Alphabet and earns a substantial share of its revenue from digital advertising.

All jobs at Google

Location

United States

Job Overview
Job Posted:
3 days ago
Workplace
On-site
Job Type
Full Time
Education
Any
Experience
8+ years

Share This Job: