Senior Staff Tech Lead Manager, TPU Workload Onboarding and Optimization

• Act as a Senior Software Manager leading the onboarding for stable stack, and optimization for training and serving solutions on the latest TPU NPIs. Set the technical direction and architect the training/inference serving onboarding and optimization frameworks for our customer needs.
• Collaborate with the ML research, ML performance, model optimization tooling, and other optimization teams.
• Ship stable stack to our customers for training/inference optimization solutions on TPU.
• Prepare and optimize very large reference models, demonstrating state of the art single-host and multi-host inference solutions, especially at large-scale.
• Work closely with customers to onboard their workloads to production and optimization.

Minimum qualifications:

• Bachelor’s degree or equivalent practical experience.
  
• 8 years of experience with software development.
  
• 5 years of experience leading ML design and optimizing ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
  
• 2 years of experience with state of the art training (e.g. Megatron-LM, DeepSpeed) and inference techniques (e.g. TensorRT-LLM, vLLM, SGLang).
• 2 years of experience in a people management or team leadership role.

Preferred qualifications:

• Master’s degree or PhD in Engineering, Computer Science, or a related technical field.
  
• Experience in optimizing machine learning models for large-scale training/inference workloads.
• Experience in different large-scale ML optimizations techniques for improving latency and throughput.
• Experience with accelerators (TPUs or GPUs), or HPC .

• Bachelor’s degree or equivalent practical experience.
  
• 8 years of experience with software development.
  
• 5 years of experience leading ML design and optimizing ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
  
• 2 years of experience with state of the art training (e.g. Megatron-LM, DeepSpeed) and inference techniques (e.g. TensorRT-LLM, vLLM, SGLang).
• 2 years of experience in a people management or team leadership role.