Staff Software Engineer, Machine Learning, TPU Workload Optimization

• Act as ML software engineer leading the onboarding for stable stack, and optimization for training and serving solutions on the latest TPU NPIs.
• Set the technical direction and architect the training/inference onboarding and optimization frameworks for our customer needs. Work closely with customers to onboard their workloads to production and optimization.
• Collaborate with the ML research, ML performance, model optimization tooling, and other optimization teams.
• Ship stable stack to our customers for training/inference optimization solutions on TPU.
• Prepare and optimize very large reference models, demonstrating state of the art single-host and multi-host inference solutions, especially at large scale.

Minimum qualifications:

• Bachelor’s degree or equivalent practical experience.
  
• 8 years of experience in software development.
  
• 4 years of experience leading ML design and optimizing ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
• 2 years of experience with state of the art training (e.g. Megatron-LM, DeepSpeed) and inference techniques (e.g. TensorRT-LLM, vLLM, SGLang).

Preferred qualifications:

• Master’s degree or PhD in Engineering, Computer Science, or a related technical field.
  
• Experience in optimizing machine learning models for large scale training/inference workloads.
• Experience in different large scale ML optimizations techniques for improving latency and throughput.
• Experience with accelerators (TPUs or GPUs), or HPC.

• Bachelor’s degree or equivalent practical experience.
  
• 8 years of experience in software development.
  
• 4 years of experience leading ML design and optimizing ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
• 2 years of experience with state of the art training (e.g. Megatron-LM, DeepSpeed) and inference techniques (e.g. TensorRT-LLM, vLLM, SGLang).