Senior Software Engineer, Machine Learning, TPU Workload Optimization

• Act as ML software engineer leading the onboarding for stable stack, and optimization for training and serving solutions on the latest TPU NPIs. Work closely with customers to onboard their workloads to production and optimization.
  
• Work on the technical details and architect the training/inference onboarding and optimization frameworks for our customer needs.
  
• Collaborate with the ML research, ML performance, model optimization tooling, and other optimization teams.
  
• Ship stable stack to our customers for training/inference optimization solutions on TPU.
  
• Prepare and optimize very large reference models, demonstrating SOTA single-host and multi-host inference solutions, especially at large scale.

Minimum qualifications:

• Bachelor’s degree or equivalent practical experience.
  
• 5 years of experience with software development in one or more programming languages.
• 2 years of experience leading ML design and optimizing ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
  
• 1 year of experience with training (e.g., Megatron-LM, DeepSpeed) and inference techniques (e.g., TensorRT-LLM, vLLM, SGLang).

Preferred qualifications:

• Master's degree or PhD in Computer Science or related technical field.
  
• Experience in optimizing machine learning models for large scale training/inference workloads.
• Experience in different large scale ML optimizations techniques for improving latency and throughput.
  
• Experience with accelerators (TPUs or GPUs), or HPC .

• Bachelor’s degree or equivalent practical experience.
  
• 5 years of experience with software development in one or more programming languages.
• 2 years of experience leading ML design and optimizing ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
  
• 1 year of experience with training (e.g., Megatron-LM, DeepSpeed) and inference techniques (e.g., TensorRT-LLM, vLLM, SGLang).