Staff Research Software Engineer, Google Research

• Design and implement novel algorithms that optimize the inference stack for LLMs to ensure the highest througput and lowest latency for billions of users including sampling techniques, speculative decoding advancements, model compression, knowledge distillation and quantization strategies.
• Work on development and optimization of the low-level software that runs AI models efficiently. Implement and optimize custom kernels for inference, focusing on improving latency, throughput, and memory usage, across different hardware and model architectures.
• Co-design algorithms that exploit the unique architectectural advantages of custom hardware and accelerators. Collaborate with hardware architects on next-generation systems and use a deep understanding of both model architecture and hardware to inform algorithm selection.
• Amplify impact and influence the research ecosystem through publications and scientific dissemination.

Minimum qualifications:

• Bachelor’s degree or equivalent practical experience.
• 8 years of experience in software development.
• 5 years of experience testing, and launching software products, and 3 years of experience with software design and architecture.
• 5 years of experience with one or more of the following: Speech/audio (e.g., technology duplicating and responding to the human voice), reinforcement learning (e.g., sequential decision making), ML infrastructure, or specialization in another ML field.
• 5 years of experience with ML design and ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
• Experience integrating generative AI tools or LLM interfaces into workflows.

Preferred qualifications:

• PhD in Machine Learning, AI, Computer Science, Statistics, Applied Mathematics, Data Science, or a related technical field.
• Experience in a university or industry labs, with primary emphasis on AI research.
• Deep understanding of transformer architecture internals.
• Deep understanding of Machine Learning (ML) systems, distributed systems, compilers, or performance engineering. Experience in high-performance computing, or AI inference optimization and hands-on experience with low-level compute frameworks.
• Strong publication record in top AI venues.

• Bachelor’s degree or equivalent practical experience.
• 8 years of experience in software development.
• 5 years of experience testing, and launching software products, and 3 years of experience with software design and architecture.
• 5 years of experience with one or more of the following: Speech/audio (e.g., technology duplicating and responding to the human voice), reinforcement learning (e.g., sequential decision making), ML infrastructure, or specialization in another ML field.
• 5 years of experience with ML design and ML infrastructure (e.g., model deployment, model evaluation, data processing, debugging, fine tuning).
• Experience integrating generative AI tools or LLM interfaces into workflows.