Staff Software Engineer, Inference Performance Optimization, GenAI, DeepMind

• Analyze and optimize AI inference workloads across the application, model, and distributed fleet infrastructure layers to methodically increase throughput-per-GPU and reduce latency.
• Design and implement inference optimization techniques.
• Investigate and resolve complex model inference performance bottlenecks across the stack.
• Model the latency-to-cost impacts of system variables (such as batch-sizing and utilization goals) and translate these insights into actionable signals that drive production systems.
• Develop investigative tools and metrics (e.g., compute/FLOPs funnels) that track where compute is spent across the fleet.

Minimum qualifications:

• Bachelor's degree in Computer Science, Computer Engineering, Electrical Engineering, Applied Mathematics, or a related technical field, or equivalent practical experience.
• 8 years of experience in software development.
• Experience in Python and C++, including navigating, debugging, and modifying serving codebases.
• Experience with AI model execution constraints, throughput-latency tradeoffs, memory bandwidth limitations, and modern serving architectures.

Preferred qualifications:

• Experience with real world LLM inference serving environments or direct contributions to modern open-source inference frameworks (e.g., vLLM, TensorRT-LLM, SGLang, Dynamo).
• Experience profiling workloads using standard ML profilers (e.g., PyTorch profiler) and internal trace analysis tools.
• Experience with observability and reliability for large distributed systems.
• Familiarity with GPU/TPU/accelerator performance concepts (e.g. memory bandwidth, quantization, collective communication, kernel), and can reason their implications to the overall inference serving performance.

• Bachelor's degree in Computer Science, Computer Engineering, Electrical Engineering, Applied Mathematics, or a related technical field, or equivalent practical experience.
• 8 years of experience in software development.
• Experience in Python and C++, including navigating, debugging, and modifying serving codebases.
• Experience with AI model execution constraints, throughput-latency tradeoffs, memory bandwidth limitations, and modern serving architectures.