Senior Principal Product Manager, AI Platform
New
We sent a six-digit code to . Enter it below — or use the link in the same email.
Or click the link in the same email — either works.
Enter the email address on your account and we'll send you a link to set a new password.
Remembered it? Sign In
Own inference and model-serving infrastructure end to end for AI agents in regulated industries, designing systems from architecture through production deployment and optimizing for reliability and scale. Requires 5+ years building ML inference systems using frameworks like TensorFlow Serving, Triton, or KServe, with strong distributed systems knowledge and hands-on experience with Kubernetes, Docker, and cloud platforms. On-site in San Mateo, California.
Written from this posting by Neural Jobs AI. The full description is below.
This is a hands-on infrastructure engineering role at an early-stage enterprise AI company building a context and data governance layer for AI agents deployed in highly regulated industries. You will own the inference and model-serving infrastructure end to end, making production AI agents fast, reliable, and scalable as concurrency grows.
Design, build, and own inference and model-serving infrastructure from initial architecture through production deployment.
Scale systems that enable AI agents to run reliably and efficiently under increasing concurrent load.
Identify and resolve infrastructure bottlenecks in collaboration with ML and platform engineering teams.
Drive performance optimization across latency, throughput, and reliability for production workloads.
5+ years building and operating ML inference systems, model-serving platforms, or ML infrastructure in production environments.
Hands-on experience designing and scaling inference-serving systems using frameworks such as TensorFlow Serving, TorchServe, Triton, KServe, or equivalent custom solutions.
Strong distributed systems fundamentals, including experience managing concurrent requests and resource allocation under load.
Proficiency with containerization and orchestration technologies, particularly Docker and Kubernetes, for ML workloads.
Experience with cloud infrastructure platforms (AWS, GCP, or Azure) for deploying and managing ML systems.
Solid monitoring and observability skills using tools such as Prometheus, Grafana, ELK, or distributed tracing solutions.
Proficiency in at least one systems or backend language: Python, Go, Rust, C++, or Java.
Familiarity with knowledge graphs, semantic search, or graph databases is a plus.
Background in agentic or autonomous AI systems, real-time inference, or enterprise data infrastructure is a plus.
On-site in San Mateo, California, United States. Visa sponsorship is not available for this role.
Here is what this employer asked for. Sign in and we will fill in your half.
Clera is an AI recruiting platform that introduces candidates directly to hiring managers at the companies they want to work for.
21 more Machine Learning Engineer roles in San Mateo
201K – 310K
Our estimate — this employer did not publish a salaryOur estimate, not the employer’s. Worked out from the middle half of 94 comparable roles on Neural Jobs that did publish a salary, in the same field, country and experience band. The real figure for this job may be different.
Search by role, company, or anything a posting mentions.
