Senior AI Data Infrastructure/Pipeline Engineer

XPENG

Full Time United States

AI Job Summary

Design and build end-to-end data pipelines processing petabyte-scale sensor data for autonomous driving systems, from onboard upload through cloud preprocessing to model training. You'll optimize pipeline performance, develop data cleaning and quality tools, and build a data management platform with version control and lineage tracking. Requires 3+ years in large-scale data processing with expertise in two of: ETL systems, distributed message queues, data lakes like Iceberg, or distributed system optimization. Python, Go, or Java required. Remote-eligible in the US; salary $145,600–$246,400.

Written from this posting by Neural Jobs AI. The full description is below.

AI Resume Tailoring Sign in to use this AI AI Cover Letter Sign in to use this AI

Job Description

XPENG is a leading smart technology company at the forefront of innovation, integrating advanced AI and autonomous driving technologies into its vehicles, including electric vehicles (EVs), electric vertical take-off and landing (eVTOL) aircraft, and robotics. With a strong focus on intelligent mobility, XPENG is dedicated to reshaping the future of transportation through cutting-edge R&D in AI, machine learning, and smart connectivity.

As a core member of our AI Infrastructure team, you will be responsible for building the end-to-end data pipeline for autonomous driving, covering the entire chain from onboard data upload → cloud-based preprocessing → dataset production → model training / simulation input. In autonomous driving systems, the stability and efficiency of the pipeline directly determine the speed of algorithm iteration. We look forward to building a reliable, observable, and cost-effective data pipeline that supports the daily flow of petabyte-scale sensor data.

Key Responsibilities

Responsible for the design and construction of core data closed loop pipelines. Develop toolchains for data cleaning, annotation quality inspection, and data mining to support the algorithm team in quickly locating model error cases and driving iterative model optimization.
Data Support for Production and R&D Processes. This includes log event tracking, connected vehicle data, internal and external data collection, data synchronization, data cleaning and standardization, data modeling, offline and real-time data processing, data as a service, and data visualization. Support business operations such as autonomous driving, smart cockpits, overseas data collection, and robotics data collection.
Responsible for optimizing the performance of the entire data pipeline (collection, cleaning, conversion). Solve bottlenecks in large-scale data transmission, memory management, I/O, etc., and build a distributed data processing system with high throughput and low latency.
Responsible for building a data management platform covering the entire process from data collection to data lake ingestion to model training. Implement capabilities for data version control, data lineage tracing, metadata management, and fast data retrieval to support unified data access and collaboration across multiple teams.
Collaborate with the large model team and other technical teams to deeply understand business requirements, respond quickly, and ensure successful implementation.

Basic Qualifications

Bachelor's degree or higher in Computer Science, Software Engineering, Artificial Intelligence, or related fields.
3-5+ years of experience in large-scale data processing or data platform development.
Proficiency in at least one programming language among Python / Go / Java. Solid software engineering foundation, good coding standards, and a strong sense of code quality.
Hands-on project experience in at least two of the following areas:
- Design and development of large-scale data pipelines / ETL systems, with end-to-end experience in data cleaning, transformation, and loading.
- Production-level experience with distributed message queues (Kafka / Pulsar / RabbitMQ), familiar with stream processing paradigms.
- Experience with distributed data lake systems (e.g., Apache Iceberg), familiar with Iceberg's table format, partition evolution, snapshot isolation, etc., with practical performance tuning and deployment experience.
- Experience with columnar storage formats (e.g., Lance) and related query engines, with practical application in large model training.
Hands-on experience using and optimizing relational databases (MySQL / PostgreSQL) and NoSQL databases (Redis / MongoDB). Understand metadata management and caching strategies.
Experience in performance optimization and troubleshooting for large-scale distributed systems, able to quickly locate and resolve complex performance bottlenecks. Experience with Kubernetes / Docker containerization deployment.
Strong cross-team communication and collaboration skills, high sense of responsibility, and proactive problem-solving attitude.

Preferred Qualifications

Familiarity with closed-loop data in the embodied AI industry will be a huge plus.
Some understanding of the autonomous driving industry, awareness of data closed loop and data flywheel concepts, and enthusiasm for this field.
Experience with AI infrastructure or model training workflows (e.g., data loading, feature engineering, data preparation for model evaluation).
Familiarity with data lake / data warehouse systems, with practical experience implementing data version control and data lineage tracing.
Open-source contributions on GitHub or a technical blog, with continuous attention to the latest technological trends in big data / AI infrastructure.

What do we provide:

A fun, supportive and engaging environment.
Infrastructures and computational resources to support your work.
Opportunity to work on cutting edge technologies with the top talents in the field.
Opportunity to make significant impact on the transportation revolution by the means of advancing autonomous driving.
Competitive compensation package.
Snacks, lunches, dinners, and fun activities.

The base salary range for this full-time position is $145,600 - $246,400, in addition to bonus, equity and benefits. Our salary ranges are determined by role, level, and location. The range displayed on each job posting reflects the minimum and maximum target for new hire salaries for the position across all US locations. Within the range, individual pay is determined by work location and additional factors, including job-related skills, experience, and relevant education or training.

We are an Equal Opportunity Employer. It is our policy to provide equal employment opportunities to all qualified persons without regard to race, age, color, sex, sexual orientation, religion, national origin, disability, veteran status or marital status or any other prescribed category set forth in federal or state regulations.

Do you match this job?

Check my match

XPENG

Robotics & Autonomy · China

XPENG is a leading smart technology company at the forefront of innovation, integrating advanced AI and autonomous driving technologies into its vehicles, including electric vehicles (EVs), electric vertical take-off and landing ( eVTOL ) aircraft, and robotics. With a strong focus on intelligent mobility, XPENG is dedicated to reshaping the future of transportation through cutting-edge R&D in AI, machine learning , and smart connectivity.

All jobs at XPENG

Location

United States

Job Overview

Job Posted:

3 months ago

Workplace

On-site

Job Type

Full Time

Education

Any

Experience

5+ years

Senior AI Data Infrastructure/Pipeline Engineer

AI Job Summary

Job Description

Location

Company

Talent

Employer

Support

Senior AI Data Infrastructure/Pipeline Engineer

AI Job Summary

Job Description

Location

Share This Job:

Related Jobs

Company

Talent

Employer

Support