Lead Data Platform Engineer - DataOps

About the Role

We are seeking a Lead Data Platform Engineer to own the resilience, scalability, and security of our core financial data ecosystem. This is a senior, hands-on engineering role for a technical expert who thrives on the unique challenge of designing, building, and running high-throughput, mission-critical batch and real-time data platforms in a regulated cloud environment.

You won’t just be maintaining pipelines; you will be driving the strategic evolution of our data infrastructure. Your mission is to build automated, self-healing platforms that eliminate operational toil, reliably support next-generation AI workloads, and ensure the seamless flow of highly sensitive financial data.

To succeed, you need a comprehensive understanding of cloud architecture and the technical depth to not only engineer sophisticated platform solutions but clearly articulate the why behind your choices to both engineering teams and stakeholders.

What You’ll Do

• Design & Implementation: Lead the architectural design and deployment of high-performance, secure data movement solutions within a highly regulated, mission-critical financial environment.
• Production Support & Reliability: Act as the subject matter expert and technical lead for resolving the most complex, high-impact incidents affecting data pipelines. Manage multiple stakeholders during critical events and perform in-depth root cause analysis to engineer out systemic failures.
• Data Security & Governance: Ensure the integrity and security of highly sensitive and critical data throughout the data lifecycle. Implement and enforce security best practices, including managing encryption at rest, in transit, access controls, and compliance.
• Automation & Tooling: Develop and implement automation for common operational tasks to reduce manual toil. Focus on building tools and monitoring solutions that provide end-to-end visibility into the health of our data ecosystem.
• Performance Optimization: Proactively analyze and tune the performance of data movement jobs, platforms, and underlying AWS infrastructure. Identify and implement optimizations to improve efficiency and reduce operational costs.
• Collaboration & Leadership: Act as a technical leader and mentor for both onsite and offshore team members. Ensure seamless collaboration, clear communication, and consistent operational standards across a distributed team while contributing to the long-term technical strategy and cloud modernization efforts.

What we’re looking for

• A Platform or SRE Background: You have a solid background in platform engineering, site reliability (SRE), or data operations, ideally within a large-scale or regulated environment.
• AWS Experience: You have a strong working knowledge of AWS and cloud-first data pipeline orchestration tools (like Airflow, Glue, or EMR).
• A Security Mindset: You understand the importance of compliance and security when managing sensitive data pipelines.
• Strong Automation Skills: You are highly comfortable using Python and Shell scripting to automate systems, manage dependencies, and handle large batch environments.
• A Great Collaborator: You enjoy mentoring others, can coordinate smoothly with distributed teams (onsite/offshore), and can explain technical concepts to non-technical stakeholders.

Where and how you'll work

This is a permanent position and is based in our Nottingham office.

We have a hybrid working model which gives you flexibility to work from our offices and from home. 

We’re big on collaboration and connection, so you’ll be based in our office 3 days a week on Tuesdays, Wednesdays and Thursdays.

What’s in it for you

• Bring us all this and you’ll be rewarded with a role contributing to the product roadmap for an organisation committed to transformation
• We’re continuing our journey into the public cloud and have problems of scale, security, availability and performance for you to help solve
• We offer high performers strong and diverse career progression, investing heavily in developing great people through our Capital One University training programmes (and appropriate external providers)
• Immediate access to our core benefits including pension scheme, bonus, generous holiday entitlement and private medical insurance – with flexible benefits available including season-ticket loans, cycle to work scheme and enhanced parental leave
• Open-plan workspaces and facilities designed to inspire and support you. Our Nottingham head-office has a fully-serviced gym, subsidised restaurant and dedicated development resources.

What you should know about how we recruit

We pride ourselves on hiring the best people, not the same people. Building diverse and inclusive teams is the right thing to do and the smart thing to do. We want to work with top talent: whoever you are, whatever you look like, wherever you come from. We know it’s about what you do, not just what you say. That’s why we make our recruitment process fair and accessible. And we offer benefits that attract people at all ages and stages.

We also partner with organisations including the Women in Finance and Race At Work Charters, Stonewall and upReach to find people from every walk of life and help them thrive with us. We have a whole host of internal networks and support groups you could be involved in, to name a few:

• REACH – Race Equality and Culture Heritage group focuses on representation, retention and engagement for associates from minority ethnic groups and allies
• OutFront – to provide LGBTQ+ support for all associates
• Mind Your Mind – signposting support and promoting positive mental wellbeing for all
• Women in Tech – promoting an inclusive environment in tech
• EmpowHER - network of female associates and allies focusing on developing future leaders, particularly for female talent in our industry
• Enabled - focused on supporting associates with disabilities and neurodiversity.

If you require a reasonable adjustment, please contact [email protected]  All information will be kept confidential and will only be used for the purpose of applying a reasonable adjustment.