Data-AI Architect

What will I be doing?

Define and evolve enterprise data architectures, evaluate trade-offs, and recommend fit-for-purpose technology patterns across batch, streaming, lakehouse, warehouse, and operational use cases.

Review and advise other architects on the data aspects of their RFCs, helping ensure consistency with enterprise data principles, governance standards, and architectural direction.

Lead the adoption of data-mesh principles, including domain-oriented ownership, data as a product, federated computational governance, self-serve platform capabilities, discoverability, quality, and measurable data-product SLAs.

Establish reference architectures and engineering standards for data products, pipelines, ingestion, storage, processing, orchestration, observability, lineage, security, and access management.

Provide oversight on operational SLAs, including latency, cost, quality, freshness, reliability, and production performance.

Design and govern streaming architectures using technologies such as Kafka, Kinesis, Flink, Spark Structured Streaming, and Databricks, supporting use cases from scheduled batch through sub-second real-time processing.

Define reliable event-processing patterns, including schema and data contracts, schema registries, event-time processing, late-event handling, idempotency, deduplication, replay and reprocessing, dead-letter flows, and freshness SLAs.

Shape semantic layers and enterprise ontologies that create consistent business meaning across domains, including canonical entities, metrics, dimensions, relationships, business definitions, metadata, lineage, and versioning.

Establish patterns that allow semantic models to serve analytics, operational applications, machine learning, and AI use cases without creating duplicated or contradictory definitions.

Guide the evolution of cloud data platforms and lakehouse capabilities, including Databricks, Unity Catalog, Delta/Iceberg tables, object storage, data warehouses, and BI consumption layers across AWS and GCP environments.

Provide architectural direction for MLOps and feature-platform capabilities, including batch and online features, model-serving integrations, low-latency data paths, model/data lineage, monitoring, and governance.

Lead or contribute to architecture RFCs, technical decisions, design reviews, migration plans, and implementation roadmaps; make complex trade-offs clear to both technical and non-technical stakeholders.

Partner with domain teams to clarify ownership, data-product responsibilities, operational handover, quality accountability, access approval, and cross-domain consumption models.

Define practical controls for data quality, observability, privacy, security, resilience, cost management, and production readiness.

Take ownership of critical architectural issues, facilitate resolution across teams, and ensure decisions are followed through to implementation and operation.

Act as a trusted advisor and technical mentor to data engineers, platform teams, data scientists, MLOps engineers, BI teams, and engineering leaders.

Communicate a cohesive architectural vision while remaining pragmatic, adaptable, and close enough to implementation to validate that designs work in production.

What skills do I need?

8–10+ years of experience designing and operating scalable data architectures, preferably in complex enterprise or high-growth environments.

Strong experience designing and implementing data-mesh architectures and operating models, including domain ownership, data products, federated governance, self-serve platforms, contracts, quality, and discoverability.

Deep experience with streaming and event-driven architectures, including Kafka or Kinesis and one or more processing engines such as Flink or Spark Structured Streaming.

Demonstrated ability to design for real-time and near-real-time workloads, including latency measurement, event-time semantics, late data, state, deduplication, idempotency, replay, and failure recovery.

Strong knowledge of semantic layers, business ontologies, canonical data models, knowledge graphs or metadata models, metric definitions, and semantic governance.

Expertise in data modeling, data lake and lakehouse patterns, warehouse design, data pipelines, data products, metadata, lineage, and data management technologies.

Experience with cloud data platforms and services, particularly AWS and/or GCP; experience with Databricks, Unity Catalog, Delta Lake, Iceberg, or comparable technologies is valuable.

Proficiency with relevant data and platform technologies such as Spark, Airflow, dbt, Kafka, Python, SQL, CI/CD, infrastructure-as-code, and observability tooling.

Experience architecting or supporting MLOps, feature stores, online/offline data serving, or other low-latency machine-learning data systems.

Ability to establish practical frameworks for data access, stewardship, governance, privacy, security, quality, and operational accountability.

Strong understanding of reliability, scalability, performance, resilience, cost, and vendor lock-in trade-offs.

Excellent stakeholder-management, communication, facilitation, and influencing skills, including the ability to balance delivery expectations and technical excellence.

Comfortable managing risk, ambiguity, and conflict; able to make decisions and explain the reasoning behind them.

Self-sufficient and proactive, with the judgment to know when to seek input and when to move forward.