LLM Research & Validation Specialist

Job title: LLM Research & Validation Specialist

Location: Abu Dhabi, UAE

Role purpose:

• Lead frontier research, quantitative evaluation and independent validation of large language models, multimodal models, retrieval-augmented generation systems and agentic AI used or proposed by ADIB. Translate mathematical and scientific methods into reproducible validation tests, challenger analyses, runtime controls and decision-useful evidence for model governance.

• The role combines deep technical research with second-line effective challenge. 

• It is expected to build validation toolkits and evaluation harnesses, independently assess conceptual soundness and production behaviour, and communicate material limitations clearly to technical teams, senior management and governance forums. 

• The role does not own model development or production approval.

Key accountabilities /responsibilities:

• Lead independent validation of LLM, multimodal, RAG and agentic AI use cases across design, implementation, deployment and ongoing monitoring.

• Assess transformer architecture, tokenisation, embeddings, attention, context-window behaviour, decoding, fine-tuning, alignment, quantisation and inference configuration.

• Design reproducible evaluation harnesses, golden datasets, adversarial suites, counterfactual tests, canary sets and statistically defensible acceptance criteria.

• Evaluate task performance, hallucination and factuality, calibration, robustness, stability, long-context behaviour, retrieval quality, grounding, citation faithfulness and uncertainty.

• Perform deep testing of prompt injection, indirect injection, data leakage, tool-use safety, excessive agency, multi-step failure propagation, kill-switches and human oversight.

• Apply probability, statistics, optimisation, information theory, numerical methods and experimental design to develop challenger tests and quantify uncertainty.

• Review data provenance, representativeness, contamination, benchmark validity, leakage, drift and limitations of synthetic or LLM-generated evaluation data.

• Build and maintain reusable Python-based validation tooling, automated test pipelines, experiment tracking, results repositories and technical documentation.

• Conduct structured research on emerging model architectures, interpretability, mechanistic analysis, scalable oversight, model evaluation and AI safety methods.

• Independently challenge model owners, vendors and developers, document findings, propose risk-based restrictions and track remediation without assuming first-line ownership.

• Prepare validation reports, research notes, standards, committee papers and senior-management briefings that clearly distinguish evidence, judgement and residual uncertainty.

• Mentor junior validators, improve team methodology and support knowledge transfer across Model Risk 

Education and experience:

• Master's degree in Theoretical Physics, Applied Physics, Mathematics, Applied Mathematics or a closely related quantitative discipline is required. A PhD or research-intensive master's is strongly preferred.

• Typically, one to three years of relevant experience in AI research, machine learning, quantitative modelling, model validation, scientific computing or a closely related field. Exceptional research profiles may be considered based on demonstrated capability.

• Deep understanding of probability, statistics, linear algebra, optimisation, numerical computation, experimental design and uncertainty quantification.

• Strong understanding of transformers, LLM training and inference, embeddings, RAG, fine-tuning, alignment, evaluation, agentic systems and AI safety failure modes.

• Advanced Python proficiency and experience with scientific and ML libraries. Exposure to PyTorch, Hugging Face, evaluation frameworks, experiment tracking, SQL, Git and cloud AI platforms is expected.

• Ability to read research papers critically, reproduce methods, design-controlled experiments and convert findings into bank-grade validation evidence.

• Experience with red teaming, adversarial testing, interpretability, calibration, robustness, privacy, security or model risk management is strongly advantageous.

• Excellent technical writing and communication, including the ability to explain mathematical concepts, assumptions and limitations to non-specialist stakeholders.

• Banking experience is advantageous but not mandatory. The role requires willingness to develop knowledge of financial services, Islamic banking, CBUAE expectations and ADIB governance.

Indicative success measures:

• Validation conclusions are reproducible, evidence-based and proportionate to use-case risk.

• Reusable evaluation assets and automation measurably improve validation coverage, consistency and efficiency.

• Material LLM and agentic risks are identified early, clearly communicated and translated into actionable controls or use restrictions.

• Research outputs strengthen ADIB validation methodology and remain traceable to tested evidence rather than unsupported claims.

• Stakeholders receive constructive, independent challenges while second-line ownership and decision rights remain clear.