Senior Manager - AI Safety and Evaluation Engineering

Abu Dhabi Commercial Bank View all jobs

  • Abu Dhabi
  • Permanent
  • Full-time
  • 1 day ago
Embark on a journey where your unique contributions are celebrated, and your professional growth is embraced. At ADCB, we nurture a diverse, inclusive community where every voice is valued.About the business areaGBS is a group of highly skilled and talented professionals who form an essential part of ADCB's continued journey of success. With a proud history of commitment, innovation and delivery, GBS constantly strives for excellence whilst ensuring the highest standards of quality and risk awareness. Each and every member of the GBS family plays an integral role in driving ADCB's strategy, growth and digital evolution by working closely with our valued business partners to achieve exceptional customer experience through our outstanding service and support.We are actively seeking an ambitious professional to join our team at ADCB to work alongside passionate colleagues who share your ambition to redefine excellence in UAE bankingIn this role, your key responsibilities include:
  • To design, implement, and scale out technical measures to ensure the reliability, safety, and security of the bank's AI solutions, including rigorous evaluation frameworks for both predictive ML and Generative AI and standardized monitoring protocols, while serving as the technical liaison to second-line risk functions to ensure AI deployments meet the bank's risk and compliance standards, Awareness of AI Governance, AI Ethics, and Responsible AI principles is required, with a primary focus on technical practitioner responsibilities.
  • Methodology Design: Define standardized evaluation rubrics and scoring systems for AI system performance, reliability, safety and safety. Tailor for different GenAI system archetypes (e.g., Conversational AI, RAG and variants, agents and agentic workflows, AI-enabled data access)
  • Test Cases: Design comprehensive test cases covering deterministic logic, probabilistic outputs, and edge-case scenarios. This should include both benign as well as adversarial scenarios
  • Explainability & Fairness: Implement XAI techniques (e.g., Shapley Values, Integrated Gradients) and unfair bias detection metrics to ensure model transparency and regulatory compliance - e.g., in lending and fraud
  • Embedding into MLOps platform: work with MLOps platform product and engineering leaders to embed above as automated capabilities
  • Integrated Observability Architecture: Partner with MLOps/ LLMOps teams to design and embed telemetry pipelines that capture inputs, outputs, and embeddings, ensuring real-time visibility into model health while maintaining strict PII and data privacy standards.
  • Standardized Metric Frameworks: Establish "Golden Signals" (e.g., latency, drift, hallucination rates, and semantic similarity) and design automated "circuit-breakers" to intercept or reroute model traffic when performance or safety thresholds are breached.
  • Act as a central consultant for individual AI project teams, providing training on evaluation best practices and safety protocols.
  • Standardization: Establish and maintain a unified "Safety Stack" (tools for monitoring, testing, and explainability) and underlying methodology to ensure consistency across the bank's AI portfolio
The ideal candidate should have the following experience:
  • At least: 7 years of experience in ML Engineering, Data Science, or AI Evals/ Testing/ Safety Ideally with experience in financial sector
  • Bachelor's degree in information technology, Computer Science, Engineering or a related discipline
  • Core Skills: Expert Python proficiency; deep knowledge of statistical testing, XAI libraries, and LLM evaluation frameworks (e.g., Ragas, Giskard, Arize).
  • Systems Design: Proven ability to design observability and monitoring systems at scale
  • Leadership: Experience leading cross-functional initiatives or acting as a technical mentor/consultant.
What we offer:Comprehensive Benefits Package: This includes market-leading medical insurance, group life and personal accident insurance, paid leave and leave airfare, employee preferential rates on loans and finance facilities, staff discounts and offers, and children education assistance (for certain job levels).Flexible and Remote Working Options: We understand the importance of work-life balance and offer flexible working arrangements, subject to eligibility and job requirements.Learning and Development Opportunities: We value and facilitate continuous learning and personal development, through a variety of exciting learning opportunities, such as structured instructor-led courses, a comprehensive e-Learning catalog, on-the-job training and professional development programs.At ADCB, we are dedicated to creating a respectful, caring and disciplined work environment that aligns with your career ambitions.Job DetailsJob Location:Abu Dhabi, AE, 939Employment Status:

Abu Dhabi Commercial Bank