AI Test Manager/Test Architect

Job Description: Description: * We are seeking an experienced AI Test Manager / Test Architect to lead and scale quality engineering capabilities across enterprise AI initiatives, with a strong focus on Generative AI and Agentic AI solutions. This role is responsible for defining and governing the end-to-end AI testing strategy, architecture, automation framework, and quality standards to ensure delivery of secure, scalable, reliable, and production-ready AI products. The individual will manage teams of AI Test Engineers while collaborating with Product, Engineering, Architecture, Security, and DevSecOps teams to drive quality transformation, Responsible AI adoption, and continuous improvement across the delivery lifecycle. Your Role: * Lead and manage teams of AI Test Engineers across multiple AI delivery programs. * Define and govern the AI testing strategy, operating model, quality standards, and delivery processes . * Establish end-to-end testing approaches across Generative AI, Agentic AI, functional, integration, performance, and security testing . * Build and scale AI-focused QE capabilities, accelerators, frameworks, and reusable assets. * Drive automation excellence through scalable and reusable frameworks using Playwright and Python . * Embed quality across the SDLC through shift-left practices, CI/CD integration, and DevSecOps quality gates . * Implement shift-right quality engineering using production monitoring, observability, and continuous feedback mechanisms. * Govern release readiness, defect management, quality metrics, and executive reporting to ensure predictable delivery outcomes. * Collaborate with cross-functional stakeholders and mentor teams to drive continuous improvement and AI quality transformation. Your Professional Experience: Must Have: * 12+ years of experience in Testing / Quality Engineering with 5+ years leading testing teams or large delivery programs. * Proven experience testing in Artificial Intelligence, Machine Learning, Generative AI, and Agentic AI solutions. * Experience implementing AI-powered QE platforms and enterprise-wide AI testing transformation initiatives. * Strong expertise in defining and governing enterprise AI testing strategies, operating models, and quality frameworks * Hands-on experience in AI validation, including model evaluation, prompt testing, output assessment, and agent behavior validation. * Strong automation experience with Playwright and Python, including building scalable and reusable automation frameworks. * Drive innovation through AI-enabled test automation and productivity improvements. * Deep understanding of QA methodologies including Agile, DevOps, Waterfall, and BDD. * Experience embedding quality controls into CI/CD and DevSecOps pipelines * Strong knowledge of Responsible AI practices, including fairness, explainability, bias, privacy, and security testing. * Experience managing test governance, defect triage, quality metrics, release readiness, and executive reporting * Expertise with test management and delivery tools such as Jira, Azure DevOps, or equivalent platforms. * Strong stakeholder management, communication, leadership, and mentoring capabilities. Desirable Experience: * Experience working within large enterprises, regulated industries, or consulting environments * Direct experience testing Agentic AI ecosystems, including multi-agent orchestration and interaction validation. * Exposure to AI observability, production monitoring, and shift-right quality practices * Knowledge of data governance, data quality, and AI lifecycle management * Experience establishing or scaling Quality Engineering / AI Testing Centers of Excellence (CoE) * Experience implementing cloud-native testing and enterprise DevSecOps practices * Familiarity with AI governance and enterprise operating models * Professional certifications such as ISTQB Foundation, ISTQB Agile Tester, ISTQB AI Tester, and cloud or AI/ML certifications.