Senior Analyst

KPMG is currently seeking an AI Agent Testing Specialist to join our Legal, Regulatory and Compliance organization.

• Execute structured test plans aligned with AI Agent Building Code and acceptance criteria. • Validate knowledge grounding to ensure responses use approved, current sources only. • Ensure adherence to instructions, guardrails, boundaries, and escalation protocols. • Test output quality for accuracy, completeness, consistency, and hallucination risks. • Conduct security checks including prompt injection, jailbreak, and data exposure testing. • Validate agent workflows, including source selection, fallback, and ambiguity handling. • Maintain comprehensive testing evidence, including results, defects, and sign-offs. • Support regression testing following changes to prompts, models, tools, or configurations. • Identify recurring issues and escalate defects to developers and stakeholders. • Contribute to continuous improvement via monitoring, reusable test assets, and collaboration.

Mandatory technical & functional skills • Bachelor’s degree in business, technology, data analytics, computer science, engineering, risk management, or related field • 1–3 years of experience in QA, testing, risk, data analytics, business analysis, or AI/automation support • Experience executing test scripts, documenting results, identifying defects, and tracking remediation • Strong attention to detail to assess accuracy, completeness, and alignment of AI outputs • Ability to follow structured testing protocols and identify edge cases or control gaps • Strong written communication skills for documenting test evidence, defects, and observations Preferred technical & functional skills • Familiarity with AI tools, generative AI, LLM workflows, chatbots, or agent-based solutions • Experience working with global (U.S.-based) stakeholders and managing multiple priorities • Understanding of risk management concepts, control frameworks, or professional services environments • Hands-on experience with Microsoft 365, SharePoint, Excel, PowerPoint, Teams, Copilot, or similar tools • Awareness of data privacy, confidentiality, access control, and responsible AI principles • Ability to analyze AI outputs critically for grounding, consistency, and reliability