IN_Senior Associate_Agentic AI Testing (Playwright + AI)_digital Integration_Advisory_Kolkata

Job Profile : At PwC, our people in software and product innovation focus on developing cutting-edge software solutions and driving product innovation to meet the evolving needs of clients. These individuals combine technical experience with creative thinking to deliver innovative software products and solutions. In quality engineering at PwC, you will focus on implementing leading practice standards of quality in software development and testing processes. In this field, you will use your experience to identify and resolve defects, optimise performance, and enhance user experience. Job Description : Job Position Title: IN_Senior Associate_Agentic AI Testing_Digital Integration_Advisory_Kolkata Responsibilities: Validate the entire agentic lifecycle, starting from user intent and prompting through agent reasoning and routing. Ensure accurate MCP tool execution and conduct thorough end-to-end workflow validation. Utilize strong observability skills to monitor and assess the effectiveness of workflows. Employ automation techniques to streamline testing and validation processes. Conduct AI quality assessments to ensure optimal functionality and performance of the system. Collaborate with cross-functional teams to refine and enhance lifecycle validation methodologies. Mandatory skill sets: Strong experience in Agentic AI testing and validation. Must have hands-on knowledge of MCP testing, including tool invocation validation, payload/schema validation, orchestration, error handling, retries, and API/integration testing. Experience with AI observability and tracing tools such as LangFuse, Phoenix, or similar platforms for debugging, root cause analysis, and agent behavior validation. Strong understanding of AI quality metrics including hallucination detection, accuracy, correctness, groundedness, consistency, determinism, and latency. Experience designing and executing prompt testing strategies, including multi-turn, multi-hop, ambiguous, negative, and guardrail/jailbreak scenarios. Understanding of intent recognition, routing validation, agent selection, context retention, memory validation, and reasoning/planning validation. Ability to create test strategies, evaluation rubrics, golden datasets, and quality metrics for AI systems. Experience with API testing and automation frameworks (e.g., Postman, Katalon, Playwright, or equivalent). Preferred skill sets: Experience automating multi-agent validation and orchestration testing. Experience with RAG validation, retrieval quality assessment, and grounding/citation validation. Knowledge of AI governance, Responsible AI, and risk-based AI validation approaches. Experience building automated evaluation pipelines and AI scoring frameworks. Familiarity with token usage, model observability, and AI performance monitoring. Years of experience required: 4+yrs Education qualification: Bachelor's degree in Computer Science, IT, or a related field. Location Kolkata