Senior Associate_Agentic AI + Automation Testing

Job Description & Summary : We are looking for someone who can validate the full agentic lifecycle—from user intent and prompting, through agent reasoning and routing, to MCP tool execution and end-to-end workflow validation—with strong observability, automation, and AI quality assessment skills Job Position Title: Senior Associate Responsibilities: · Validate the entire agentic lifecycle, starting from user intent and prompting through agent reasoning and routing. · Ensure accurate MCP tool execution and conduct thorough end-to-end workflow validation. · Utilize strong observability skills to monitor and assess the effectiveness of workflows. · Employ automation techniques to streamline testing and validation processes. · Conduct AI quality assessments to ensure optimal functionality and performance of the system. · Collaborate with cross-functional teams to refine and enhance lifecycle validation methodologies. Mandatory skill sets: · Strong experience in Agentic AI testing and validation. · Must have hands-on knowledge of MCP testing, including tool invocation validation, payload/schema validation, orchestration, error handling, retries, and API/integration testing. · Experience with AI observability and tracing tools such as LangFuse, Phoenix, or similar platforms for debugging, root cause analysis, and agent behavior validation. · Strong understanding of AI quality metrics including hallucination detection, accuracy, correctness, groundedness, consistency, determinism, and latency. · Experience designing and executing prompt testing strategies, including multi-turn, multi-hop, ambiguous, negative, and guardrail/jailbreak scenarios. · Understanding of intent recognition, routing validation, agent selection, context retention, memory validation, and reasoning/planning validation. · Ability to create test strategies, evaluation rubrics, golden datasets, and quality metrics for AI systems. · Experience with API testing and automation frameworks (e.g., Postman, Katalon, Playwright, or equivalent). Preferred skill sets: · Experience automating multi-agent validation and orchestration testing. · Experience with RAG validation, retrieval quality assessment, and grounding/citation validation. · Knowledge of AI governance, Responsible AI, and risk-based AI validation approaches. · Experience building automated evaluation pipelines and AI scoring frameworks. · Familiarity with token usage, model observability, and AI performance monitoring. Years of experience required: 4+yrs Education qualification: Bachelor's degree in Computer Science, IT, or a related field. Location Kolkata