Lead AI Automation Engineer (Next-Gen QA)

About CredServ

CredServ is building the

AI Operating Layer for Embedded Receivable Workflows

. We are transforming B2B trade by deploying specialized Agentic AI to automate the entire receivables lifecycle—from Partner Onboarding and Credit Underwriting to Voice Collections and Digital Legal recovery.

The Role: Lead AI Quality Engineer

You are not just testing AI; you are

using

AI to redefine how testing is done.

We are looking for a pioneering Lead QA Engineer who bridges the gap between deterministic software and non-deterministic AI Agents. You will use AI copilots to hyper-accelerate automation, migrate legacy suites to self-healing frameworks, and build the infrastructure that ensures our core services and autonomous agents operate flawlessly, safely, and within strict financial regulations.

What You Will Do (Core Responsibilities)

Next-Gen Automation & Core QA

AI-Assisted Scripting & Regression

Utilize AI copilot tools (e.g., Cursor, GitHub Copilot) to rapidly generate, maintain, and scale automated regression, integration, and functional test suites.

Modern Frameworks

Architect and scale automation using Playwright or Cypress, transitioning away from and migrating legacy Selenium suites to AI-native, self-healing frameworks.

Visual & API Testing

Implement Visual AI testing (e.g., Applitools) to catch UI anomalies across multiple devices. Own the end-to-end API testing strategy for our deterministic rule-based logic and core banking integrations.

Root Cause Analysis

Perform deep triage and root cause analysis of pipeline failures and flaky tests using AI-powered log analysis and observability tools.

Agentic AI Testing Strategy

Onboarding Agent

Test conversational KYC flows, document extraction (OCR/NLP) accuracy, and self-serve onboarding logic.

Underwriting Assistant

Validate the AI's ability to draft accurate credit summaries and reason codes for "Human-in-the-Loop" (HITL) limit reviews.

Voice Collection Bot

Measure voice latency (Time-to-First-Token), intent recognition, and compliance-grade empathy for recovery calls.

Digital Legal Module

Verify the automated triggering and tracking of real-time legal notices and case updates.

Non-Deterministic QA & Guardrails

LLM-as-a-Judge

Implement hallucination metrics, context precision (RAG testing), and semantic similarity checks to ensure AI outputs never breach regulatory guardrails.

Red Teaming & Security

Write adversarial test cases to evaluate prompt injection vulnerabilities and data leakage prevention.

Who You Are (Requirements)

Experience

• 5+ years
• of proven experience as an SDET or QA Automation Engineer, with
• at least 1+ years
• specifically testing LLMs, RAG systems, or deploying AI-assisted QA workflows.

Automation Mastery

Deep expertise in

Playwright

or Cypress. Strong understanding of core QA principles, including the Page Object Model (POM), exhaustive regression testing, and robust CI/CD integration (GitHub Actions/GitLab CI).

Coding Proficiency

Strong programming skills in

Python

• (for the AI evaluation stack) and/or
• Node.js/TypeScript
• (for UI/API automation).

AI QA Tooling

Hands-on experience with AI coding assistants (Cursor), Visual AI (Applitools), and LLM evaluation frameworks (e.g., LangSmith, DeepEval or Ragas).

Financial Precision

Deep understanding of testing "Human-in-the-Loop" workflows, complex reconciliation logic, and "exception-clearing" AI where accuracy is non-negotiable.

Why Join CredServ?

You will be defining how global supply chains and enterprises evaluate and trust Artificial Intelligence. If you want to work at the bleeding edge of Next-Gen Automation, Voice AI, and B2B trade, this is your place.

Competitive Salary & Equity

Flexible Work Options

Comprehensive Health & Wellness Benefits