AI Analyst

Job Summary

The ERDM Sr. AI Operations Analyst is responsible for ensuring that Sales & Finance AI solutions operate with the highest levels of trust, accuracy, control, and compliance.We are seeking a detail-obsessed, data driven expert to serve as the guardian of accuracy for our Sales & Finance AI chat assistants. You will design validation frameworks, partner with data engineering teams, and ensure every metric the AI surfaces - especially Bookings and Revenue - is accurate, consistent, and trustworthy. You will lead incident response on any data accuracy or hallucination evenThe role also serves as the bridge between Finance users and technical development teams. This role is responsible for translating business requirements into AI-enabled solutions and ensuring that Finance AI capabilities align with business goals and leadership needs.Core Responsibilities:

Requirements & Solution DefinitionGather and document business requirements from Finance users and stakeholders through interviews, workshops, observation, and feedback analysisWrite clear, detailed user stories with well-defined acceptance criteria that IT can act on without ambiguityEnsure requirements reflect metric integrity standards and executive-readiness expectations AI Quality AssuranceDevelop and maintain AI-specific testing frameworks - define test scenarios, expected outputs, edge cases, and pass/fail criteria tailored to conversational AI behaviorEstablish and document quality standards for AI-generated responses - accuracy, completeness, formatting, tone, and contextual relevanceValidate AI-generated responses against Snowflake source data using SQL queries to confirm metric accuracy and consistencyConduct thorough UAT, regression testing, and release testing for every new feature, enhancement, prompt update, data refresh, or model change before go-liveConfirm metric integrity - verify that Bookings and Revenue figures surfaced by the AI match official Finance reportingCoordinate business sign-off activities - manage the approval workflow with Finance stakeholders before releases are promoted to production Hallucination PreventionDesign and execute validation test strategies specifically targeting hallucination risks - test for fabricated data, incorrect metric associations, misleading trends, and unsupported narrativesIdentify and catalog high-risk Finance use cases where hallucination potential is elevated (e.g., multi-step calculations, year-over-year comparisons, segment-level breakdowns, forecast vs. actuals)Monitor response quality on an ongoing basis - conduct periodic spot-checks and audits of AI outputs across different query types and user scenariosEscalate and resolve quality concerns immediately - flag suspected hallucinations to the Data Accuracy Specialist (IC4) and team lead; coordinate with IT/Engineering to implement fixes and guardrailsCore Responsibilities

Executive Use CasesSupport in designing and testing executive-focused use cases - ensure AI responses meet the precision, tone, and contextual depth expected by C-suite and SVP usersTest and validate executive prompt frameworks - verify that curated queries, guided flows, and personalized dashboards produce accurate, polished outputsContribute to improving usability and decision-support capabilities - identify opportunities to make the executive experience more intuitive, actionable, and insightful Adoption & Change ManagementDevelop and execute adoption strategies - create structured plans to drive Finance user engagement, from initial awareness through active daily useLead user onboarding - design and deliver onboarding experiences for new users, including guided walkthroughs, sandbox environments, and office hoursDeliver training programs - conduct live demos, workshops, and drop-in sessions tailored to different user groups (analysts, managers, directors)Manage communications - draft and distribute release notes, feature announcements, tips & tricks, known issue updates, and adoption progress reports to Finance stakeholdersCollect and synthesize user feedback through surveys, interviews, and usage analytics to continuously improve the experienceRequired Qualifications:

Bachelor's degree Required7+ years in Business Analysis / Finance Analytics / FP&A OperationsExperience with AI-powered tools, chatbots, or conversational AI interfacesUnderstanding of Finance metrics and reporting (Bookings, Revenue, or related areas)Working knowledge of SQL and SnowflakeUnderstanding of AI/LLM fundamentals - how large language models retrieve, process, and present information; awareness of common failure modes (hallucination, data staleness, context window limitations)Prompt Engineering Certificate or related experiencePreferred Qualifications:

Familiarity with Abacus or similar platformsHands-on experience with prompt engineering — designing, testing, and refining prompts to improve AI response quality and accuracyExperience creating training materials, user guides, or knowledge base contentExperience with AI testing frameworks or quality assurance for data-driven applicationsFamiliarity with AI governance or responsible AI principles