AI Data Quality Analyst

About TaskUs: TaskUs is a provider of outsourced digital services and next-generation customer experience to fast-growing technology companies, helping its clients represent, protect and grow their brands. Leveraging a cloud-based infrastructure, TaskUs serves clients in the fastest-growing sectors, including social media, e-commerce, gaming, streaming media, food delivery, ride-sharing, HiTech, FinTech, and HealthTech. The People First culture at TaskUs has enabled the company to expand its workforce to approximately 45,000 employees globally. Presently, we have a presence in twenty-three locations across twelve countries, which include the Philippines, India, and the United States. It started with one ridiculously good idea to create a different breed of Business Processing Outsourcing (BPO)! We at TaskUs understand that achieving growth for our partners requires a culture of constant motion, exploring new technologies, being ready to handle any challenge at a moment’s notice, and mastering consistency in an ever-changing world. What We Offer: At TaskUs, we prioritize our employees' well-being by offering competitive industry salaries and comprehensive benefits packages. Our commitment to a People First culture is reflected in the various departments we have established, including Total Rewards, Wellness, HR, and Diversity. We take pride in our inclusive environment and positive impact on the community. Moreover, we actively encourage internal mobility and professional growth at all stages of an employee's career within TaskUs. Join our team today and experience firsthand our dedication to supporting People First. AI Data Quality Analyst Why this role exists: As AI-driven solutions expand across industries, ensuring the fidelity of training and evaluation datasets is essential for building reliable models. TaskUs needs a diligent, detail-focused Data Quality Analyst who can maintain annotation standards and drive continuous improvements so that essential AI data remains accurate, efficient, and scalable. Data Analysis Quality Audits - Perform quality audits on annotated datasets to ensure that they meet established guidelines and quality benchmarks. Statistical Reporting - Leverage statistical based quality metrics such as F1 score and inter-annotator agreement to evaluate data quality. Root Cause Analysis - Analyze annotation errors, trends, project processes, and project documentation to identify and understand the root cause of errors and propose remediation strategies. Edge-Case Management - Resolve and analyze edge-case annotations to ensure quality and identify areas for improvement. Tooling - Become proficient in using annotation and quality control tools to perform reviews and track quality metrics. Guidelines - Become an expert in the project specific guidelines and provide feedback for potential clarifications or improvements. Continuous Improvement Automation - Identify opportunities to use automation to help enhance analytics, provide deeper insights, and improve efficiency. Documentation - Develop and maintain up-to-date documentation on quality standards, annotation guidelines, and quality control procedures. Feedback - Provide regular feedback that identifies areas for improvement across the annotation pipeline. Collaboration & Communication Cross-Functional Teamwork - Work closely with key project stakeholders and clients to understand project requirements and improve annotation pipelines. Training - Assist with training annotators, providing guidance, feedback, and support to ensure data quality. Reporting - Provide regular updates that highlight data quality metrics, key findings, and actionable insights for continuous process improvements. Experiences you’ll bring: 1+ years of experience as a data analyst with exposure to data quality and/or data annotation - ideally within an AI/ML context. Familiarity with the basic concepts of AI/ML pipelines and data. Core skills you'll need: Strong analytical and problem-solving skills with an exceptional eye for detail. Excellent written and verbal communication skills, with the ability to clearly articulate quality issues and collaborate with diverse teams. Ability to work independently and manage time effectively to meet deadlines. A strong problem-solver who thinks critically and drives innovation and continuous optimization. A quick learner with the ability to work independently in a fast-paced environment. A strong focus on detail, balanced against strategic priorities. A positive can-do attitude and the ability to easily adapt to new environments. Not afraid to speak up. Nice to have: Familiarity with data annotation tools (e.g. Labelbox, Dataloop, LabelStudio etc.). Experience working with multi-modal AI/ML datasets (images, videos, text, audio). Prior experience in an agile or fast-paced tech environment with exposure to AI/ML pipelines. Knowledge of programming languages (e.g. Python). Knowledge of the concepts and principles of data quality for AI/ML models and the impacts it can have on model performance. Working understanding of common quality metrics and statistical methods used in AI/ML data quality. Knowledge of AI/ML concepts and experience with data for AI/ML models. Experience in prompt engineering and leveraging LLMs in your day-to-day work. Why you’ll love this role: Work remote: Work from anywhere you’re most productive! Every dataset is unique: Audit, analyze, and improve complex AI/ML datasets, no two projects are ever the same. Visible impact: Your quality checks, root cause analyses, and automation ideas will directly shape how AI models perform in real-world applications. Room to grow: Build deep expertise in AI data quality and scale your influence as our projects and capabilities expand. Mission-driven team: Work alongside analysts, engineers, and stakeholders who share a passion for creating accurate, reliable, and trustworthy AI. The impact you’ll make: Guard data integrity through audits: Conduct quality audits on annotated datasets against established guidelines and statistical benchmarks (e.g., F1 score, inter‑annotator agreement) to uphold data reliability. Pinpoint and resolve annotation issues: Analyse error patterns, edge-case annotations, and root causes to surface insights and elevate annotation quality. Optimise annotation processes: Become proficient in annotation and QA tools, propose automation, and maintain clear documentation of quality standards, guidelines, and procedures. Collaborate and coach: Work closely with stakeholders and annotators to clarify requirements, provide constructive feedback, and support skill development within the annotation pipeline. Track and communicate quality performance: Regularly report on data quality metrics, key findings, and actionable insights to drive continuous improvements across projects. How We Partner To Protect You: TaskUs will neither solicit money from you during your application process nor require any form of payment in order to proceed with your application. Kindly ensure that you are always in communication with only authorized recruiters of TaskUs. DEI: In TaskUs we believe that innovation and higher performance are brought by people from all walks of life. We welcome applicants of different backgrounds, demographics, and circumstances. Inclusive and equitable practices are our responsibility as a business. TaskUs is committed to providing equal access to opportunities. If you need reasonable accommodations in any part of the hiring process, please let us know. We invite you to explore all TaskUs career opportunities and apply through the provided URL https://www.taskus.com/careers/. Click here to learn more about our #RidiculouslyGood Culture & life at TaskUs. At TaskUs, we put our frontline first, period. We have state-of-the-art workspaces, top-notch benefits, and some of the best teammates in the world. No matter where you are around the globe, we invest in you and your career growth. Join Us! Let's change the way the world works. TaskUs is looking for ridiculously good talent, and we are excited that you are considering TaskUs for your next career step. In doing so, we take your security and safety very seriously to ensure as part of the hiring process. Here are a few things that are important to know about our company and recruiting process: You will never be asked for payment information or banking information during the application or interview process. We WILL NOT ask you for your national ID number (Social Security or Government ID) or date of birth when applying for a position with us Please be aware that scammers may try to take advantage of you by impersonating TaskUs or TaskUs employees in order to access money or personal information and offer false employment opportunities in our name. We have identified several clues to watch out for to help you identify and avoid recruiting scams: TaskUs will never ask for financial information of any kind or for payment of money during the job application process.We do not require any financial, credit card or bank account information and/or any payment of any kind in order to be considered for employment. If you are asked to pay for things like equipment fees, travel fees, or some other deposit or application-processing fee, even if claimed you will be reimbursed, this is not Us. It is a scam. TaskUs does not require personal information including Social Security Number, National ID, Government ID or date of birth as part of the application process. We may ask for basic contact information such as your email address, phone number, and/or home address to ensure we can contact you. If you were contacted by a third-party organization claiming to represent TaskUs or hiring on behalf of TaskUs, or if you were contacted in a chat room such as Google Hangouts, Freelancer.com or Telegram app, you should be careful in sharing any information. TaskUs official chat room is located on our Careers website (taskus.wd1.myworkdayjobs.com/Careers). This is the only chat room where you will find our recruiters.