Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting)

This position is listed on behalf of a partner company, who manages all applications and next steps. Our partner is looking for a Business Domain Expert — AI Model Evaluation Pilot (Admin, Marketing, HR, Accounting) based in Canada.

This is a short-term, structured opportunity for experienced business professionals to help evaluate the quality of next-generation AI <models.You> will complete one realistic professional task based on your domain expertise and then assess five AI-generated attempts at that same task.Your professional judgment will serve as the benchmark for evaluating accuracy, reasoning, clarity, and practical <relevance.You> will provide evidence-based scores and detailed rationales that can directly influence future AI model development.The study is designed for practitioners who understand how work is actually performed in real business <environments.It> offers a focused way to contribute your expertise to AI research without requiring a long-term commitment.The engagement is fully remote, flexible, and expected to require approximately 8–12 hours of your time.

Accountabilities

• Sign a confidentiality agreement before receiving project materials and ensure all study information is handled appropriately.
• Complete one realistic, domain-specific professional task using the source materials provided.
• Review five AI-generated responses to the same task and rank them from 1 to 5 with no ties.
• Assign each model response a score from 0–100 based on the quality of the work and your professional judgment.
• Write clear, specific, evidence-based rationales that reference relevant portions of each response to support your evaluation.
• Assess the accuracy, reasoning, practical applicability, and overall quality of the model-generated work within your professional domain.
• Provide structured feedback on the task itself and the overall study design to help improve future evaluation processes.
• Complete all work independently without using LLMs or other AI assistants during task completion, evaluation, scoring, or rationale writing.

Requirements

• 5+ years of hands-on professional experience in Human Resources or Accounting.
• Current or recent professional practice in your field, with sufficient experience to evaluate work according to real-world professional standards.
• Strong ability to assess professional work critically and distinguish high-quality reasoning from flawed, incomplete, or unsupported conclusions.
• Excellent written communication skills, with the ability to produce concise, specific, and evidence-grounded explanations for your judgments.
• Strong attention to detail, analytical thinking, and professional judgment.
• Ability to work reliably within a short turnaround period and complete a structured evaluation independently.
• Comfortable working without a predefined rubric or “golden answer,” using your own professional expertise as the primary basis for evaluation.
• Must be legally able to participate as an independent contractor in Canada.
• Must not have previously worked on Project Alchemy in any capacity or accessed its source-world data, as prior exposure would invalidate the study.
• Candidates requiring H-1B or STEM OPT sponsorship are not eligible for this engagement.

Benefits

• Fully remote, independent-contractor engagement that can be completed on your own schedule.
• Estimated commitment of 8–12 hours for the one-time evaluation study.
• Opportunity to contribute directly to the development and evaluation of advanced AI systems.
• Apply your professional expertise to realistic business scenarios without a long-term production commitment.
• Competitive payment for completed work, with weekly payments via Stripe or Wise.
• Flexible project structure; engagements may be extended, shortened, or concluded early depending on project needs and performance.
• No access to confidential or proprietary information belonging to your current or former employers, clients, or institutions.
• Opportunity to earn up to $400 per successful referral, subject to applicable restrictions.