Skip to content

AI Evaluation

Andromeda Shimu Deep Research Reviewer

Welo Data logoWelo Data$7/hrRemote in Mexico, PhilippinesContractPosted 1d ago

Its $7/hr is below the $70/hr median of the AI Evaluation & Rating (RLHF) roles open now, whose typical pay runs $35–$130/hr. How that pay moves week by week is in our research on AI Evaluation & Rating (RLHF) work.

Half of all AI Evaluation & Rating (RLHF) roles come down within 19 days of being listed, and 34.8% are still up after four weeks. This one was posted 1 day ago.

About the role

About the Role

We're looking for a QA Deep Research Reviewer to evaluate long-form, AI-generated research outputs against detailed rubrics and guidelines. You'll read documents that include tables and citations, judge each criterion on its own merits, and write clear, evidence-based rationales that help us improve the quality of AI research tools. This role suits someone with a sharp editorial eye, a fact-checker's instincts, and the discipline to apply guidelines consistently

What You'll Do

  • Review long form research responses covering US consumer topics such as personal finance and taxes, healthcare and insurance, home repair, pets, and travel planning

• Evaluate each response against rubric criteria, scoring every criterion independently so that an overall impression doesn't sway individual judgments

• Check claims, tables, and citations for accuracy, and distinguish primary sources from low-quality or content-farm pages

• Write short, specific rationales (1-3 sentences) that point to concrete evidence

• Follow guideline updates and calibrate your decisions consistently with the QA team.

Must-Have Qualifications

• Native or near-native US English (C2), with the ability to read long documents containing tables and citations

• Familiarity with US consumer topics: personal finance and taxes, healthcare and insurance, home repair, pets, and travel planning

• Bachelor's degree, or equivalent experience in research, editing, fact-checking, teaching, or QA

  • Proven experience with rubric- or guideline based evaluation (e.g., AI/LLM rating, search quality rating, content moderation, grading, or editorial QA)

• Ability to write concise rationales that cite evidence

• Ability to judge each criterion independently, without letting an overall impression influence the call

• Reliable availability of 15-20 hours per week, with a stable internet connection and a laptop or desktop computer.

Where you can work

This role is 100% remote, for applicants located in Mexico and Philippines.

From Welo Data's job posting, checked 18 Nov 2026.

The application

Your details and CV, then 1 question.

Have readyCVLinkedIn (optional)GitHub (optional)Portfolio (optional)Website (optional)
  1. 1.
    Which location are you applying for?Pick one

From Welo Data's apply page, checked 18 Nov 2026.

About Welo Data

Welocalize's AI-data community: annotation, evaluation, red-teaming, and multilingual work.

How it pays: Tasks go on purchase orders, by default one a week, that you invoice in Workday; Welo Data invoices are paid 14 days after receipt, into Hyperwallet. US raters are part-time W-2 employees (Welocalize talent help center, Payment Terms).

When it pays: 14 days after each invoice, through Hyperwallet. How often others pay

Interview: none to get in (Timed Welo Works assessments, then a client exam (per role)). Platforms with no interview

Similar open roles

Some links on AITraining.jobs are referral links. If you sign up through one, AITraining.jobs may earn a commission at no extra cost to you. It never changes what a platform pays you. A role at a platform that pays us a referral fee weighs more in this order.

Want companies to ask for you?

One profile for every AI training platform. Companies ask us for people with a specific language, country or credential: add yours once, and when a role fits, we introduce you, only with your yes. Free.

Create my profile

Get new Welo Data roles by email

We check the Welo Data job board every hour and email you its newest roles each week, plus similar paid roles elsewhere. Free, unsubscribe anytime.