Andromeda Shimu Deep Research Reviewer (EN-US)
Its $28/hr is below the $70/hr median of the AI Evaluation & Rating (RLHF) roles open now, whose typical pay runs $35–$130/hr. How that pay moves week by week is in our research on AI Evaluation & Rating (RLHF) work.
Half of all AI Evaluation & Rating (RLHF) roles come down within 19 days of being listed, and 34.8% are still up after four weeks. This one was posted 1 day ago.
About the role
About the Role
We're looking for a QA Deep Research Reviewer to evaluate long-form, AI-generated research outputs against detailed rubrics and guidelines. You'll read documents that include tables and citations, judge each criterion on its own merits, and write clear, evidence-based rationales that help us improve the quality of AI research tools. This role suits someone with a sharp editorial eye, a fact-checker's instincts, and the discipline to apply guidelines consistently
What You'll Do
- Review long form research responses covering US consumer topics such as personal finance and taxes, healthcare and insurance, home repair, pets, and travel planning
• Evaluate each response against rubric criteria, scoring every criterion independently so that an overall impression doesn't sway individual judgments
• Check claims, tables, and citations for accuracy, and distinguish primary sources from low-quality or content-farm pages
• Write short, specific rationales (1-3 sentences) that point to concrete evidence
• Follow guideline updates and calibrate your decisions consistently with the QA team.
Must-Have Qualifications
• Native or near-native US English (C2), with the ability to read long documents containing tables and citations
• Familiarity with US consumer topics: personal finance and taxes, healthcare and insurance, home repair, pets, and travel planning
• Bachelor's degree, or equivalent experience in research, editing, fact-checking, teaching, or QA
- Proven experience with rubric- or guideline based evaluation (e.g., AI/LLM rating, search quality rating, content moderation, grading, or editorial QA)
• Ability to write concise rationales that cite evidence
• Ability to judge each criterion independently, without letting an overall impression influence the call
• Reliable availability of 15-20 hours per week, with a stable internet connection and a laptop or desktop computer.
Where you can work
This role is 100% remote, for applicants located in United States.
From Welo Data's job posting, checked 18 Nov 2026.
The application
Just your details and CV.
From Welo Data's apply page, checked 18 Nov 2026.
About Welo Data
Welocalize's AI-data community: annotation, evaluation, red-teaming, and multilingual work.
How it pays: Tasks go on purchase orders, by default one a week, that you invoice in Workday; Welo Data invoices are paid 14 days after receipt, into Hyperwallet. US raters are part-time W-2 employees (Welocalize talent help center, Payment Terms).
When it pays: 14 days after each invoice, through Hyperwallet. How often others pay
Interview: none to get in (Timed Welo Works assessments, then a client exam (per role)). Platforms with no interview
Similar open roles
Some links on AITraining.jobs are referral links. If you sign up through one, AITraining.jobs may earn a commission at no extra cost to you. It never changes what a platform pays you. A role at a platform that pays us a referral fee weighs more in this order.

Prompt & Response Evaluator (opens the job details)
micro1 · AI Evaluation · Remote· 5d agoDetails
$40–$70/hr
Document Annotation Specialist (PDF) (opens the job details)
SuperAnnotate (SME) · Data Annotation · Remote· 1w agoDetails
up to $10/hr
General AfterQuery Expert (Data Annotator) (opens the job details)
AfterQuery · Data Annotation · Remote· 1y+ agoDetails
$43/hr
Postal Clerk AI Response Evaluator (opens the job details)
OpenTrain AI · Domain Experts · Remote· 1mo agoDetails
$30–$45/hr
Want companies to ask for you?
One profile for every AI training platform. Companies ask us for people with a specific language, country or credential: add yours once, and when a role fits, we introduce you, only with your yes. Free.
Get new Welo Data roles by email
We check the Welo Data job board every hour and email you its newest roles each week, plus similar paid roles elsewhere. Free, unsubscribe anytime.