Skip to content

Research report ยท October 2026 edition

The Market for AI Evaluation & Rating (RLHF) Work

On 31 Oct 2026, 207 roles in AI Evaluation & Rating (RLHF) were open on 33 platforms (Outlier (Scale AI) listed 44 of them (21.3%)); their median advertised rate was $35/hr, 13% below the $40/hr of all AI training work; each open role drew 1.23x the apply clicks of the average role.

Read the month's story: The AI Training Job Market in October 2026

Provisional figures. This month is complete, but late data can still arrive until 3 Nov 2026, when the edition is frozen for good.

Data window
1 Oct 2026 to 31 Oct 2026
31 days, UTC
Computed
1 Nov 2026, 01:17 UTC
Final once frozen
Source
The public job boards of the AI training platforms this site tracks, read every hour. Apply clicks are made on AITraining.jobs; full descriptions come from the platforms whose boards publish them.
Method
Version 5. Definitions
CC BY 4.0, how to cite

Every figure on this page as CSV and JSON, sent to your work email.

Key findings

  1. 1

    Between the week of 5 Oct 2026 and the week of 19 Oct 2026, open roles in AI Evaluation & Rating (RLHF) on the platforms tracked since 1 Oct 2026 went down 24.2%, from about 131 to 100, like for like across the changes in how this site reads some of their boards, against -3.1% for all AI training work on the same platforms; platforms first tracked later, left out of this panel, listed about 93 more in the week of 19 Oct 2026.

    Figure 1
  2. 2

    33 platforms listed the 207 open roles in AI Evaluation & Rating (RLHF); Outlier (Scale AI) held 44 of them (21.3%) and the three largest 89 (43.0%) (HHI 904, unconcentrated, against 522 for all AI training work).

    Figure 3
  3. 3

    133 of the 207 open roles in AI Evaluation & Rating (RLHF) (64.3%) stated an hourly rate, with a median of $35/hr (typical pay $20 to $120), 13% below the $40/hr of all AI training work.

    Figure 4
  4. 4

    Each open role in AI Evaluation & Rating (RLHF) drew 4.21 apply clicks a week on AITraining.jobs, 1.23x the average AI training role: the 3rd highest of 12 kinds of work whose demand reads on its own.

    Figure 6
  5. 5

    23 of the 207 open roles in AI Evaluation & Rating (RLHF) (11.1%) accepted applicants from any country, against 18.5% of all AI training roles; of the 20 countries compared, residents of the United States (171), India (82) and the United Kingdom (75) could apply to the most, and residents of Palestine to the fewest (54).

    Figure 9
  6. 6

    The words used most in the titles of open roles in AI Evaluation & Rating (RLHF) were Evaluator or reviewer (54), Lead or manager (32) and Quality assurance (29), of 207 open roles.

    Figure 10
  7. 7

    Of the 198 open roles in AI Evaluation & Rating (RLHF) with a full description, 46 (23.2%) mention a PhD or master's degree, 5 (2.5%) a professional credential and 77 (38.9%) an assessment or test; the 57 stating years of experience ask for a median of 3 years.

    Figure 11
  8. 8

    Over the 31 days from 1 Oct 2026 to 31 Oct 2026, open roles in AI Evaluation & Rating (RLHF) on the platforms tracked at its start went down 43.5% (178 to 101, like for like across the changes in how this site reads some of their boards): platforms listed 110 new roles and removed 228, and a removed role had been listed for a median of 30 days (30 across all AI training work).

    Figure 2

Figure 1

Open roles, week by week

Is the number of open roles in AI Evaluation & Rating (RLHF) rising or falling?

WeekOpen roles (fixed panel)Index
1 to 4 OctPartial week
164
124.8
5 to 11 Oct
131
100.0
12 to 18 Oct
96
75.6
19 to 25 Oct
93
75.8
26 to 31 OctPartial week
90
67.0

Sample. Average open roles in AI Evaluation & Rating (RLHF) over each week's days, on the 18 platforms tracked since 1 Oct 2026 (the fixed panel; 16 of them list this work), 1 Oct 2026 to 31 Oct 2026.

The index is the panel's open roles against the week of 5 Oct 2026 (100), chained week to week, beside the same index for all AI training work on the same platforms.

This site changed how it reads 6 platforms' boards in this span (Alignerr on 14 Oct 2026; Mindrift on 14 Oct 2026 and 22 Oct 2026; OpenTrain AI on 14 Oct 2026; Handshake AI on 14 Oct 2026; Prolific on 18 Oct 2026; Outlier (Scale AI) on 30 Oct 2026). From each change to the end of its week the roles that came into view are counted as already listed, not as new listings, and the roles that left view are not counted as removals; changes in open roles are measured like for like across it, on the roles that did not move with it. Each platform's own report says what changed.

Perle Labs (first tracked 10 Oct 2026), SuperAnnotate (SME) (first tracked 10 Oct 2026), Turing (first tracked 10 Oct 2026), RemoExperts (first tracked 10 Oct 2026), Sovrano AI (first tracked 11 Oct 2026), Anyone AI (first tracked 11 Oct 2026), Vetto (first tracked 14 Oct 2026), OneForma (first tracked 14 Oct 2026), Claru (first tracked 14 Oct 2026), Weekday (first tracked 18 Oct 2026), Innodata (first tracked 18 Oct 2026), Rise Data Labs (first tracked 18 Oct 2026), Lifted (Upwork) (first tracked 18 Oct 2026), Scale AI Human Frontier Collective (first tracked 18 Oct 2026), Anthropic (first tracked 21 Oct 2026), DataAnnotation (first tracked 22 Oct 2026), TELUS Digital AI (first tracked 22 Oct 2026), iMerit (first tracked 22 Oct 2026), RWS TrainAI Community (first tracked 22 Oct 2026), EvolutionIQ (first tracked 29 Oct 2026) and JazzX AI (first tracked 29 Oct 2026) list this work but were added to the dataset later, so they are left out of the panel and counted only in the last column: a platform's arrival never reads as growth.

Weeks cut by the span, or with fewer than 14.4 open roles on the panel (the weight of the rest of AI training work's change, fitted on the two: below it the market's movement says more than the kind's own roles), are marked.

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, October 2026 edition.

Figure 2

Listings added and removed

How fast do roles in AI Evaluation & Rating (RLHF) turn over, and is the stock growing on the same platforms?

WorkOpen on 1 Oct 2026Open on 31 Oct 2026, same platformsChangeNew listings
AI Evaluation & Rating (RLHF)178101
-43.5%
110
All AI training work2,6652,554
-4.2%
1,638

Sample. Roles listed from 1 Oct 2026 to 31 Oct 2026: 482 in AI Evaluation & Rating (RLHF), 7,396 across all AI training work.

Change compares the roles open on the first and the last day of the window on the platforms tracked when it opened, so a platform added during the window does not read as growth.

The kind's change reads on its own (not marked) from 9.8 roles open at the start: the weight of the rest of AI training work's change, fitted on the two.

A new listing first appeared in the window on a platform already tracked before that week; a platform's first week (its back catalogue) is not counted.

Median days listed runs from a removed role's listing date to its last sighting, for removed roles with a known listing date (at least 3; 223 here).

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, October 2026 edition.

Figure 3

Who lists it

Which platforms list roles in AI Evaluation & Rating (RLHF), and what does each advertise?

PlatformOpen rolesShare of open rolesMedian per hour
Outlier (Scale AI)44
21.3%
n/a
SuperAnnotate (SME)25
12.1%
n/a
micro120
9.7%
$50
Turing17
8.2%
n/a
Mercor10
4.8%
$60
TELUS Digital AI10
4.8%
$35
Alignerr8
3.9%
$80
Invisible8
3.9%
$35
Welo Data8
3.9%
$20
iMerit7
3.4%
$25
Prolific7
3.4%
$25
Weekday6
2.9%
$30
Braintrust5
2.4%
$25
Innodata5
2.4%
$15
Handshake AI4
1.9%
n/a
AfterQuery2
1.0%
n/a
Claru2
1.0%
n/a
OpenTrain AI2
1.0%
n/a
RemoExperts2
1.0%
n/a
RWS TrainAI Community2
1.0%
n/a
Anyone AI1
0.5%
n/a
Appen1
0.5%
n/a
DataAnnotation1
0.5%
n/a
EvolutionIQ1
0.5%
n/a
JazzX AI1
0.5%
n/a
Lifted (Upwork)1
0.5%
n/a
OneForma1
0.5%
n/a
Perle Labs1
0.5%
n/a
Rise Data Labs1
0.5%
n/a
Scale AI Human Frontier Collective1
0.5%
n/a
Terac1
0.5%
n/a
Vetto1
0.5%
n/a
xAI (SpaceXAI)1
0.5%
n/a

Sample. 207 roles in AI Evaluation & Rating (RLHF) open at the end of 31 Oct 2026, by the platform that lists them: HHI 904 (unconcentrated), against 522 for all AI training work.

HHI: the sum of every platform's squared percentage share of the open roles, 0 to 10,000; under 1,500 is unconcentrated, 1,500 to 2,500 moderately and above 2,500 highly concentrated (US merger-guideline thresholds).

Shares are of listings, not of people hired or money paid: a platform posting one role per language or country weighs more here than its hiring may.

A median needs at least 3 roles stating an hourly rate.

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, October 2026 edition.

Figure 4

What it pays

What do roles in AI Evaluation & Rating (RLHF) advertise per hour, against AI training work as a whole?

WorkMedian per hourTypical pay fromTypical pay toStating an hourly rate
AI Evaluation & Rating (RLHF)
$35
$20$12064.3%
All AI training work
$40
$30$12581.4%

Sample. Roles open on 31 Oct 2026: 207 in AI Evaluation & Rating (RLHF), 133 of them stating an hourly rate; 4,381 across all AI training work.

Pay is the hourly figure a posting advertises (the middle of a stated range), counted once per distinct rate as the job board counts it. The median and typical pay are the board's: typical pay runs from the 25th percentile of what roles pay (the middle of each advertised range) to the 90th-percentile advertised ceiling, stated from 3 distinct rates.

Roles paid per task or per project, or that state no figure, do not state an hourly rate.

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, October 2026 edition.

Figure 5

How its rates are distributed

What share of roles in AI Evaluation & Rating (RLHF) pay $50 an hour or more?

Advertised rateOpen rolesShare of priced rolesAll AI training work
Under $15/hr8
6.0%
6.9%
$15 to $25/hr26
19.5%
10.6%
$25 to $35/hr21
15.8%
11.9%
$35 to $50/hr19
14.3%
26.0%
$50 to $75/hr35
26.3%
14.7%
$75 to $100/hr15
11.3%
15.3%
$100 to $150/hr6
4.5%
9.4%
$150/hr or more3
2.3%
5.1%

Sample. 133 roles in AI Evaluation & Rating (RLHF) open on 31 Oct 2026 that state an hourly rate, against 3,565 across all AI training work.

Each role counts once, at the middle of its advertised range. The bands are those of the pay-transparency report.

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, October 2026 edition.

Figure 6

Applicant demand per open role

Do roles in AI Evaluation & Rating (RLHF) draw more or fewer applicants per opening than the average AI training role?

RolesClicks per role a weekIndex
AI Evaluation & Rating (RLHF)
4.21
1.23x
All AI training work
3.41
1.00x
Its roles stating an hourly rate
4.34
1.27x
Its roles stating no hourly rate
3.69
1.08x
Its roles open worldwide
6.90
2.02x
Its roles limited by country
3.49
1.02x

Sample. Apply clicks on a specific role on AITraining.jobs, from 1 Oct 2026 to 31 Oct 2026: 3,018 on roles in AI Evaluation & Rating (RLHF), against the days those roles were open.

Clicks per role a week = apply clicks / (open role-days / 7). Index 1.00 is the rate of every role on the site.

AI Evaluation & Rating (RLHF) against every kind of work: Rows with fewer than 3.6 open role-days (0.5 role-weeks) are marked as a small sample and not ranked: below that the pooled rate of the kinds of work weighs more than a row's own clicks (the weight fitted on their own clicks).

Its roles by stated pay, against each other: Rows with fewer than 254.1 open role-days (36.3 role-weeks) are marked as a small sample and not ranked: below that the pooled rate of the two sides weighs more than a row's own clicks (the weight fitted on their own clicks). Its roles by reach, against each other: Rows with fewer than 10.7 open role-days (1.5 role-weeks) are marked as a small sample and not ranked: below that the pooled rate of the two sides weighs more than a row's own clicks (the weight fitted on their own clicks). All AI training work is one set, read from its first open role-day.

Apply clicks measure interest among visitors to AITraining.jobs, who see roles open in their country and recent, priced roles first. Clicks are never split by platform.

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, October 2026 edition.

Figure 7

Applicant demand by listing age

Do newer roles in AI Evaluation & Rating (RLHF) draw more applicants than older ones?

Listing ageClicks per role a weekIndex (this work = 1)
Listed during the window
7.59
1.80x
Listed up to 4 weeks before it
2.78
0.66x
Listed 4 to 12 weeks before it
0.39
0.09x
Listed more than 12 weeks before it
8.27
1.96x
Listing date unknown
1.05
0.25x

Sample. Apply clicks on roles in AI Evaluation & Rating (RLHF), from 1 Oct 2026 to 31 Oct 2026, by when each role was listed relative to the window's first day (1 Oct 2026).

Listing date: the date the job board gives for the posting, or else the first day the role appeared in the feed. A role that arrived with a platform's back catalogue and gives no date has none.

The index divides each row's rate by the rate of all roles in this kind of work, so it reads within the kind.

Rows with fewer than 1.8 open role-days (0.3 role-weeks) are marked as a small sample and not ranked: below that the pooled rate of the listing ages weighs more than a row's own clicks (the weight fitted on their own clicks).

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, October 2026 edition.

Figure 8

Who may apply: location rules

Can people apply to roles in AI Evaluation & Rating (RLHF) from anywhere?

Location ruleOpen rolesShare of its open rolesAll AI training work
Open worldwide23
11.1%
18.5%
Worldwide except listed countries43
20.8%
18.1%
Only listed countries141
68.1%
63.0%
No rule stated0
0.0%
0.4%

Sample. 207 roles in AI Evaluation & Rating (RLHF) open at the end of 31 Oct 2026, against 4,381 across all AI training work.

The rule is read from each posting's location text, then, where that names no country, from what the posting's own description states, and where both are silent from what the platform states about where it hires.

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, October 2026 edition.

Figure 9

Who may apply: roles open to each country

How many roles in AI Evaluation & Rating (RLHF) can a resident of each country apply to?

CountryOpen rolesShare of its open roles
Any country (worldwide roles only)
23
11.1%
United States
171
82.6%
India
82
39.6%
United Kingdom
75
36.2%
Germany
72
34.8%
Brazil
70
33.8%
Egypt
68
32.9%
Indonesia
68
32.9%
Japan
68
32.9%
Chile
67
32.4%
Saudi Arabia
67
32.4%
Argentina
65
31.4%
Bangladesh
65
31.4%
South Korea
65
31.4%
Colombia
64
30.9%
Philippines
64
30.9%
Ghana
63
30.4%
Nigeria
63
30.4%
Pakistan
58
28.0%
Cambodia
55
26.6%
Palestine
54
26.1%

Sample. 207 roles in AI Evaluation & Rating (RLHF) open at the end of 31 Oct 2026, for residents of a fixed set of 20 countries.

A country's count is the roles open worldwide (including roles that state no rule, as on the job board), plus roles whose list names it, plus roles whose exclusion list does not.

The 20 countries are the same in every report of this series: those named most often by the location rules of open AI training roles on 9 Oct 2026, either as the only countries accepted or as excluded.

Median pay is over the roles open to that country that state an hourly rate (at least 3).

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, October 2026 edition.

Figure 10

What the roles are called

What are roles in AI Evaluation & Rating (RLHF) called: annotator, evaluator, expert or trainer?

Title wordOpen rolesShare of its open roles
Evaluator or reviewer
54
26.1%
Lead or manager
32
15.5%
Quality assurance
29
14.0%
Rater
24
11.6%
Specialist
24
11.6%
Analyst or researcher
22
10.6%
Trainer
22
10.6%
Expert or SME
16
7.7%
Voice actor or speaker
10
4.8%
Engineer or developer
6
2.9%
Annotator or labeler
5
2.4%
Red teamer or tester
4
1.9%
Writer or editor
4
1.9%
Contributor or participant
1
0.5%
Tutor
1
0.5%

Sample. 181 of the 207 roles in AI Evaluation & Rating (RLHF) open on 31 Oct 2026 name one of these words in their title (87.4%); 15 title words are named.

A title can use several of these words ('Expert AI Trainer') and counts for each. The vocabularies are those of the job-titles, languages and expert-fields reports.

Median pay needs at least 3 roles stating an hourly rate.

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, October 2026 edition.

Figure 11

What the descriptions ask for

Do roles in AI Evaluation & Rating (RLHF) ask for degrees, credentials, experience or tests?

RequirementRolesShare of described roles
Mentions a PhD26
13.1%
Mentions a PhD or master's46
23.2%
Mentions any degree101
51.0%
Mentions a professional credential5
2.5%
States years of experience57
28.8%
Asks for 5 years or more16
8.1%
States weekly hours74
37.4%
Mentions an assessment or test77
38.9%
Mentions an AI-led or video interview28
14.1%
Mentions a background check1
0.5%

Sample. 198 roles in AI Evaluation & Rating (RLHF) open on 31 Oct 2026 whose job board publishes a full description (AfterQuery, Anyone AI, Appen, Braintrust, Claru, DataAnnotation, EvolutionIQ, Handshake AI, iMerit, Innodata, Invisible, JazzX AI, Lifted (Upwork), Mercor, micro1, OneForma, OpenTrain AI, Outlier (Scale AI), Perle Labs, Prolific, RemoExperts, Rise Data Labs, RWS TrainAI Community, Scale AI Human Frontier Collective, SuperAnnotate (SME), TELUS Digital AI, Terac, Turing, Vetto, Weekday, Welo Data and xAI (SpaceXAI)), 95.7% of its open roles; 4,000 described roles across all AI training work.

A mention can be a requirement or a preference ('a PhD is a plus'): the figure counts both. The patterns are those of the requirements report.

Only some platforms publish full descriptions, so this figure describes their roles, not every platform's.

A kind of work's figure is shown when its described roles outweigh the average of the kinds (5.8 described roles: the weight of every kind's shares around the shares of all described roles, fitted on their own counts).

Source. AITraining.jobs Research, The Market for AI Evaluation & Rating (RLHF) Work, October 2026 edition.

For labs, data vendors and researchers

Buy the data behind this report

The figures above are a summary. Behind them are records we collect ourselves, every day:

  • 11,482 roles tracked, each with its history

    Every role on 62 job boards, read since 13 Jul 2026 (every hour since 18 Oct 2026): its pay and place, and the first and last day it was open.

  • 58,632 apply clicks in 28 days

    Which roles people actually apply to, by field, pay and kind of work. It happens on this site, so nobody else can count it.

  • 39 lab and vendor hiring boards

    The job boards of 25 AI labs and 14 human-data vendors, read every day: who is hiring for human data, and who is hiring to buy it.

What you can buy

  • The records

    The rows behind any figure, with their history, not only the totals.

  • A regular feed

    The same figures every week or month, as files.

  • A custom cut or benchmark

    Your countries, languages or fields, or your own platform against the market.

Building AI and need the experts behind this data? (you contract them yourself) or (we deliver the expert work).

Investing in AI-data companies? Hiring radar is these records role by role, every day, as files.

Method and definitions

The public job boards of the AI training platforms this site tracks, read every hour. Apply clicks are made on AITraining.jobs; full descriptions come from the platforms whose boards publish them. Every figure is computed by one database query over the window shown, the same query for the live report and for every edition. Each figure states its sample beneath it.

Kind of work
Each role is classified from its title and description into one of the twelve kinds of AI training work this site tracks; this report counts the roles classified as AI Evaluation & Rating (RLHF).
Fixed panel
The platforms already tracked when the weekly series opens. The index follows their open roles, so a platform added later (such as OpenTrain AI and Welo Data in September 2026) never reads as growth.
Index
The panel's average open roles in a week, divided by the same in the base week, times 100; shown beside the same index for all AI training work on the same platforms.
Open role
A role listed by a given day and still in its platform's feed that day.
New listing
A role that first appeared during the window on a platform already tracked before that week. A platform's first week in this dataset is its back catalogue and is not counted as new. When this site changes how it reads a platform's board (a cap lifted, a page kind or department read too, a relevance rule changed), the roles that came into view or left it from that change to the end of its week are this site's reading, not the platform's hiring: they are not new listings or removals, and changes in open roles are measured like for like across it. Each platform's report says what changed.
Removed listing
A role that left its platform's feed during the window.
Days listed
From a removed role's listing date to the last day it was seen in its feed. A median needs at least 3 removed roles.
HHI
Herfindahl-Hirschman Index: the sum of every platform's squared percentage share of open roles, 0 to 10,000. Under 1,500 unconcentrated, 1,500 to 2,500 moderately, above 2,500 highly concentrated.
Advertised pay
The hourly figure a posting states (the middle of a stated range), counted once per distinct rate as the job board counts it. The median and typical pay are the board's: typical pay runs from the 25th percentile of what roles pay (the middle of each advertised range) to the 90th-percentile advertised ceiling, stated from 3 distinct rates.
Clicks per open role a week
Apply clicks on AITraining.jobs on a specific role, divided by the days the roles were open, times 7.
Demand index
Clicks per open role a week divided by the same rate across every role on the site. Above 1, the roles draw more applicants than the average role.
Listing age
When a role was listed relative to the window's first day: during the window, up to 4 weeks before it, 4 to 12 weeks before or longer. The listing date is the job board's posting date, or else the first day the role appeared in the feed.
Location rule
Read from each posting's location text and, where that names no country, from what the posting's own description states: open worldwide, worldwide except listed countries, only listed countries, or no rule stated.
Open to a country
A role is open to residents of a country if it is open worldwide, its list names the country, or its exclusion list does not. The 20 countries compared are the same in every report of the series.
Window
Live: the last 28 complete days, in UTC, and the last twelve complete ISO weeks (from the week of 3 Aug 2026 at the earliest) for the weekly series. Edition: one calendar month, in UTC, and its weeks.

What this data cannot tell you

  • Platforms that recruit without a public job board are not counted. Several large ones invite workers to projects directly.
  • A listing is not a hire. Some platforms keep evergreen roles open for months, and some post one listing per language or country.
  • Apply clicks measure interest among the people who use AITraining.jobs, and what is shown first gets clicked more. Clicks are never published by platform.
  • Full descriptions are published by a few platforms only, so the requirements figure describes their roles and appears only when this kind's described roles read on their own against the other kinds' (the weight of every kind's shares around the shares of all described roles, fitted on their own counts).
  • A kind of work with few open roles gives figures that move with a handful of postings: rows on too little data are marked as a small sample.

How the job feeds are read and roles are classified: site methodology. How fresh, complete and consistent the data is, checked every day: data quality and coverage.

Editions

The live report covers the last 28 complete days and changes every day. Each complete month is also published as an edition, frozen two days after the month ends, so a figure you cite from an edition never moves.

Cite and reuse

Free to quote, chart and republish under CC BY 4.0 with a link to this page. This edition is frozen, so its figures will read the same later.

Citation

AITraining.jobs (2026). The Market for AI Evaluation & Rating (RLHF) Work, October 2026 edition. https://aitraining.jobs/research/ai-evaluation-rlhf-market/2026-10

BibTeX

@misc{aitrainingjobs_ai_evaluation_rlhf_market_2026_10,
  author = {{AITraining.jobs}},
  title = {The Market for AI Evaluation & Rating (RLHF) Work, October 2026 edition},
  year = {2026},
  howpublished = {\url{https://aitraining.jobs/research/ai-evaluation-rlhf-market/2026-10}},
}
Data: every figure as CSV (one line per value) and JSON, sent to your work email.Pay and open-roles data

More research

The same figures for every other kind of work:

All 81 research reports