AI Evaluation Specialist for Large Language Models
Half of all AI Evaluation & Rating (RLHF) roles come down within 18.5 days of being listed, and 33.6% are still up after four weeks. This one was posted 46 days ago.
About the role
The work
You will test large language models and assess how well they perform. The work involves careful reasoning, accurate instruction-following, and independent review in a primarily asynchronous remote setting.
- Test large language models as part of AI training projects.
- Use general academic knowledge and reasoning to assess model performance.
- Review model behavior carefully and identify ways systems can improve.
- Follow detailed instructions accurately while completing evaluation tasks.
- Work independently in a remote, primarily asynchronous setting.
What it pays and takes
This is a short-term, part-time freelance opportunity for people who want practical experience evaluating AI systems. All academic disciplines are welcome, and no professional AI training experience is required.
- Pay: Up to $17 per hour.
- Location: Open to candidates in the United States.
- Language: Fluent in English.
- Schedule: No minimum weekly commitment.
- Education: Currently enrolled in or have completed an associate, bachelor's, or master's degree.
- Skills: Strong reasoning, exceptional attention to detail, consistent accuracy, and comfort working independently.
- Training: Willingness to complete brief training and learn new evaluation workflows.
How it works
Apply on OpenTrain with your resume, then complete the application on the hiring site. If selected, you will work on short-term AI training projects involving language-model evaluation.
About AI training work
AI training is the human work behind systems that generate and understand language. Contributors review model responses and behavior so AI systems can become more useful, accurate, and reliable. Experienced reviewers are valued because careful judgment helps distinguish strong model performance from subtle errors.
Where you can work
This role is 100% remote, for applicants located in United States.
From OpenTrain AI's job posting, checked 16 Nov 2026.
About OpenTrain AI
A unified board of AI-training jobs with real per-role pay.
How it pays: Stripe payouts to your bank, in the 120 countries its payout docs list.
When it pays: When the employer approves each funded milestone. How often others pay
Interview: on some roles or projects (AI interview (switched on per job by the employer)). How the OpenTrain AI AI interview works
Similar open roles

Open Source GitHub Maintainer Code Review (opens the job details)
OpenTrain AI · Coding · Remote· 1mo agoDetails
$120–$200/hr
CBRN Specialist (AI Safety Review) (opens the job details)
OpenTrain AI · Red-Teaming · Remote· 2mo agoDetails
up to $60–$100/hr

SaaS Contracts Counsel (Technology Transactions) (opens the job details)
OpenTrain AI · Domain Experts · Remote· 1mo agoDetails
$80–$110/hr
Language Expert (Korean Audio Transcription) (opens the job details)
OpenTrain AI · Transcription & Audio · Remote· 1mo agoDetails
$20–$36/hr
Want companies to ask for you?
One profile for every AI training platform. Companies ask us for people with a specific language, country or credential: add yours once, and when a role fits, we introduce you, only with your yes. Free.
Get new OpenTrain AI roles by email
We check the OpenTrain AI job board every hour and email you its newest roles each week, plus similar paid roles elsewhere. Free, unsubscribe anytime.