AI Safety Annotation Reviewer
Half of all Red-Teaming & AI Safety roles come down within 21 days of being listed, and 19.3% are still up after four weeks. This one was posted 1 day ago.
About the role
The work
You will audit evaluations and written assessments created for AI training. The work focuses on policy alignment, reasoning quality, scoring accuracy, required formats, and complete annotations across sensitive review tasks.
You will identify individual and recurring errors, give clear feedback, recommend corrections, and document patterns that can improve calibration, quality, and reviewer training. You will need to maintain accuracy while handling sensitive content and time-sensitive priorities.
- Review trainer-submitted evaluations for policy errors, incomplete reasoning, scoring drift, format errors, and inconsistent annotations.
- Apply detailed AI safety policies consistently across review tasks.
- Check scoring accuracy, policy alignment, required schemas, and annotation completeness.
- Provide direct, constructive feedback and recommend corrective actions.
- Document recurring error patterns and support calibration, quality improvement, and reviewer training.
What it pays and takes
This is a remote contract, part-time role with flexible working hours and a substantial weekly commitment.
- Location: Open worldwide.
- Time commitment: 20+ hours per week.
- Language: Professional fluency in English.
- Experience level: Intermediate.
- Strong written communication, attention to detail, and analytical and logical reasoning.
- Ability to interpret and apply detailed policy guidelines consistently.
- Ability to identify systematic errors in annotations and evaluations.
- Clear, constructive communication and independent work under time pressure.
- Understanding of US cultural context and communication norms.
- Useful background includes AI annotation review, trust and safety quality assurance, test engineering, content moderation, editorial fact-checking, schema validation, policy compliance review, or quality auditing.
- Experience with sensitive or high-stakes content is relevant.
How it works
Apply on OpenTrain with your resume, then complete the application on the hiring site.
About AI training work
AI training is the human work behind systems that learn from examples, including reviewing written responses and rating model behavior. OpenTrain is a free platform where people find AI training and data-labeling projects, build a profile, and apply in minutes.
Where you can work
This role is 100% remote: you can work from any country except Andorra, Afghanistan, Barbados, Burkina Faso, Burundi, Brazil, Belarus, Belize, DR Congo, Central African Republic, Congo (Republic), Cameroon, China, Cuba, Cape Verde, Djibouti, Dominica, Eritrea, Fiji, Micronesia, Grenada, Georgia, Guinea, Equatorial Guinea, Guinea-Bissau, Honduras, Haiti, Iraq, Iran, Kyrgyzstan, Kiribati, Comoros, St. Kitts and Nevis, North Korea, Lebanon, Liberia, Lesotho, Libya, Montenegro, Marshall Islands, Mali, Myanmar, Mauritania, Maldives, Malawi, Nicaragua, Nepal, Nauru, Papua New Guinea, Palestine, Palau, Russia, Solomon Islands, Seychelles, Sudan, Sierra Leone, Somalia, Suriname, South Sudan, São Tomé and Príncipe, Syria, Eswatini, Chad, Togo, Tajikistan, Timor-Leste, Turkmenistan, Tonga, Tuvalu, Ukraine, Uganda, Vatican City, St. Vincent and Grenadines, Venezuela, Vanuatu, Samoa, Yemen, Zambia and Zimbabwe.
From OpenTrain AI's job posting, checked 16 Nov 2026.
About OpenTrain AI
A unified board of AI-training jobs with real per-role pay.
How it pays: Stripe payouts to your bank, in the 120 countries its payout docs list.
When it pays: When the employer approves each funded milestone. How often others pay
Interview: on some roles or projects (AI interview (switched on per job by the employer)). How the OpenTrain AI AI interview works
Similar open roles

Open Source GitHub Maintainer Code Review (opens the job details)
OpenTrain AI · Coding · Remote· 1mo agoDetails
$120–$200/hr
CBRN Specialist (AI Safety Review) (opens the job details)
OpenTrain AI · Red-Teaming · Remote· 2mo agoDetails
up to $60–$100/hr

SaaS Contracts Counsel (Technology Transactions) (opens the job details)
OpenTrain AI · Domain Experts · Remote· 1mo agoDetails
$80–$110/hr
Language Expert (Korean Audio Transcription) (opens the job details)
OpenTrain AI · Transcription & Audio · Remote· 1mo agoDetails
$20–$36/hr
Want companies to ask for you?
One profile for every AI training platform. Companies ask us for people with a specific language, country or credential: add yours once, and when a role fits, we introduce you, only with your yes. Free.
Get new OpenTrain AI roles by email
We check the OpenTrain AI job board every hour and email you its newest roles each week, plus similar paid roles elsewhere. Free, unsubscribe anytime.