LLM Annotator, Master's Degree

The work

You will help evaluate and improve large language models by testing how well they retrieve, analyze, and reason over structured data. You will review AI-generated answers, identify failures, and explain your findings using clear evidence.

The work includes prompt design, response evaluation, dataset validation, and annotation according to project guidelines. You will focus on factual accuracy, logical reasoning, completeness, hallucinations, inconsistencies, and gaps in reasoning.

• Create challenging prompts that test model performance on structured data.
• Assess AI-generated responses for factual accuracy, reasoning quality, and completeness.
• Identify hallucinations, inconsistencies, reasoning gaps, and other model failures.
• Validate outputs against datasets and supporting evidence.
• Write clear explanations that justify your evaluations.
• Follow annotation guidelines and maintain consistent quality standards.

What it pays and takes

This is a remote contractor assignment with a four-week duration.

• Pay: Not provided in the listing.
• Schedule: Choose 20, 30, or 40 hours per week.
• Time zone: At least four hours of overlap with Pacific Time is required.
• Location: Remote and open worldwide.
• Education: Master's degree or higher in any discipline.
• Experience: At least three years of professional, research, or teaching experience.
• Communication: Excellent written English.
• Skills: Strong analytical and critical-thinking abilities, careful attention to detail, and the ability to validate information against source data.
• Helpful background: Large language models, generative AI, prompt engineering, AI evaluation, data annotation, model testing, or structured datasets such as CSV files, spreadsheets, and databases.
• Additional strength: Ability to identify edge cases and design prompts that expose model limitations.

How it works

Apply on OpenTrain with your resume, then complete the application on the hiring site.

About AI training work

AI training is the human work behind systems that learn from examples, including writing prompts, rating model responses, and checking facts. People with strong analytical skills or relevant experience help make these systems more accurate and useful, while OpenTrain provides a free place to find projects and build an AI training career.

Where you can work

This role is 100% remote: you can work from any country except Andorra, Afghanistan, Barbados, Burkina Faso, Burundi, Brazil, Belarus, Belize, DR Congo, Central African Republic, Congo (Republic), Cameroon, China, Cuba, Cape Verde, Djibouti, Dominica, Eritrea, Fiji, Micronesia, Grenada, Georgia, Guinea, Equatorial Guinea, Guinea-Bissau, Honduras, Haiti, Iraq, Iran, Kyrgyzstan, Kiribati, Comoros, St. Kitts and Nevis, North Korea, Lebanon, Liberia, Lesotho, Libya, Montenegro, Marshall Islands, Mali, Myanmar, Mauritania, Maldives, Malawi, Nicaragua, Nepal, Nauru, Papua New Guinea, Palestine, Palau, Russia, Solomon Islands, Seychelles, Sudan, Sierra Leone, Somalia, Suriname, South Sudan, São Tomé and Príncipe, Syria, Eswatini, Chad, Togo, Tajikistan, Timor-Leste, Turkmenistan, Tonga, Tuvalu, Ukraine, Uganda, Vatican City, St. Vincent and Grenadines, Venezuela, Vanuatu, Samoa, Yemen, Zambia and Zimbabwe.