Business Analyst, Computer-Using Agent Evaluation

The work

You will evaluate how computer-using AI agents follow instructions, analyze information, reason through problems, and complete tasks in digital environments. You will review their actions and responses, check claims through online research, and give clear feedback.

The work includes creating questions and scenarios for testing AI models, rating generated responses, and explaining errors or missing information. You will also analyze and summarize complex written content, interpret data, compare evidence, solve logic and arithmetic questions, and identify unsupported conclusions.

• Review AI-generated responses and actions for accuracy and task completion.
• Research claims, sources, and supporting evidence online.
• Create evaluation scenarios and questions for large language models.
• Rate responses and write annotations that explain errors, gaps, and inaccurate claims.
• Analyze, summarize, and structure complex written information clearly.
• Solve analytical, logical reasoning, data interpretation, arithmetic, and puzzle-based questions.

What it pays and takes

This is an ongoing freelance engagement for part-time evaluation work. The listing does not provide a pay rate.

• Workload: 20 or more hours per week.
• Location: Open worldwide.
• Language: Fluent English communication and reading comprehension.
• Skills: Analytical, logical, critical, and creative reasoning; evidence evaluation; pattern recognition; data interpretation; and basic calculations.
• Work style: Work independently while following detailed instructions, quality standards, and timelines.
• Useful tools: Excel, Google Sheets, Google Docs, and Google Workspace.
• Relevant study in engineering, literature, journalism, communications, arts, statistics, business, or a related field is welcome. Equivalent skills and experience may also be considered.

How it works

Apply on OpenTrain with your resume and then complete the application on the hiring site.

About AI training work

AI training work is the human work behind systems that learn from examples, including reviewing model responses, rating results, and checking information. People with strong analytical and communication skills help make these systems more accurate and useful.

Where you can work

This role is 100% remote: you can work from any country except Andorra, Afghanistan, Barbados, Burkina Faso, Burundi, Brazil, Belarus, Belize, DR Congo, Central African Republic, Congo (Republic), Cameroon, China, Cuba, Cape Verde, Djibouti, Dominica, Eritrea, Fiji, Micronesia, Grenada, Georgia, Guinea, Equatorial Guinea, Guinea-Bissau, Honduras, Haiti, Iraq, Iran, Kyrgyzstan, Kiribati, Comoros, St. Kitts and Nevis, North Korea, Lebanon, Liberia, Lesotho, Libya, Montenegro, Marshall Islands, Mali, Myanmar, Mauritania, Maldives, Malawi, Nicaragua, Nepal, Nauru, Papua New Guinea, Palestine, Palau, Russia, Solomon Islands, Seychelles, Sudan, Sierra Leone, Somalia, Suriname, South Sudan, São Tomé and Príncipe, Syria, Eswatini, Chad, Togo, Tajikistan, Timor-Leste, Turkmenistan, Tonga, Tuvalu, Ukraine, Uganda, Vatican City, St. Vincent and Grenadines, Venezuela, Vanuatu, Samoa, Yemen, Zambia and Zimbabwe.