Member of Technical Staff, Realm Lab AI Research
Remote in 48 countries
United Arab Emirates, Argentina, Austria, Bangladesh, Belgium, Bahrain, Canada, Switzerland, Chile, Colombia, Germany, Denmark, Algeria, Egypt, Spain, France, United Kingdom, Ghana, Greece, Indonesia, India, Italy, Jordan, Japan, Kenya, Kuwait, Kazakhstan, Sri Lanka, Morocco, Mexico, Malaysia, Nigeria, Netherlands, Oman, Peru, Philippines, Pakistan, Portugal, Qatar, Saudi Arabia, Singapore, Thailand, Tunisia, Taiwan, United States, Uzbekistan, Vietnam, South Africa
The middle of its range, $115/hr, is above the $65/hr median of the Coding & Software Evaluation roles open now, whose typical pay runs $40–$135/hr. How that pay moves week by week is in our research on Coding & Software Evaluation work.
Half of all Coding & Software Evaluation roles come down within 18 days of being listed, and 45.4% are still up after four weeks. This one was posted today.
About the role
The work
- Own research and evaluation projects from task design and dataset development through benchmark validation and performance analysis.
- Design and validate capability evaluations, reinforcement learning environments, and benchmark suites for computer use, agent workflows, and emerging model capabilities.
- Develop datasets, task specifications, evaluation rubrics, and methods that improve post-training data quality and model assessment.
- Analyze model behavior through failure analysis, error taxonomies, and identification of capability gaps.
- Build reproducible evaluation pipelines and feedback loops for iterative model development.
- Contribute to benchmarks, research papers, technical reports, and open research artifacts.
What it pays and takes
- Pay: $100 to $130 per hour.
- Work arrangement: Contract work listed as part-time.
- Location: Open to candidates in the countries listed for this role across North America, South America, Europe, Asia, Africa, and the Middle East.
- Language: Professional fluency in English.
- Requirements: A strong technical background in computer science, machine learning, AI, or a related field; experience designing ML evaluations, benchmark suites, research datasets, or experimental frameworks; and proficiency in Python.
- Requirements: Experience building data pipelines, evaluation tools, or ML research infrastructure; understanding of large language models, foundation models, post-training techniques, and evaluation methods; and the ability to turn complex workflows into structured tasks, measurable criteria, and research experiments.
- Requirements: Strong analytical skills in model failure analysis, experimental design, and evaluation quality.
- Helpful background: Experience with reinforcement learning environments, reward modeling, simulators, feedback-driven training systems, computer use agents, tool use, browser automation, multi-step agent workflows, multimodal reasoning, long-context evaluation, instruction following, deep research benchmarks, human-in-the-loop evaluations, expert-generated datasets, post-training data, AI research publications, open-source benchmarks, or evaluation frameworks.
How it works
You apply here in about two minutes and finish on the hiring platform. The next step is a short AI interview (technical roles include a coding challenge). Passing it is how you qualify for paid tasks, so take it soon after you are invited.
The hiring platform's application for this job asks for your resume, your LinkedIn profile, your phone number, your location and answers to 4 screening questions.
About AI training work
AI training work is the human side of building artificial intelligence. People create and review datasets, evaluate model responses, and design tests that help AI systems improve, so roles requiring research and evaluation expertise pay for that specialized knowledge.
Where you can work
This role is 100% remote, for applicants located in these 48 countries: United Arab Emirates, Argentina, Austria, Bangladesh, Belgium, Bahrain, Canada, Switzerland, Chile, Colombia, Germany, Denmark, Algeria, Egypt, Spain, France, United Kingdom, Ghana, Greece, Indonesia, India, Italy, Jordan, Japan, Kenya, Kuwait, Kazakhstan, Sri Lanka, Morocco, Mexico, Malaysia, Nigeria, Netherlands, Oman, Peru, Philippines, Pakistan, Portugal, Qatar, Saudi Arabia, Singapore, Thailand, Tunisia, Taiwan, United States, Uzbekistan, Vietnam and South Africa.
From OpenTrain AI's job posting, checked 17 Nov 2026.
About OpenTrain AI
A unified board of AI-training jobs with real per-role pay.
How it pays: Stripe payouts to your bank, in the 120 countries its payout docs list.
When it pays: When the employer approves each funded milestone. How often others pay
Interview: on some roles or projects (AI interview (switched on per job by the employer)). How the OpenTrain AI AI interview works
Similar open roles
Some links on AITraining.jobs are referral links. If you sign up through one, AITraining.jobs may earn a commission at no extra cost to you. It never changes what a platform pays you. A role at a platform that pays us a referral fee weighs more in this order.

Prompt & Response Evaluator (opens the job details)
micro1 · AI Evaluation · Remote· 4d agoDetails
$40–$70/hr
Document Annotation Specialist (PDF) (opens the job details)
SuperAnnotate (SME) · Data Annotation · Remote· 6d agoDetails
up to $10/hr
General AfterQuery Expert (Data Annotator) (opens the job details)
AfterQuery · Data Annotation · Remote· 1y+ agoDetails
$43/hr
Generalist AI Trainer (opens the job details)
RemoExperts · AI Evaluation · Remote· 1y+ agoDetails
$30–$65/hr
An experienced engineer?
One profile for every AI training platform. Companies ask us for people with a specific language, country or credential: add yours once, and when a role fits, we introduce you, only with your yes. Free.
Get new OpenTrain AI roles by email
We check the OpenTrain AI job board every hour and email you its newest roles each week, plus similar paid roles elsewhere. Free, unsubscribe anytime.