Senior Python Engineer, LLM Evaluation
Half of all Coding & Software Evaluation roles come down within 18 days of being listed, and 44.8% are still up after four weeks. This one was posted 12 days ago.
About the role
The work
- Curate code examples and create precise programming solutions for AI model training datasets.
- Evaluate and refine AI-generated code for correctness, efficiency, scalability, and reliability.
- Build Python agents and automated verification tools that identify error patterns and validate software solutions.
- Assess model capabilities across prototyping, architecture design, API design, implementation, launch, experimentation, monitoring, and operational maintenance.
- Design verification methods for software engineering tasks and compare coding solutions against performance benchmarks.
- Contribute code in Python, JavaScript including ReactJS, C/C++, Java, Rust, and Go.
What it takes
- Pay: No pay amount is provided for this role.
- Time: 20+ hours per week.
- Location: Open to candidates in the United States.
- Experience: At least 3 years of software engineering experience.
- Strong Python expertise, including production frameworks, tooling, and best practices.
- Experience building full-stack applications and deploying scalable software.
- Deep knowledge of software architecture, development, debugging, and code quality review.
- Ability to write clear, structured rationales for technical evaluations.
- Familiarity with backend services, data pipelines, and machine learning infrastructure.
- Experience building production systems in a high-scale engineering environment and a strong computer science foundation are helpful. Formal education is helpful, but practical engineering skill is valued.
How it works
Apply on OpenTrain with your resume, then complete the application on the hiring site.
About AI training work
AI training is the human work behind artificial intelligence, including reviewing code and judging whether model responses are correct and useful. Engineers are needed for specialized projects because their software knowledge helps produce accurate evaluations and better training data.
Where you can work
This role is 100% remote, for applicants located in United States.
From OpenTrain AI's job posting, checked 18 Nov 2026.
About OpenTrain AI
A unified board of AI-training jobs with real per-role pay.
How it pays: Stripe payouts to your bank, in the 120 countries its payout docs list.
When it pays: When the employer approves each funded milestone. How often others pay
Interview: on some roles or projects (AI interview (switched on per job by the employer)). How the OpenTrain AI AI interview works
Similar open roles
Some links on AITraining.jobs are referral links. If you sign up through one, AITraining.jobs may earn a commission at no extra cost to you. It never changes what a platform pays you. A role at a platform that pays us a referral fee weighs more in this order.

Data-Video Generalist (US-based) (opens the job details)
micro1 · Robotics Data · Remote· todayDetails
$13–$15/hr
Audio and Video Data Annotator (opens the job details)
OpenTrain AI · Data Annotation · Remote· 3mo agoDetails
$6–$8/hr
Document Annotation Specialist (PDF) (opens the job details)
SuperAnnotate (SME) · Data Annotation · Remote· 1w agoDetails
up to $10/hr
Full-Stack Software Engineer - Testing & Evaluation (opens the job details)
AfterQuery · AI Evaluation · Remote· 11mo agoDetails
$75/hr
Prompt & Response Evaluator (opens the job details)
micro1 · AI Evaluation · Remote· 6d agoDetails
$40–$70/hr
An experienced engineer?
One profile for every AI training platform. Companies ask us for people with a specific language, country or credential: add yours once, and when a role fits, we introduce you, only with your yes. Free.
Get new OpenTrain AI roles by email
We check the OpenTrain AI job board every hour and email you its newest roles each week, plus similar paid roles elsewhere. Free, unsubscribe anytime.