Expert data: red-teaming and safety
Red-teaming and safety testing for AI, by US domain experts.
Security engineers, chemists, biologists, clinicians, psychologists and attorneys from our own network try to make your AI fail in the ways that matter in their field, and write down how. We plan the project, staff it, check the work and deliver it.
For labs and AI companies testing a model before launch, and enterprises that must show their AI was tested by people qualified to judge it.
What we deliver
Red-teaming and safety expert work your AI is built and judged on
Red-teaming
Experts attack your system in their own field: exploits and insecure code, chemical and biological uplift, dangerous medical advice, self-harm and crisis conversations, legal harm, and the policy your system is meant to keep.
Expert evaluation
Experts grade your system's responses to risky requests against your written policy, so you can measure safety before launch and at each new version.
Rubrics and grading criteria
Experts write the policy's lines in their field: what a safe answer gives, what it withholds, and why.
Preference and comparison data
Experts compare two responses to the same risky request and say which is safer and still useful, and why.
Benchmarks
Experts write sets of hard risky prompts by field, for repeatable safety testing.
The experts
Who does the work, and how we check them
Security engineers, chemists, biologists and CBRN specialists, physicians and psychologists, attorneys, and trust and safety professionals.
Each expert is checked for the field they test: licences on their public record (the NPI Registry for clinicians, the bar for attorneys), research records for scientists, and their work and a spoken assessment for security engineers.
- Health and medicine782 (111 US)
- Law182 (36 US)
- Science, math and engineering681 (78 US)
- Software and IT1,384 (197 US)
For a field we are short of, we recruit the professionals the project needs.
What experts cost
What red-teaming and safety experts are paid for AI work today
We read the pay of every open AI-training role on the major platforms every hour, so a quote starts from what that kind of expert is paid now, not a guess. A project's price per expert hour is the expert's rate plus our fee for running it, and the quote shows both.
| Expert | Typical rate | Range | Measured from |
|---|---|---|---|
| Physicians | $130/hr | $100 to $200 | 58 rates, 12 platforms |
| Lawyers | $110/hr | $80 to $400 | 89 rates, 18 platforms |
| Psychologists | $90/hr | $85 to $200 | 13 rates, 6 platforms |
| Cybersecurity experts | $80/hr | $50 to $125 | 30 rates, 8 platforms |
| Chemists | $70/hr | $40 to $125 | 35 rates, 13 platforms |
| Biologists | $70/hr | $40 to $125 | 39 rates, 11 platforms |
| Nuclear and CBRN experts | $55/hr | $40 to $140 | 16 rates, 7 platforms |
Typical rate: the middle of what the open roles naming that profession pay, for roles open to US experts; the range runs from the 25th percentile of those roles to the 90th-percentile advertised ceiling. Read live.
How it works
From your brief to delivered work
Tell us what your AI needs
The field, the kind of expert work, how much of it and when, and anything about the data. A short chat or an email is enough.
We plan it from our own network
The experts we would put on it, with their credentials and how we assessed them, how the work is checked, and a first batch with what counts as accepted written down.
A paid pilot
One kind of expert, one field or one set of tasks, with as many experts as you need, done and checked against the agreed standard, so you judge real work before you commit to more.
The work runs, checked and delivered
In your tools or ours, reviewed before it reaches you, delivered and invoiced on the schedule you set, with one contact for the whole project.
Questions
What teams ask us about red-teaming and safety expert data
Who decides what counts as a failure?
Your policy, written down before the work starts. Our experts apply it in their field, and where it is silent they say what they would expect and why, so your policy improves with the testing.
Are you a recruiting agency or a staffing firm?
No. We deliver the work: we plan the project, put experts from our own network on it, check what they produce and deliver it to you. You deal with one company for the whole project, not with the experts one by one.
Where do your experts come from?
From AITraining.jobs, the site US professionals use to find paid AI training work. 8,524 people have a profile with us, 645 of them US experts (in the US and licensed, holding a doctorate or a Verified Expert, as their profile reads). We read each profile, check licences on public registries where we can (the NPI registry for clinicians, the New York and California bar records for attorneys, and each state board's record before anyone starts a project), assess people in a spoken interview, and know who is available now. For a field we are short of, we recruit the professionals the project needs.
How do you check the quality of the work?
Against a written standard agreed with you before the first batch: a reviewer from the same field checks the work, with test items whose correct answers are known, and anything that misses the standard is redone before it reaches you. You see the checks, not only the result.
How is it priced?
Per expert hour: the expert's rate for that kind of work plus our fee for running the project. We set the expert's rate from the market, because we read the pay of every AI-training role on the major platforms every hour, so you pay what that expertise costs today, not a guess. The quote shows the expert's rate and our fee separately, and a paid pilot (one kind of expert, one field or one set of tasks, with as many experts as you need) is priced below our usual fee.
What about our data and confidentiality?
Your data and instructions are used for your project only. Every expert on it has accepted written confidentiality and assigns their work to you. Health data that identifies a person is handled only under a signed business associate agreement.
Can the experts work in our own tools?
Yes. Most projects run in the client's own labeling or evaluation tool; where you have none, we provide one.
Why would experts choose to work through you?
Because we pay for every hour a project asks of them, onboarding and calibration included, never track their screens, write the pay and the terms down before they accept, and let them answer any review of their work. The best experts have many offers; this is how we keep them.
Other fields
Expert data in every field we staff
For companies building AI
Tell us what your AI needs
A short chat. We reply with the experts we would put on it, how we would check the work and a plan for the first batch. Or have us find the experts for you to contract yourself.
We find the experts
We find and assess the experts your work needs and introduce the ones who said yes. You contract them yourself.
A fee per expert you accept.
We run the project
We staff, run, review and deliver the expert work your AI needs, with professionals from our own network.
Priced per expert hour.
Talk to Yuma Heymans, Founder & CEO of AITraining.jobs.
Founder and CEO of AITraining.jobs and co-founder of the AI recruitment platform HeroHunt.ai, based in San Francisco, with years spent mapping how AI-training and data work is sourced, staffed, and paid.