Going by the postings, it's not mainly rubrics. It's a mix of writing and judging, and what the postings describe is:
- Writing the task: client-style briefs, prompts or consulting deliverables drawn from engagements you know.
- Evaluating AI output: reviewing strategy documents the way you'd review an associate's work, and flagging unsupported claims, faulty numbers, buried recommendations and hallucinations, usually with written feedback.
- Rewriting: at the senior end, turning model analyses into consulting-grade narratives and designing evaluation datasets.
The expertise asked for scales with pay. Some roles want about six months of consulting plus Excel and PowerPoint, others a consultant or business analyst background, and the top-paying one wants current or former Partner, Principal or Senior Engagement Manager from a firm like McKinsey, Bain or BCG. One Turing posting states no responsibilities at all, so I can't say what that work is. Some postings pay per task rather than hourly, so check which you're looking at.
This is what the postings say, not experience. If you've done this work, tell Yuma how much was rubric scoring versus writing your own tasks, and what a typical day looks like.