Move beyond final answers toward professional work with context, evidence, and reviewable reasoning.
Published results and documented interfaces, not CausalMind experiment results.
The answer is only the endpoint
In complex professional tasks, the same conclusion can come from very different paths. Final text alone cannot separate sound evidence-based work from a lucky guess. Time boundaries, available material, constraints, and deliverable format belong in the sample.
Make judgment inspectable
Useful expert traces preserve decisive actions, source evidence, tradeoffs, and uncertainty. Rubrics can separate factual accuracy, evidence, constraint compliance, and communication. When reviewers disagree, record the disagreement and refine the task or criteria.
Close the feedback loop
Real rollouts expose leakage, ambiguous grading, and missing environment state. Feed those findings back into data design. Traceable quality improvement matters more than one-off labeling.
What the public study measured
The first GDPval release has 1,320 tasks across 44 occupations. Task writers averaged more than 14 years of experience, and each task received five review rounds on average. This effort suggests that the hard part of expert data is verifying representativeness, sufficient context, and fair review, not merely collecting a polished answer. OpenAI's business eval guide recommends reviewing 50 to 100 early outputs to build an actionable error taxonomy.
A protocol we would use
Give two independent experts the same time-bounded materials and record conclusions, decisive evidence, rejected alternatives, and uncertainty. Ask a third reviewer, blind to authorship, to score factual accuracy, evidence, constraints, and communication separately. Disagreement about task meaning calls for a clearer brief; disagreement at the scoring boundary calls for a revised rubric. Keep the inputs, outputs, and review reasons traceable before using the trace in training or evaluation.
Primary sources
Editorial analysis by CausalMind; benchmark names refer to the cited original projects.