02 / EXPERT / JUDGMENT
Expert tasks & decision traces
Preserve the context, actions, evidence, and feedback behind expert judgment.
SFTRLHFGDPvalALE Task
Task forms include SFT, RLHF, GDPval-style occupational deliverables, and ALE Task. Depending on the objective, deliveries can contain demonstrations, preference comparisons, grading reasons, and reviewable work products. Public benchmark names describe task forms only.
DELIVERY / CONTENT
What a delivery can include
Defined with each project and its authorization scope.
- 01Task materials, time boundary, and deliverable specification
- 02Expert demonstrations, preference pairs, and decision traces
- 03Rubrics, evidence, review notes, and disagreement records
SCENARIO / EXAMPLE
A task in context
An analyst compares new industry evidence with an earlier thesis and produces a cited memo; reviewers score factual accuracy, reasoning, and deliverable quality separately.
QUALITY SIGNAL / 02
What we verify
Check evidence provenance, temporal consistency, interpretable disagreement, and train/eval separation.
CONTEXT→ACTION→FEEDBACK