How the data is shaped

Owner corrections of AI agent work

One JSON record per line (JSONL, UTF-8). Each record is one place where the owner corrected the agent: what it was asked, what it produced, what the owner said, what a good response does, and a rubric. Records come in three tiers:

A record's shape says how much context it carries: conversation includes the turns leading up to the corrected response, failure_only holds the corrected output and its correction, and prompt_only holds a task and its rubric. metadata.verbatim marks which of the prompt, the response and the correction are word for word, and metadata.confidence is high or medium (low-confidence records are held back).

How a record is laid out, with placeholders in place of values:

{
  "id": "BB-T-0000",
  "tier": "A",
  "model_tested": "<model>",
  "failure_class": "<class>",
  "shape": "conversation",
  "messages": [{ "role": "user", "content": "<the ask>" }, { "role": "assistant", "content": "<the response>" }],
  "operator_correction": "<the owner's words>",
  "flaw": "<what the response got wrong>",
  "expected_behavior": "<what a good response does>",
  "rubric": [{ "id": "r1", "criterion": "<criterion>", "points": 3, "critical": true }],
  "metadata": { "source": "transcript", "verbatim": { "prompt": true, "response": true, "correction": true }, "confidence": "high", "scrubbed": true }
}

Every field

From the current release; the last column counts the records that carry the field.

Promised versus proven: the work ledger

One JSON record per line: one job with its whole timeline. Each job an agent was handed, every status change, each claim that it was done with its stakes, the reviewer model's verdict, and the owner's sign-off or send-back. Structure only: every value is a date, a count, an id or a fixed label, with no titles, notes or evidence text, and the six businesses appear as sector labels.

Releases

Releases are quarterly. Each one adds the new records and keeps the earlier ones, and the dataset page lists every release with what changed. Record ids are stable across releases.

Request API

The forms on this site use the same two endpoints, so a request can also be sent from a script.