Rook Concepts and Data Model

Rook represents autonomous agent testing with seven visible, file-backed concepts. Each concept is stored in the workspace rather than hidden in an opaque local database.

Core Concepts

Concept	Meaning
Project	The grouping under which work is filed and the unit synchronized upstream. One project is active at a time.
Agent	An AI agent discovered in the source material, with a derived description of what it does. A project can contain several agents; one is active.
Feature	One capability extracted during exploration. Scenarios refer to features and pin the exact feature revision they were generated against.
Scenario	One test goal with independently graded acceptance criteria, classification, category, rationale, and source evidence.
Profile	The scripts and environment references Rook uses to invoke an agent. An agent can have several profiles, such as staging and production.
Run	One execution of selected scenarios through one profile. It includes snapshots of the agent, features, profile, scenarios, and results as they existed at execution time.
Verdict	The judge's conclusion for one scenario, including per-criterion status, quoted evidence, confidence, and explicit verification gaps.

Relationships

The same concepts appear in two review surfaces. rook ui --local shows files in the current workspace and selected project. Its redesigned agent view has Summary, Profiles, Features, Scenarios, and Runs tabs. rook ui opens the hosted Web UI's synchronized projects, versions, and shared results. A local record need not exist upstream, and a hosted record need not be present on this machine. See the UI comparison and rollout note.

Verified

Project
└── Agent
    ├── Features
    │   └── Scenarios pinned to feature revisions
    ├── Profiles and hook scripts
    └── Runs
        ├── Frozen agent, feature, profile, and scenario snapshots
        └── Per-scenario verdicts and evidence

This model separates three kinds of change that should not be confused:

implementation changes update the agent and its features;
test-definition changes update scenarios;
target-environment changes update or switch profiles.

A run pins all three, so an old verdict remains explainable after the working tree moves on.

A Complete Scenario

Rook writes editable YAML. A generated scenario resembles:

Verified

title: Process eligible physical item refund under threshold
feature_id: F-005
goal: >-
  I bought a jacket 10 days ago (Order #ORD-50122) for $250, but it does not
  fit. I would like to return it and get a full refund.
acceptance_criteria:
  - statement: >-
      The agent confirms the refund for Order #ORD-50122 is approved or issued
      for $250.
    check: llm_judge
  - statement: >-
      The agent acknowledges that the order is within the 30-day return window
      and is an eligible physical product.
    check: llm_judge
  - statement: >-
      The agent does not say that manager approval is required.
    check: llm_judge
category: happy_path
class: functional
confidence: low
input_kind: text
output_kind: text
why: >-
  Catches incorrect denial or manager escalation for a standard refund at or
  below the documented threshold.
local_id: SC-001
origin: generated
executable: true
sources:
  - .claude/agents/refund-desk.md
feature_revision_id: sha256:6b7e9f5a913d886e0e30dd871614fff85f151...

Why the Fields Matter

Field	Purpose
`goal`	The user-level task passed to the agent.
`acceptance_criteria`	Independent requirements. A scenario fails when any verified required criterion fails.
`class` and `category`	Position the test within the functional, non-functional, or adversarial taxonomy.
`why`	Lets a reviewer decide whether the test deserves to exist.
`sources`	Connects the test to the material from which Rook derived it.
`feature_revision_id`	Distinguishes a product regression from a test whose source definition changed.
`executable`	Records whether the scenario can run against the current profile and evidence surface.

Every field is reviewable and editable. Regeneration can retain current scenarios whose inputs and feature pins still match, while --force deliberately re-derives them.

Local State Is the Record

The .testmuai/rook/ directory is the authoritative workspace record. Synchronization copies reviewed state upstream; it does not replace the local model with hidden cloud state.

Local UI: See the Scenario Model

Run rook ui --local, open the agent's Scenarios tab, and choose a scenario. In this CommerceCare demo, SC-006 belongs to feature F-002, has a goal and three criteria, and links to execution history. The definition and the result remain different records. See the local UI rollout note for older layouts.

Local scenario page illustrating the relationship between scenario ID, feature ID, goal, criteria, and history

Hosted Web UI: The Synchronized Definition

Run rook ui and open project → agent → Scenarios → scenario ID. This hosted capture shows SC-002 from the separate triage example. Follow a history entry to see what happened in a particular run; editing the current definition does not rewrite that run's snapshots.

Hosted SC-002 definition showing its goal, acceptance criteria, metadata, and execution history

In this capture, History shows 0 pass despite the corresponding result recording four passing criteria. See the screenshot display notes before interpreting that count.