Assurance in CI and from Agents

The conversational assurance commands — context extract, design tests, and maintain reconcile — are interactive by default. This page is the contract for running them headless: from CI, from a script, or from an AI agent driving kane-cli.

The ask policy: --mode

On a terminal, extract and design open a chat. Headless is an explicit opt-in — a bare non-TTY invocation exits 2 and mutates nothing:

extract: no TTY — pass an explicit --mode agent|ci|override to run headless

--mode decides both what happens when the agent has a question, and what the command writes to stdout:

Mode	Questions	stdout
interactive	asked in the chat (TTY default)	the Ink chat UI
agent	every question pauses the session — exit 3, resumable; the agent that drives the run answers at resume or escalates to its person	NDJSON events (one JSON object per line); prose diagnostics go to stderr
ci	any high-risk question fails closed — exit 1, error code HIGH_RISK_CI; low- and medium-risk questions take their recommended default (each reported)	prose transcript
override	every default is auto-taken, including high-risk (each flagged in the commit record)	prose transcript

Rule of thumb: agent when something can read the pause and answer (an AI agent, a human on the next shift); ci when a pipeline must fail rather than guess on anything high-risk; override when you accept the recommended defaults wholesale and want one unattended pass.

The same matrix drives maintain reconcile, with two reconcile-specific rules: no headless mode ever archives anything — ARCHIVE decisions wait for an interactive session — and a ci-mode run that hits a decision needing a human stores the plan and exits 2 (the work isn't lost; walk the stored plan interactively or apply it in agent mode).

Exit codes

Consistent across extract, design, and the maintain commands that embed them:

Code	Meaning
0	Complete.
1	Runtime failure. For extract and design, a ci-mode fail-close on a high-risk question also exits 1; reconcile's ci fail-close stores the plan and exits 2 instead.
2	Usage / auth / refusal — bad flags, failed input validation, no store, bare non-TTY without --mode, missing --yes on a destructive command. Nothing was mutated.
3	Paused and resumable — the only meaning of 3. A session is saved; resume it within 24 hours.

The NDJSON stream (--mode agent)

With --mode agent, stdout speaks a versioned NDJSON vocabulary — envelope {"type": "<name>", "v": 1, "verb": "extract"|"design", ...}, one object per line. The vocabulary is open: new event types may appear, so tolerate unknown types.

type	payload highlights
run_start	mode, trace (the per-run log path); design adds use_case
corpus	extract: the sources[] this run covers + already-extracted skipped[]
source_start / source_skipped	source_id, index/total, resumed / reason
plan	the --plan transcription payload
assumed_default	a question auto-answered with its recommended default: id, selected_index, risk. ci and override runs only, since an agent run pauses instead
variables_declared	design: the stubs a phase commit wrote — file (the pool file) and variables[] (name, description, secret). Only names that existed in no variable file, and it fires after the commit that minted the tests
variables_summary	design, at the end of the run: every name still needing a value, in the same shape as variables_declared
agent_activity	progress: kind (tool / decision / progress / thinking_done) + a display label
usage	per agent turn: credits + running total_credits
validate_failed	a proposal failed kane-side validation: codes[], repairing (the agent self-repairs)
commit	what landed: counts + minted[] (cid + logical_id); extract adds proposal_id
receipt	design: per-phase commit receipt — commit_n, phase, committed[], warnings[], parity, and a human-readable next hint
message_sent	your --message was delivered: sid, chars
session_paused	sid, the verbatim resume command, expires_at, and pending_questions[] in full
session_complete	sid
gate_refused	a design gate refused the run (may be the first event)
error	message + a stable code where one exists (NO_STORE, PREFLIGHT, SOURCE_MISSING, BLOB_MISSING, HIGH_RISK_CI, STALE_BASIS)
done	always the last event: status (complete/paused/error/refused/interrupted/aborted) + exit_code

The done guarantee: every --mode agent invocation ends its stream with exactly one done event — including refusals and graceful interrupts. The one exception is operator force: a second Ctrl+C can hard-kill the process (exit 130) without a done. Any other stream that ends without done should be treated as a crash. One more parsing note: the agent may also repair a draft mid-turn on its own — that surfaces only as agent_activity lines (labels like validation failed, refining the draft); treat activity labels as display text, never script against them.

Reconcile's stream

maintain reconcile --mode agent speaks the same envelope with verb: "reconcile" and its own event set:

type	payload highlights
reconcile_plan	the triage ahead: source_id, plan_path, rows[] (kind, ref, why), archive[] (proposed archivals with their evidence-decay reasons)
reconcile_row_start	per row: kind, ref, plus the impact counts where they apply (stale, direct)
reconcile_row_end	the row's outcome (applied | failed | skipped | plan-only | paused) + exit_code, and an additive detail carrying a failure's reason and hint. A row's embedded design run is folded in here, so the stream stays single-writer with exactly one done
reconcile_paused	plan_path + pending[] (ref, why) — resume with the same reconcile command (or --apply)
reconcile_summary	the honest totals, always the same field set: applied, skipped, deferred, plan_only, failed, paused, stale_created
done	always last — same guarantee as above

Validation failures (bad inputs, unknown source, the fork guard) ride the stream as error + done with exit 2 — never stderr alone.

The pause → answer → resume loop

This is the heart of driving assurance from an agent. A real exchange (events abridged, payloads shortened):

$ kane-cli context extract --mode agent
{"type":"run_start","v":1,"verb":"extract","mode":"agent","trace":".context/logs/extract-….log"}
{"type":"corpus","v":1,"verb":"extract","sources":[{"source_id":"prd-online-store","cid":"sha256:0661…"}],"skipped":[]}
{"type":"agent_activity","v":1,"verb":"extract","kind":"decision","label":"asking to resolve an ambiguity"}
{"type":"session_paused","v":1,"verb":"extract","sid":"ext-20260716T140742-prd-online-store",
  "resume":"kane-cli context extract --resume ext-20260716T140742-prd-online-store --mode agent",
  "expires_at":"2026-07-17T14:07:53Z",
  "pending_questions":[{"id":"q1",
    "text":"The PRD conflicts on guest checkout; should I treat checkout as account-required or guest-allowed?",
    "risk":"high",
    "rationale":"Lines L20-L21 say all customers must create an account, but L35 says guest checkout is allowed.",
    "options":[{"label":"Account required","detail":"…"},{"label":"Guest allowed","detail":"…"}],
    "recommended_index":0,"allow_free_text":true}]}
{"type":"done","v":1,"verb":"extract","status":"paused","exit_code":3}

The pause event carries everything needed to decide: the question, why it matters, the options, and the recommendation. Answer in plain words — no question ids, no option indexes. After the resumed run's usual run_start, corpus, and source_start (with "resumed": true) events, the stream continues:

$ kane-cli context extract --resume ext-20260716T140742-prd-online-store --mode agent \
    --message "Account required — treat the update section as superseding: no guest checkout"
{"type":"message_sent","v":1,"verb":"extract","sid":"ext-…","chars":115}
{"type":"usage","v":1,"verb":"extract","credits":2.45,"total_credits":2.45}
{"type":"commit","v":1,"verb":"extract","derived":5,"minted":[{"cid":"sha256:6d68…","logical_id":"uc-create-an-account-to-order"}, …]}
{"type":"session_complete","v":1,"verb":"extract","sid":"ext-…"}
{"type":"done","v":1,"verb":"extract","status":"complete","exit_code":0}

The agent maps your statement to its own pending questions. A statement that answers nothing pending is treated as steering ("also cover the coupon path"); if it leaves a high-risk ambiguity standing, the run pauses again with refreshed questions.

Between the pause and the resume, everything is inspectable without contending the session:

kane-cli context sessions --json                 # one row per resumable session, with its resume command
kane-cli context sessions show <sid> --json      # the pending questions in wire shape + any assumed defaults

Abandoned sessions expire after 24 hours; kane-cli context sessions clean garbage-collects them.

Headless review

Trust promotion deliberately has no auto-approve — but it does have a non-interactive path. Prepare verdicts as JSON and land them atomically:

cat > verdicts.json <<'EOF'
[
  {"ref": "uc-create-an-account-to-order", "resolution": "approved"},
  {"ref": "uc-manage-the-cart",            "resolution": "approved"}
]
EOF
kane-cli context review --verdicts verdicts.json --json

resolution is one of approved | edited | rejected | skipped | supersede (optional reason, edit, supersede_target). One unresolvable ref fails the whole file (exit 2, nothing committed). With --json, each landed verdict echoes as one NDJSON row.

Two 0.7.1 additions:

• Structured verdict flags — for scripted single decisions without a file: --approve <refs...> lands approvals; --skip <refs...> and --defer <refs...> record nothing and leave the items queued. Mutually exclusive with --verdicts.
• Archives require explicit consent. A headless rejection no longer destroys anything: rejected entries are held as non-destructive pending_archive facts (exit 0, loudly summarized). Destroying them takes --allow-archive plus --because "<reason>" — and under --mode ci, archives are refused under any flag (exit 2).

The rule stands: there is no auto-approve. These paths land your decisions faster; they never make them.

Coverage on the stream (0.7.1)

The sync verbs on the stream

kane-cli context sync, kane-cli context push, kane-cli context pull, kane-cli context clone and the subcommands kane-cli context sync add, kane-cli context sync list, kane-cli context sync remove, kane-cli context sync status and kane-cli context sync doctor all take --mode agent and speak the same strict envelope with verb: "sync": stdout is NDJSON only, stderr stays empty, and done is last. None of them calls the agent or spends credits. kane-cli context sync setup is the one command that needs a terminal, and under --mode agent it answers error{code: TTY_REQUIRED} naming kane-cli context sync add and kane-cli context clone as the alternatives. What these commands do is in Sharing the context graph with your team.

type	payload highlights
sync_probe_started / sync_probe	kane-cli context sync add and kane-cli context clone, around the location check (the slow part on a first GitHub fetch): name, kind, then tier (1 means the location can publish, 2 or 3 download only) and detail
sync_status	kane-cli context sync status: name, relation — kind (up-to-date, behind, ahead, diverged, empty-storage, empty-local, foreign-lineage), local and storage (each {seq, hash} of that side's last record, or null when that side is empty) and, on diverged, at (the first record number where the two sides differ, so the last shared record is at - 1) — then rebase_id (the open rebase, or null) and decisions[] (each in the sync_rebase_decision shape). The event needs the location: when it cannot be reached the command refuses instead, and the decisions are not listed
sync_status with --show <n>	needs no location: record[] (the saved record in full, one line each), decision, rebase_id, decisions[], while name and relation are null
sync_locations / sync_removed	kane-cli context sync list / kane-cli context sync remove
sync_pull_done	kane-cli context pull, kane-cli context sync, kane-cli context clone: name, imported, from, to, blobs, proposals, and clone adds dir
sync_push_done	kane-cli context push, kane-cli context sync: name, from, to, pushed, blobs, proposals, already_there
sync_rebase_started	kane-cli context pull --rebase --yes: rebase_id, from, backup_path, moved[], quarantined_tests[]
sync_rebase_item	one per saved record: seq, intent (extract, review, names, retire and so on), outcome (reapplied, already-present, not-reapplied, decision), reason?
sync_rebase_decision	one per open decision, in dependency order: decision_id, kind, intent, seq, label, location, mine and theirs (one line per side, the local change and the location's), answers[] (only the legal ones, each {answer, consequence})
sync_rebase_done	once per walk, and once when kane-cli context sync doctor --abort closes a readable rebase: the four counts, decisions_open, status (complete, paused, aborted)
sync_rebase_open	kane-cli context push or kane-cli context sync while a rebase is still open, followed by sync_error{SYNC_REBASE_PENDING}
sync_doctor / sync_rebase_exported	kane-cli context sync doctor / kane-cli context sync doctor --export
sync_behind	the advisory from kane-cli context extract, kane-cli design tests and kane-cli maintain reconcile when a teammate has pushed since you pulled: name, local_seq, storage_seq, text. Nothing is refused
gitignore_updated	kane-cli context clone and kane-cli context sync doctor --export (and kane-cli context ingest on its extract stream) when the new store's folder is inside a git repository and .context/ was added to its .gitignore: path. When the line could not be written, the store is still created and a warning{message} says why

Exit codes keep their meanings, with one addition: exit 3 is also a person has to decide something, which covers a kane-cli context push or kane-cli context pull refused because you are behind or diverged (the remedy names the command), and a rebase that stopped on open decisions (answer them with --answer, or on a terminal). Exit 2 is a precondition (a location that cannot be reached, a rebase still open, missing keys), and exit 1 a record that cannot be used. A refusal says what stopped, not that nothing happened.

Coverage on the stream

cover --mode agent and cover gaps --mode agent speak the same envelope (verb: "cover" / "gaps"): the full --json payload arrives as one coverage (or gaps) event — (0.8.2) cover gaps <uc-id> emits the document closed over that use-case — and done closes the stream carrying the worklist's ready-to-paste commands in next[]. --mode ci speaks the identical stream. Any refusal is an error event + done with exit 2.

When releases don't match

Sessions bind to the kane-cli release that created them, and the refusals are loud with the remedy in the message: PAIR_MISMATCH at startup (exit 2 — reinstall so the installed pieces match), BINDING_MISMATCH on resume (exit 2 — the session belongs to another release: start fresh, committed work is kept, or resume on the release that created it), and a mid-run "this version of kane-cli is no longer supported — update kane-cli and retry" (a message-only runtime failure, exit 1). Hitting BINDING_MISMATCH on a paused session right after upgrading is expected, not corruption.

Machine-readable reads

These read commands have structured forms: context list --json and context sessions --json (one JSON object per line), context explain --json, context view --json (the full computed graph payload), context view --no-open --out graph.html (render without a browser), and cover --json.

Headless maintain

• maintain reconcile --from <file> --source-id <id> --plan — safe preview: records the source change, stages every proposed row into a stored plan, touches nothing else. Exit 0; when the source actually changed, the plan path is the last stdout line (an unchanged source is a no-op that stores nothing).
• maintain reconcile … --mode override (or --mode ci) — unattended application: ADD and MODIFY rows apply, archiving never happens headless, and ci fail-closes the moment human judgement is needed (the plan is stored; exit 2).
• Re-running the same reconcile command is idempotent — it resumes a pending plan, reports an applied one, and recomputes a superseded one (details).
• Bare headless runs without an explicit --mode refuse with exit 2 — by design.

A CI shape that works

# fail the pipeline on unresolved high-risk ambiguity, never guess:
kane-cli context extract --mode ci

# or: let it pause, surface the questions as a build artifact, resume in a follow-up job:
kane-cli context extract --mode agent > extract.ndjson; code=$?
if [ "$code" -eq 3 ]; then
  kane-cli context sessions --json > pending-sessions.ndjson   # hand to a human or an agent
fi

# design a specific use-case unattended, bounded:
kane-cli design tests --use-case uc-checkout --max 8 --mode ci

# keep the suite honest on requirement changes:
kane-cli maintain reconcile --from ./docs/prd.md --source-id prd --plan

Author and batch the resulting tests with the same CI patterns as any other test — see the CI/CD recipes.

Next steps

• The assurance overview — where each command sits.
• Building the context graph · Designing tests · Maintaining the suite.