Search BrigadeStack

Search BrigadeStack pages and docs.

BrigadeStack
Menu
Docs menu

Docs / Reference

outcome

The outcome ledger, content fingerprints, capability-aware and recency-weighted ranking, and the fingerprint-aware promotion ratchet.

Use this to see which skills and notes helped your agents get work right, and which to drop.

The outcome ledger is the second loop. Filing notes earns nothing. The ledger earns trust. BrigadeStack promotes a learned skill only when a real signal proves it helped, and rolls a regressed one back the moment a signal says it broke. The model never grades its own work. The whole ledger is append-only JSON and markdown under memory/outcome/, tracked in git and readable without BrigadeStack.

Recording a result

brigadestack outcome capture <skill-or-card> --run-id latest records the result of a verify run, a real exit code rather than an opinion, against the artifact that produced it. brigadestack outcome record writes a result directly.

Both stamp each new record with two facts about the world at capture time:

  • A content_fingerprint, the SHA-256 of the artifact’s content, taken from the installed copy first and the registry master as a fallback.
  • A context manifest (Brigade version, interpreter major.minor, platform, plus best-effort agent tool and model tagged with their source) and a capability_fingerprint over the low-cardinality vector {harness, model_family, python, platform}.
brigadestack outcome capture my-skill --run-id latest
brigadestack outcome record my-skill

A content hash cannot see the agent tool a signal was earned under, so the two fingerprints answer different questions: content_fingerprint asks “is this signal about the text that still ships?”, and capability_fingerprint asks “did this signal come from a context like mine?“.

content_fingerprint

The fingerprint is what lets an edited artifact re-earn its score instead of coasting on signals for text that no longer exists.

  • Skill bundles. A skill’s content_fingerprint covers its whole bundle, every file’s path plus content hash, skipping .DS_Store and the skill.json sidecar. Editing a bundled helper invalidates the skill’s signals the same way editing SKILL.md does. A skill whose only content file is SKILL.md hashes to exactly sha256(SKILL.md), byte-identical to the prior scheme, so existing single-file records are never invalidated.
  • Card links. A card’s content_fingerprint folds in the transitive closure of the cards it [[links]], each reachable card’s content hash. Editing a linked card invalidates every card that reaches it. The walk is cycle-safe, deterministic, and tolerant of dead links: a [[missing]] link contributes nothing until that card exists, and a card with no resolvable links hashes to exactly sha256(card content).

Pre-fingerprint records are grandfathered into the current score, so a never-edited artifact keeps its score and rank output stays byte-identical until the artifact is actually edited.

Ranking

brigadestack outcome rank ranks each artifact by a Wilson lower bound, so something that passed twice never outranks something vetted across twenty runs. The default sort drops proven-stale records, those fingerprinted against a different revision, while keeping lifetime counts visible. An edited skill earns its score back against the text that now ships.

brigadestack outcome rank
brigadestack outcome rank --by-capability
brigadestack outcome rank --recency
brigadestack outcome rank --recency --recency-half-life 30

Which checks count

Only a receipt from a registered verifier manifest can score an artifact: run brigadestack work verify run --manifest <id> against a manifest tracked under verify/manifests/. Receipts from ad-hoc --command checks are audit-only evidence. Neither path decides which skill caused a result. You name the skill when you capture or record the outcome.

--by-capability

brigadestack outcome rank --by-capability sorts by “what worked under my current context”. It resolves the current runtime capability and scores each artifact’s content-current records earned under it, pulling a thin cohort toward the pooled rate by deterministic shrinkage (helped + kappa*pooled_rate)/(total + kappa) with a single documented kappa=4.0. The sort key is the shrunk estimate, then on-capability sample size, then the pooled score. Records with no capability fingerprint are grandfathered into the current-capability cohort, so a pre-context ledger’s default output stays byte-identical until signals under a different capability accumulate.

--recency

brigadestack outcome rank --recency weights the score it sorts by over recency-weighted counts, so a signal’s weight halves every half-life and credit earned under a drifted-away environment fades without rewriting the append-only log. The default half-life is 45 days. Override it with --recency-half-life DAYS. Recency applies to the pooled score by default and to the capability-shrunk score with --by-capability, and the two flags compose. It is off by default, so rank output stays byte-identical until you ask for it. The promotion ratchet never uses recency.

Explaining a decision

brigadestack outcome explain prints the full signal trail behind any decision: which run produced each result, the threshold it crossed, and the reversible action taken. It also surfaces a per-capability breakdown, so you can see how an artifact scored under each {harness, model_family, python, platform} cohort rather than only in aggregate.

brigadestack outcome explain my-skill

The ratchet: reconcile

brigadestack outcome reconcile is the gate. It is dry-run by default. With --apply it installs a skill that earned it across your agent tools, or rolls a regressed one back to its last good version.

brigadestack outcome reconcile
brigadestack outcome reconcile --apply

The ratchet is fingerprint-aware. outcome reconcile and outcome fork score the current-fingerprint cohort when deciding, so an edited skill must re-earn install_min_helped against the text that now ships instead of coasting on signals for text that no longer exists. The decision rules, the thresholds, cooldown, and forward-only ratchet, are unchanged. Only the score fed in narrows. Grandfathering keeps a never-edited artifact’s decision, receipt, and output byte-identical to before. Decisions that drop proven-stale evidence carry audit fields: content_fingerprint, lifetime_*, stale_records, and legacy_records, so a promotion that ignored old signals shows exactly which ones and why.

The default rank sort, capability scoring, and recency all read the ledger without touching it. Only reconcile --apply and fork change what is installed, and both are reversible. Schedule brigadestack outcome reconcile in your own cron to run it hands-off. Outcome scoring does not start a daemon.