feat(doctor): tamper-evident skills manifest + freshness guard (#159)

skills/ ships fat-markdown files that agents execute as instructions, but
nothing detected silent edits to them (#159). This adds lightweight tamper
EVIDENCE (a committed sha256 manifest), not a signature system:

- skills/skills.lock.json — committed manifest mapping every bundled file
  under skills/ to its sha256 (sorted relative paths, deterministic JSON,
  trailing newline; excludes itself from its own hash set).
- src/core/skills-integrity.ts — pure, unit-testable core: compute /
  render / verify(dir, manifest) -> { modified, missing, extra }.
- scripts/generate-skills-manifest.ts — regenerator
  (bun run scripts/generate-skills-manifest.ts).
- gbrain doctor check `skills_manifest_integrity` (SKILL group) — WARNS on
  drift, never fails or blocks; fail-safe ok/skip when no manifest is
  present at the resolved skills dir (user workspaces, compiled-binary
  installs far from the repo).
- scripts/check-skills-manifest-fresh.sh — CI freshness guard modeled on
  check-eval-glossary-fresh.sh; wired as `check:skills-manifest` in
  package.json and into bun run verify (run-verify-parallel.sh).
- test/skills-integrity.test.ts — clean / modified / missing / extra /
  self-exclusion / deterministic-render, over tmp-dir fixtures.

Docs: regeneration workflow in docs/TESTING.md + a KEY_FILES.md entry.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-07-27 16:19:41 -07:00
co-authored by Claude Fable 5
parent 032af6e5f7
commit f489243159
11 changed files with 423 additions and 0 deletions
+9
View File
@@ -46,6 +46,15 @@ If a shard wedges (per-shard `GBRAIN_TEST_SHARD_TIMEOUT` cap, default 600s), the
- `tests/heavy/*.sh` → ops-shape shell scripts. Cost minutes per run; NOT in default `bun test`. Run via `bun run test:heavy` or scheduled nightly via `.github/workflows/heavy-tests.yml`. Examples: pg_upgrade matrix (boot legacy brain → walk to head), RSS budget gate (measure peak worker RSS vs committed baseline), read-latency-under-sync (p50/p95/p99 under concurrent writer load), sync lock regression (N concurrent syncs assert 1 winner + N-1 lock-busy + zero leaked `gbrain_cycle_locks` rows). See `tests/heavy/README.md` for when to add a script here vs `*.slow.test.ts`. Files prefixed with `_` (e.g. `tests/heavy/_build_legacy_fixtures.sh`) are helpers/libs invoked by sibling tests — the runner skips them.
- `test/fuzz/*.test.ts` → property-based fuzz harness. Pure-validator targets in `pure-validators.test.ts` are guarded by `scripts/check-fuzz-purity.sh` (in `bun run verify`), which `bun build --target=bun` bundles each target and greps the resulting bundle for banned transitive imports (`node:fs`, `node:child_process`, engine modules). Anything that fails the guard moves to `mixed-validators.test.ts` (still property-tested, but no purity guarantee) or `filesystem-validators.test.ts` (fs-backed, uses temp dirs). Fuzz tests run in the default `bun test` loop because they're fast (~3s for ~12 properties × 1000 runs each).
### Skills-manifest freshness guard
`skills/skills.lock.json` is a committed sha256 inventory of every bundled file under
`skills/` (tamper evidence, not signatures — see `src/core/skills-integrity.ts`).
Any change under `skills/` must regenerate it: `bun run scripts/generate-skills-manifest.ts`.
`scripts/check-skills-manifest-fresh.sh` (`bun run check:skills-manifest`, wired into
`bun run verify`) regenerates to a tmp file and diffs, failing CI on drift; at runtime
`gbrain doctor` reports the same drift as a warn-only `skills_manifest_integrity` check.
### Test-isolation lint and helpers
The cross-file flake class is enforced statically by `scripts/check-test-isolation.sh`, wired into `bun run verify` and `bun run check:all`. Rules (non-serial unit files only; `*.serial.test.ts` and `test/e2e/*` are skipped):