2 Commits

Include renames

Author	SHA1	Message	Date
3 people	3aedffadc0	fix(docs): comprehensive drift audit — contradictions, broken links, stale refs (#1201) ... A community member reported docs 'have quite a bit of drift and some broken links' and contradictions like 'says don't use bun but also to use bun.' This PR is a top-to-bottom audit + fix across every doc file at the repo root and under docs/. Where docs disagreed with each other, the code was the tie-breaker. ## Categories of fix ### 1. Stale CLI commands (skillpack install → scaffold) `gbrain skillpack install` was retired in v0.36.0.0 (replaced by the scaffold/reference/migrate-fence model). The CLI now errors out with a hint: $ gbrain skillpack install Error: 'gbrain skillpack install' was removed in v0.33. Use 'gbrain skillpack scaffold <name>' instead. But the docs still recommended it: - README.md line 29 — primary install path - docs/INSTALL.md lines 12 — primary install path Both updated to `gbrain skillpack scaffold --all` with the v0.36.0.0 retirement explained inline + the migrate-fence escape hatch for users upgrading from older releases. ### 2. The 'bun install -g vs bun link' contradiction The community member's exact complaint. The drift: - README.md + docs/INSTALL.md: recommended `bun install -g github:garrytan/gbrain` - INSTALL_FOR_AGENTS.md line 29: 'Do NOT use `bun install -g github:garrytan/gbrain`.' Reading the code + CHANGELOG: `bun install -g` IS the canonical path. Bun occasionally blocks the top-level postinstall hook on global installs (issue #218), but the postinstall now prints a loud recovery hint when that happens, and `gbrain doctor` flags `schema_version: 0` and routes users to `gbrain apply-migrations --yes`. The 'do not use' warning was correct in 2024 when the postinstall silently swallowed errors with `|| true`; it's stale now. Reconciled: - INSTALL_FOR_AGENTS.md Step 1: now recommends `bun install -g` as the primary path, documents #218 as a known issue with the recovery command, and keeps `git clone + bun link` as a documented fallback. - AGENTS.md Install (5 min): same reconciliation; clone path is the fallback, not the default. - docs/INSTALL.md CLI standalone: added the #218 callout so the deterministic fallback is one click away when the default fails. ### 3. Broken internal links - README.md → `docs/integrations/voice.md` (file doesn't exist). The real voice recipe lives at `recipes/twilio-voice-brain.md` (Twilio + OpenAI Realtime). Fixed to point there with an accurate one-line summary. - CONTRIBUTING.md → `docs/SQLITE_ENGINE.md` (file doesn't exist; superseded by PGLite per docs/ENGINES.md). Replaced with a paragraph explaining the supersession and pointing at the live ENGINES.md. - docs/GBRAIN_V0.md → `docs/SQLITE_ENGINE.md` (2 references; same supersession). Added a historical-doc banner at the top + rewrote both references to point at the current ENGINES.md. ### 4. Stale API key recommendations INSTALL_FOR_AGENTS.md Step 2 only mentioned OpenAI + Anthropic. As of v0.36.2.0 ZeroEntropy is the default embedding + reranker stack (README opens with this); the agent install guide didn't reflect it. Added `ZEROENTROPY_API_KEY` as the default, kept OpenAI/Voyage as documented fallbacks, noted that keys can live in `~/.gbrain/config.json` (file plane) or env. ### 5. Stale upgrade workflow INSTALL_FOR_AGENTS.md 'Upgrade' section assumed the clone+bun-install model (`cd ~/gbrain && git pull && bun install && gbrain init && gbrain post-upgrade`) and didn't mention `gbrain upgrade` (the single-command path that exists in the CLI today: binary self-update + schema migrations + post-upgrade prompts in one). Split into two paths — `gbrain upgrade` for the bun-install-g case (now the default per Step 1), clone-path for the fallback case. Also fixed AGENTS.md 'Migrate' bullet (was `gbrain apply-migrations` only; now leads with `gbrain upgrade` and keeps apply-migrations as the manual schema-only path). ### 6. Stale cron-workflow INSTALL_FOR_AGENTS.md Step 7 referenced cron docs but didn't mention `gbrain autopilot --install` (the built-in self-maintaining daemon that exists in the CLI today) or `gbrain sync --watch` (continuous loop). Added both as alternatives to platform-cron glue. ### 7. ZeroEntropy version typo docs/INSTALL.md said 'the v0.36.0.0 ZE switch' — ZE landed in v0.36.2.0 (v0.36.0.0 was the skillpack-scaffold retirement). Fixed. ## What I did NOT change - CHANGELOG.md, CLAUDE.md, TODOS.md prose mentions of historical commands like `gbrain skillpack install` are correct as history — they're documenting what was true in past releases. Only forward-looking docs got updated. - The 'broken link' false-positive matches in CHANGELOG / CLAUDE / TODOS are inside code-fence examples or regex patterns (`[Name](people/slug)`, `[a-z0-9](?:[a-z0-9-]{0,30}[a-z0-9])`, `[--json](interrupted)`); they're illustrative syntax, not real links. Leaving alone. - llms.txt / llms-full.txt regenerated via `bun run build:llms` so the agent-fetch documentation map matches the new content. ## Verification - `bun run src/cli.ts --help` cross-checked against every command/flag the install docs reference: init, doctor, apply-migrations, upgrade, post-upgrade, skillpack scaffold/reference/migrate-fence, embed --stale, sync --watch, autopilot --install, dream, integrations list, extract links/timeline, graph-query, query, search modes — all real, all current. - `bun run src/cli.ts skillpack install` confirmed to error out with the retirement hint pointing at scaffold (proves the README guidance was actively misleading users into a dead-end). - Re-ran the broken-internal-link scanner across all root .md + docs/**/*.md; zero real broken links remain (5 residual matches are illustrative syntax inside prose, not actionable links). Co-authored-by: garrytan-agents <agents@garrytan-agents.local>	2026-05-19 05:32:24 -07:00
3 people	cdba533a04	v0.36.2.0 feat: ZeroEntropy as default + zero-based README rewrite (#1136) ... * feat(dims): OpenAI text-embedding-3 Matryoshka range validation (D13) dimsProviderOptions now fail-loud at the embed boundary when the configured embedding_dimensions is outside the model's native range (1..1536 for -small, 1..3072 for -large). Paste-ready fix hint in the AIConfigError.fix field. Closes the silent-HTTP-400 path that would have bit OpenAI-fallback users on v0.36.0.0 ZE-default installs. 16 new test cases in test/ai/dims-openai.test.ts pinning the contract across native-openai and openai-compatible adapter paths. * feat(ai): flip defaults to ZeroEntropy zembed-1 1280d + zerank-2 reranker Default embedding model is now zeroentropyai:zembed-1 at 1280d via Matryoshka. Real-corpus benchmark: 2.2x faster than OpenAI, 2.6x cheaper at regular pricing, wins 11/20 head-to-head queries. 1280 is the closest valid ZE Matryoshka step to the prior OpenAI 1536d default (valid set: 2560/1280/640/320/160/80/40). 1024 (Voyage's step) is NOT on ZE's list — pinned by AIConfigError fail-loud in dims.ts. balanced mode bundle now defaults reranker_enabled=true. zerank-2 reshuffles 60% of top-1 results in benchmarks. Missing-key fail-open contract in src/core/search/rerank.ts handles unauthenticated cases. Opt out with: gbrain config set search.reranker.enabled false Existing tests updated (gateway.test.ts, search-mode.test.ts) and a new test/balanced-reranker-default.test.ts (10 cases) pins the fail- open invariants. * feat(retrieval-upgrade): RetrievalUpgradePlanner + interactive prompt UX New src/core/retrieval-upgrade-planner.ts is the consolidated planner that computes the brain's pending retrieval-upgrade work (chunker bumps + ZE switch) in one pass and applies the schema transition + config updates atomically. Tagged-union ApplyResult enum (D15): 'applied' | 'skipped_already_ applied' | 'skipped_no_work' | 'declined' | 'planned' | 'failed'. No string-parsing reasons. Three config keys (D12): ze_switch_prompt_shown (UI state), ze_switch_requested (user intent), ze_switch_applied (work done). Plus ze_switch_previous_snapshot (JSON, full prior config for --undo per D16) and ze_switch_declined_at (90-day re-ask window). Schema transition (D18) is atomic: DROP indexes + ALTER COLUMN + CREATE INDEX inside a single engine.transaction(). HNSW recreation is part of the same transaction — no silent slow-search window. C3 eligibility logic: ze_switch_offered iff NOT on ZE + NOT declined recently + NOT applied + (legacy default OR >100 pages). C4 cost math: MAX(chunker_pending, dim_pending) not SUM — one re-embed pass invalidates both surfaces simultaneously. New src/core/retrieval-upgrade-prompt.ts wires the planner to a TTY-only interactive prompt with two-line cost split (D10) and privacy callout for the reranker flip. Tests: test/retrieval-upgrade-planner.test.ts (24 cases) pins the state machine. test/asymmetric-encoding-contract.test.ts (6 cases) pins D17: search read path uses gateway.embedQuery() not embed(), asserted via __setEmbedTransportForTests mock. * feat(cli): gbrain ze-switch — manual lever for the ZE switch New gbrain ze-switch CLI with --dry-run, --json, --resume, --force, --undo, --non-interactive, --confirm-reembed, --ignore-missing-key flags. Mirrors the upgrade prompt's UX symmetry: --undo presents a cost-warning before re-embedding back to the prior width. src/cli.ts: dispatch case + CLI_ONLY entry. ze-switch owns its own engine lifecycle (mirrors the doctor pattern). test/ze-switch-cli.test.ts (11 cases): --help, --dry-run, --json, --non-interactive, --ignore-missing-key, --resume, --undo, --confirm-reembed. Uses captureExit harness to test process.exit() paths without breaking the test process. * feat(doctor): ze_embedding_health + embedding_width_consistency checks Two new doctor checks (D-A5): ze_embedding_health: when embedding_model starts with zeroentropyai:, verify ZEROENTROPY_API_KEY is set (env or config). Paste-ready setup hint with the signup URL on failure. embedding_width_consistency: cross-check that the configured embedding_dimensions matches the actual vector(N) column width on content_chunks.embedding. Catches the half-applied switch state (schema migrated but config write crashed) with a paste-ready gbrain ze-switch --resume hint. Wired into runDoctor between reranker_health and the existing sync_freshness checks. Both checks gracefully no-op on non-ZE embedding configs. test/doctor-ze-checks.test.ts (8 cases) pins both checks across happy + missing-key + missing-config + drift paths. Uses withEnv() helper to clear ZEROENTROPY_API_KEY for the no-key path so tests are hermetic against contributor env state. test/e2e/v0_28_5-fix-wave.test.ts + test/openai-compat-multimodal.test.ts: updated to explicit-configure the gateway when the test depends on specific dims that diverge from the v0.36.0.0 default (1280d). * docs: README zero-based rewrite (884 -> 139 lines) + new docs files Strip 4 months of accreted "New in v0.X.Y" hero blocks and reorganize around what gbrain does today. 33 H2s -> 8. The Commands section (136 lines duplicating gbrain --help) moved out; the 6-table skills enumeration collapsed to a one-paragraph capability description with a link to skills/RESOLVER.md. Hero retains load-bearing facts: OpenClaw + Hermes credit, production numbers (17,888 pages / 4,383 people / 723 companies), BrainBench numbers (P@5 49.1% / R@5 97.9% / +31.4 lift), ZE comparison numbers, 30-min install claim. Adds one paragraph announcing the v0.36.0.0 ZE default with the explicit gbrain config set escape for OpenAI/Voyage users. New files: - docs/INSTALL.md: every install path consolidated (agent platform, CLI standalone, MCP server). Thin-client mode covered. - docs/architecture/RETRIEVAL.md: why the hybrid + graph stack works. BrainBench numbers, why each strategy alone fails, the source-aware ranking + intent classification + multi-query expansion story. - docs/ethos/ORIGIN.md: origin story lifted from the old README so the front door stays factual + concrete. test/readme-hero-anchors.test.ts (5 cases) is the D9 regression guard. Five load-bearing strings: OpenClaw, Hermes, ZE, production-numbers regex, P@5/R@5. Light anchors that let voice/ structure evolve but block accidental loss of headline facts. scripts/check-test-real-names.sh: allowlist entries for OpenClaw + Hermes literals in the anchor test (it explicitly asserts those strings appear in README). * chore: bump version and changelog (v0.36.0.0) ZeroEntropy as the new default for embedding (zembed-1 at 1280d via Matryoshka) and reranker (zerank-2 cross-encoder, on by default in balanced mode bundle). README zero-based rewrite (884 -> 139 lines). 3 new docs files. Two new doctor checks. New gbrain ze-switch CLI with --undo for symmetric reversibility. skills/migrations/v0.36.0.0.md tells the agent how to surface the retrieval-upgrade prompt post-upgrade. llms-full.txt regenerated via bun run build:llms. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix(docs): scrub Wintermute from RETRIEVAL.md per privacy rule * chore: rebump version 0.36.0.0 → 0.36.2.0 (queue collision) Three open PRs were claiming v0.36.0.0 (#1130 skillpack, #1139 hindsight, #1136 this PR). Ship-aware queue allocator says this branch lands at v0.36.2.0. Trio audit: VERSION 0.36.2.0 package.json 0.36.2.0 CHANGELOG ## [0.36.2.0] - 2026-05-17 Updates: VERSION, package.json, CHANGELOG header + body refs, README "New default in v0.36.2.0" announcement + credit line, skills/migrations/v0.36.0.0.md renamed to v0.36.2.0.md with frontmatter + body refs updated. llms-full.txt regenerated. * fix(test): pin gateway dim=1536 in cross-file-stateful PGLite tests CI shard 1 reported 10 failures across `query-cache.test.ts` (6) and `consolidate-valid-until.test.ts` (4). Both files hardcode 1536-dim vectors but rely on `PGLiteEngine.initSchema()` to size `vector(__EMBEDDING_DIMS__)` at the right width. Root cause: v0.36.2.0 flipped DEFAULT_EMBEDDING_DIMENSIONS from 1536 to 1280 (ZE Matryoshka step). The gateway module is process-singleton; when ANOTHER test file in the same shard's bun-test process configures the gateway before us, `pglite-engine.ts:216` reads `getEmbeddingDimensions() === 1280` and sizes the schema columns at vector(1280). The hardcoded 1536-dim INSERTs then fail with "expected 1280 dimensions, not 1536". Locally these tests pass in isolation because the gateway falls back through the try/catch at pglite-engine.ts:218 (1536 default). CI runs multiple test files in one process, so cross-file state poisons the schema width. Fix: explicit `resetGateway()` + `configureGateway({embedding_dimensions: 1536, ...})` at the top of `beforeAll`, plus `resetGateway()` in `afterAll`. Pins the schema width regardless of cross-file state. --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>	2026-05-18 21:11:02 -07:00