mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-30 03:12:32 +00:00
c250a4d0142e569ab49dacd9bb123788fa546751
3
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
|
1e73e93344 |
v0.22.12 feat: structured error code summary for sync --skip-failed (closes #500) (#518)
* feat: structured error code summary for sync --skip-failed (#500)
When sync encounters per-file failures, the blocked/skip-failed messages
now include a breakdown by error code (SLUG_MISMATCH, YAML_PARSE, etc.)
instead of just a raw count. This makes it immediately obvious *why*
files failed without requiring manual investigation.
Changes:
- Add classifyErrorCode() — maps error messages to ParseValidationCode
- Add summarizeFailuresByCode() — groups failures into sorted code summary
- SyncFailure now carries a 'code' field (backfilled on acknowledge)
- acknowledgeSyncFailures() returns AcknowledgeResult {count, summary}
- sync blocked + skip-failed messages show code breakdown
- doctor sync_failures check shows code breakdown for both unacked and historical
- 12 new tests for classifyErrorCode, summarizeFailuresByCode, and structured returns
Before:
Sync blocked: 2688 file(s) failed to parse.
After:
Sync blocked: 2688 file(s) failed to parse:
SLUG_MISMATCH: 2685
YAML_DUPLICATE_KEY: 3
Closes #500
* test(sync): broaden classifier regexes and pin coverage with 6 new unit tests
Eng review of PR #501 found two ship-blocking gaps in the classifier:
1. Four real production error sites in src/core/import-file.ts emit strings
that bucketed to UNKNOWN — exactly the silent-systemic-failure pattern
that motivated #500 in the first place. Add two regex lines:
FILE_TOO_LARGE — covers import-file.ts:199, 352, 401
SYMLINK_NOT_ALLOWED — covers import-file.ts:347
2. Three existing classifier regexes (MISSING_OPEN, MISSING_CLOSE,
EMPTY_FRONTMATTER) only matched the literal code-name prefix. The actual
message strings emitted by markdown.ts:159-244 (e.g. "Frontmatter must
start with --- on the first non-empty line") wouldn't match. Broaden
each to match production message text. NESTED_QUOTES already worked.
Add 6 unit tests pinning the contract between markdown.ts/import-file.ts
strings and the classifier regex set. If anyone reworks a validator
message, both sides have to move together — the test fails loudly otherwise.
Test count: 22 → 28 in test/sync-failures.test.ts, all green.
* test(e2e): add failure-loop E2E for sync --skip-failed (issue #500 ship-blocker)
The full code path (record → classify → block → skip → doctor render →
second cycle) had only mocked-JSONL unit coverage. For a hotfix that
changes user-visible CLI output and the doctor surface, that's thin.
One comprehensive E2E test covers the loop:
1. First sync of clean repo — succeeds, bookmark advances
2. Add file with bad slug — sync returns 'blocked_by_failures',
bookmark stays put, JSONL has 1 unacked entry coded SLUG_MISMATCH
3. --skip-failed — bookmark advances past the bad commit, entry
transitions to acknowledged, AcknowledgeResult.summary aggregates
4. Second broken file (different path, same code) — sync blocks again,
1 acked + 1 unacked, dedup honors path identity
5. --skip-failed again — both acked, summary correctly counts 2
Hermetic on a developer machine: saves ~/.gbrain/sync-failures.jsonl
before the test, restores it after. Doctor rendering verified by calling
the same primitives doctor.ts uses (loadSyncFailures + summarizeFailuresByCode)
rather than runDoctor() — runDoctor is a CLI entrypoint with stdout/exit
side effects that truncate the test mid-flow.
E2E count: 13 → 14 in test/e2e/sync.test.ts. All 14 pass under real
Postgres + pgvector (gbrain-test-pg/pgvector:pg16).
* v0.22.12: structured error code summary for sync --skip-failed
Closes issue #500. PR #501 by @wintermute is the foundation (cherry-picked
as
|
||
|
|
27eb87f1f4 |
feat: slugify file paths with spaces and special characters (v0.5.1) (#29)
* feat: slugify file paths with spaces and special characters Apple Notes files (e.g., "2017-05-03 ohmygreen.md") now get clean, URL-safe slugs instead of raw filenames with spaces. Spaces become hyphens, special chars are stripped, accented chars normalize to ASCII. Both import (inferSlug) and sync (pathToSlug) pipelines now use the same slugifyPath() function, eliminating the case-preservation mismatch. * feat: one-time migration to slugify existing page slugs Extends Migration interface with optional TypeScript handler for application-level data transformations. Adds version 2 migration that renames all existing slugs to their slugified form, including link rewriting. Collision handling via try/catch + warning. * test: slugify unit tests, E2E tests, and updated expectations 22 new unit tests for slugifySegment and slugifyPath covering spaces, special chars, unicode, dots, empty segments, and all 4 bug report examples. Updated pathToSlug tests for new lowercase behavior. Updated E2E tests for slugified Apple Notes slugs. Added 2 new E2E tests for space-named file import and sync. * chore: bump version and changelog (v0.5.1) Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com> * docs: fix changelog example to show directory with spaces --------- Co-authored-by: Claude Opus 4.6 <noreply@anthropic.com> |
||
|
|
e9f3c9c24d |
docs: live sync setup + verification runbook + API key loading (#24)
* docs: add SKILLPACK Section 18 — Live Sync (MUST ADD) Contract-first guide for keeping the vector DB in sync with the brain repo. Documents the pooler prerequisite (Session mode required for transactions), sync + embed primitives, four example approaches (cron, --watch, webhook, git hook), isSyncable exclusions, silent skip warning, and OpenClaw/Hermes cron registration examples. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add GBRAIN_VERIFY.md installation verification runbook Six-check runbook: schema (doctor), skillpack loaded, auto-update, live sync (coverage check + embed check + end-to-end push-and-search test), embedding coverage, brain-first lookup protocol. Emphasizes "sync ran" != "sync worked" — the real test is searching for corrected text after a push. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add setup Phases H (Live Sync) and I (Verification) Phase H: MUST ADD live sync setup — pooler prerequisite check, automatic sync configuration (agent picks approach), sync+embed chaining, coverage verification. Phase I: run GBRAIN_VERIFY.md end-to-end before declaring setup complete. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add install steps 8-9 (live sync + verification) Step 8: set up automatic sync with SKILLPACK Section 18 reference. Step 9: run GBRAIN_VERIFY.md runbook. Add GBRAIN_VERIFY.md to docs section. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add API key loading instructions to CLAUDE.md Source ~/.zshrc before running Tier 2 tests so OPENAI_API_KEY and ANTHROPIC_API_KEY are available. Without this, embedding and skills tests skip silently. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * chore: bump version to v0.5.0 Live sync, verification runbook, API key loading instructions. Version markers updated in SKILLPACK and RECOMMENDED_SCHEMA. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add anti-hand-roll rule to skill routing in CLAUDE.md Explicitly prohibit manually running git commit + push + gh pr create when /ship is available. /ship handles VERSION, CHANGELOG, document-release, reviews, and coverage audit. Hand-rolling skips all of these. Added "commit and ship" / "push and ship" variants to the ship routing rule. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: changelog voice rule + rewrite 0.5.0 changelog to sell the upgrade CLAUDE.md: add changelog voice guidance — lead with benefits, not implementation details. Make users want to upgrade. CHANGELOG: rewrite 0.5.0 entries from dry feature descriptions to capability-focused bullets ("your brain never falls behind" not "SKILLPACK Section 18 added"). SKILLPACK Section 17: update the auto-update message template to instruct agents to sell the upgrade, not just summarize the diff. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: add v0.5.0 migration directive for live sync + verification Agents upgrading from v0.4.x will automatically: check their pooler connection string, set up automatic sync, and run the verification runbook. Without this migration file, upgrading agents would learn about live sync (by re-reading Section 18) but wouldn't set it up. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: sharpen migration file guidance in CLAUDE.md Replace vague "requires agent action" with concrete trigger list: new setup steps existing users don't have, MUST ADD skillpack sections, schema changes, deprecated commands, new verification steps, new crons. Add the key test: "if an existing user upgrades and does nothing else, will their brain work worse?" Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: make Section 17 upgrade flow work for direct user requests Section 17 was structured as a cron-initiated flow only. An agent handling "upgrade gbrain" might just run the command and stop, missing the post-upgrade steps where the value is (re-read skills, run migrations, schema sync). Added explicit entry point for direct upgrade requests. Made Steps 2-4 more concrete about where to find files and why migrations can't be skipped. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * test: add E2E sync tests — git-to-DB pipeline (11 tests) Tests the full sync lifecycle against real Postgres+pgvector: - First sync imports all pages from a git repo - Second sync with no changes returns up_to_date - Incremental sync picks up new files (add → commit → sync → verify) - Incremental sync picks up modifications — THE CRITICAL TEST: corrected text appears in DB and keyword search after sync - Incremental sync handles deletes - Non-syncable files are excluded (README, .raw/, ops/) - Sync state (last_commit, last_run) persisted to config - Sync logged to ingest_log - --full reimports everything - --dry-run shows changes without applying Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * docs: strengthen CLAUDE.md to always run ALL test tiers Replace passive "source zshrc" suggestion with ALWAYS directive. Explicitly state that "run all tests" means ALL tiers including Tier 2 with API keys. Do not skip Tier 2 just because keys need loading. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> * fix: Tier 2 E2E tests — correct openclaw CLI invocation The tests used `openclaw -p` which doesn't exist. The correct command is `openclaw agent --local --agent <id> --message <prompt>`. Also fixed JSON output parsing (structured JSON goes to stderr, not stdout — use non-JSON mode instead). Fixed ingest test to assert on agent response text rather than test DB state (the agent writes to its own configured DB, not the ephemeral test DB). 82 tests pass, 0 fail, 0 skip across all 5 E2E files. Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com> |