Port of #1731 (diazMelgarejo) onto current master. gbrain import wrote
~/.gbrain/import-checkpoint.json with the caller's raw dir argument, so
a checkpoint left behind by an interrupted run (e.g. SIGTERM) could carry
"." or a symlinked spelling — an identity that resolves to whatever CWD
the next consumer happens to run from. Downstream tooling that treated
the checkpoint dir as an owned staging boundary could then act on the
wrong directory.
- runImport captures the import target ONCE via resolveImportTargetDir
(resolve + realpathSync) and threads that canonical value through
collection, checkpoint load/save, and resume filtering
- checkpoints are self-describing (schema_version: 1, owner: "gbrain",
kind: "import"); loadCheckpoint tolerates absent metadata (legacy
path-based files) but rejects present-and-wrong metadata and any
relative dir
- checkpoint contract documented in docs/guides/live-sync.md (llms
bundle regenerated)
- test/import-resume.test.ts fixture now realpaths its tmpdir so planted
checkpoints match the canonicalized dir (macOS /var -> /private/var)
Fixes#1728
Co-authored-by: Sinabina <sinabina@Sinabinas-MacBook-Pro-4.local>
Co-authored-by: Lawrence Melgarejo <Lawrence@cyre.me>
Co-authored-by: Claude Fable 5 <noreply@anthropic.com>
* feat(sync): sort files newest-first for faster salience on recent content
Problem: sync processes files in git-diff order (alphabetical), so
meetings/2020-* embeds before meetings/2026-*. After a burst of writes,
new pages can be invisible to search for hours while older pages process first.
Fix: sort addsAndMods descending in both incremental sync and full import.
Brain paths are date-prefixed by convention, so lexicographic descending
naturally prioritizes recent content.
This ensures the most relevant pages become searchable first.
* feat(import): path-based checkpoint resume + sort-newest-first helper
Replace gbrain import's positional `processedIndex` checkpoint with a
path-set checkpoint via `src/core/import-checkpoint.ts`. A file is only
"done" when its processFile returns success — failed files never enter
the set, parallel workers can't lose slow files, and sort-order changes
don't drop the newest N files on resume.
Three bug classes fixed:
- Parallel import + slow worker = silent file drop on crash-resume
- Failed file = checkpoint advanced past it, never retried until manual clear
- Sort-order flip (v0.33.x) = cross-version resume drops newest N files
Old positional checkpoints are detected on first resume and discarded
with a stderr log line. Re-walking is cheap because content_hash
short-circuits unchanged files.
Also extracts the descending-lex sort into src/core/sort-newest-first.ts
so import.ts and sync.ts share a single source of truth.
Tests:
- test/sort-newest-first.test.ts (5 hermetic cases)
- test/import-checkpoint.test.ts (18 unit cases over the helpers)
- test/import-resume.test.ts (refactored — GBRAIN_HOME isolation,
drives runImport against PGLite, 5 integration cases including
SLUG_MISMATCH retry regression)
Includes the original sort-newest-first contribution from
@garrytan-agents's PR #964 (commit 8dbcf6a5).
* chore: bump version and changelog (v0.34.2.0)
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* docs: update project documentation for v0.34.2.0
Add CLAUDE.md Key Files entries for the path-based import checkpoint
work: new entries for src/core/import-checkpoint.ts and
src/core/sort-newest-first.ts, plus a dedicated src/commands/import.ts
entry covering the v0.34.2.0 refactor. Update src/commands/sync.ts
entry to reference sortNewestFirst. Regenerate llms-full.txt.
Co-Authored-By: Claude Opus 4.7 <noreply@anthropic.com>
* fix(tests): swap banned /data/brain placeholder for /tmp/example-brain
scripts/check-privacy.sh banlist includes /data/brain/ (legacy private
OpenClaw fork layout). New test files must not use it — CI privacy
guard caught this on PR #988's first push.
No behavior change. test/import-checkpoint.test.ts is unit-level with
no fs access; the dir string is just an identity marker for the
loadCheckpoint dir-mismatch guard.
---------
Co-authored-by: garrytan-agents <garrytan-agents@users.noreply.github.com>
Co-authored-by: Claude Opus 4.7 <noreply@anthropic.com>
