mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
* fix: splitBody and inferType for wiki-style markdown content - splitBody now requires explicit timeline sentinel (<!-- timeline -->, --- timeline ---, or --- directly before ## Timeline / ## History). A bare --- in body text is a markdown horizontal rule, not a separator. This fixes the 83% content truncation @knee5 reported on a 1,991-article wiki where 4,856 of 6,680 wikilinks were lost. - serializeMarkdown emits <!-- timeline --> sentinel for round-trip stability. - inferType extended with /writing/, /wiki/analysis/, /wiki/guides/, /wiki/hardware/, /wiki/architecture/, /wiki/concepts/. Path order is most-specific-first so projects/blog/writing/essay.md → writing, not project. - PageType union extended: writing, analysis, guide, hardware, architecture. Updates test/import-file.test.ts to use the new sentinel. Co-Authored-By: @knee5 (PR #187) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: JSONB double-encode bug on Postgres + parseEmbedding NaN scores Two related Postgres-string-typed-data bugs that PGLite hid: 1. JSONB double-encode (postgres-engine.ts:107,668,846 + files.ts:254): ${JSON.stringify(value)}::jsonb in postgres.js v3 stringified again on the wire, storing JSONB columns as quoted string literals. Every frontmatter->>'key' returned NULL on Postgres-backed brains; GIN indexes were inert. Switched to sql.json(value), which is the postgres.js-native JSONB encoder (Parameter with OID 3802). Affected columns: pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata. page_versions.frontmatter is downstream via INSERT...SELECT and propagates the fix. 2. pgvector embeddings returning as strings (utils.ts): getEmbeddingsByChunkIds returned "[0.1,0.2,...]" instead of Float32Array on Supabase, producing [NaN] cosine scores. Adds parseEmbedding() helper handling Float32Array, numeric arrays, and pgvector string format. Throws loud on malformed vectors (per Codex's no-silent-NaN requirement); returns null for non-vector strings (treated as "no embedding here"). rowToChunk delegates to parseEmbedding. E2E regression test at test/e2e/postgres-jsonb.test.ts asserts jsonb_typeof = 'object' AND col->>'k' returns expected scalar across all 5 affected columns — the test that should have caught the original bug. Runs in CI via the existing pgvector service. Co-Authored-By: @knee5 (PR #187 — JSONB triple-fix) Co-Authored-By: @leonardsellem (PR #175 — parseEmbedding) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: extract wikilink syntax with ancestor-search slug resolution extractMarkdownLinks now handles [[page]] and [[page|Display Text]] alongside standard [text](page.md). For wiki KBs where authors omit leading ../ (thinking in wiki-root-relative terms), resolveSlug walks ancestor directories until it finds a matching slug. Without this, wikilinks under tech/wiki/analysis/ targeting [[../../finance/wiki/concepts/foo]] silently dangled when the correct relative depth was 3 × ../ instead of 2. Co-Authored-By: @knee5 (PR #187) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: gbrain repair-jsonb + v0.12.1 migration + CI grep guard - New gbrain repair-jsonb command. Detects rows where jsonb_typeof(col) = 'string' and rewrites them via (col #>> '{}')::jsonb across 5 affected columns: pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter. Idempotent — re-running is a no-op. PGLite engines short-circuit cleanly (the bug never affected the parameterized encode path PGLite uses). --dry-run shows what would be repaired; --json for scripting. - New v0_12_1.ts migration orchestrator. Phases: schema → repair → verify. Modeled on v0_12_0 pattern, registered in migrations/index.ts. Runs automatically via gbrain upgrade / apply-migrations. - CI grep guard at scripts/check-jsonb-pattern.sh fails the build if anyone reintroduces the ${JSON.stringify(x)}::jsonb interpolation pattern. Wired into bun test via package.json. Best-effort static analysis (multi-line and helper-wrapped variants are caught by the E2E round-trip test instead). - Updates apply-migrations.test.ts expectations to account for the new v0.12.1 entry in the registry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.12.1) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: update project documentation for v0.12.1 - CLAUDE.md: document repair-jsonb command, v0_12_1 migration, splitBody sentinel contract, inferType wiki subtypes, CI grep guard, new test files (repair-jsonb, migrations-v0_12_1, markdown) - README.md: add gbrain repair-jsonb to ADMIN command reference - INSTALL_FOR_AGENTS.md: fix verification count (6 -> 7), add v0.12.1 upgrade guidance for Postgres brains - docs/GBRAIN_VERIFY.md: add check #8 for JSONB integrity on Postgres-backed brains - docs/UPGRADING_DOWNSTREAM_AGENTS.md: add v0.12.1 section with migration steps, splitBody contract, wiki subtype inference - skills/migrate/SKILL.md: document native wikilink extraction via gbrain extract links (v0.12.1+) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
165 lines
6.6 KiB
TypeScript
165 lines
6.6 KiB
TypeScript
/**
|
|
* Tests for `gbrain apply-migrations` — the migration runner CLI.
|
|
*
|
|
* Unit-scope: exercises the pure helpers (parseArgs, indexCompleted, buildPlan,
|
|
* statusForVersion). End-to-end integration against real orchestrators is
|
|
* covered by test/e2e/migration-flow.test.ts (Lane C-5).
|
|
*/
|
|
|
|
import { describe, test, expect } from 'bun:test';
|
|
import { __testing } from '../src/commands/apply-migrations.ts';
|
|
import type { CompletedMigrationEntry } from '../src/core/preferences.ts';
|
|
|
|
const { parseArgs, indexCompleted, buildPlan, statusForVersion } = __testing;
|
|
|
|
describe('parseArgs', () => {
|
|
test('default flags', () => {
|
|
const a = parseArgs([]);
|
|
expect(a.list).toBe(false);
|
|
expect(a.dryRun).toBe(false);
|
|
expect(a.yes).toBe(false);
|
|
expect(a.nonInteractive).toBe(false);
|
|
expect(a.mode).toBeUndefined();
|
|
expect(a.specificMigration).toBeUndefined();
|
|
expect(a.hostDir).toBeUndefined();
|
|
expect(a.noAutopilotInstall).toBe(false);
|
|
});
|
|
|
|
test('--list / --dry-run / --yes / --non-interactive', () => {
|
|
expect(parseArgs(['--list']).list).toBe(true);
|
|
expect(parseArgs(['--dry-run']).dryRun).toBe(true);
|
|
expect(parseArgs(['--yes']).yes).toBe(true);
|
|
expect(parseArgs(['--non-interactive']).nonInteractive).toBe(true);
|
|
});
|
|
|
|
test('--mode accepts valid values', () => {
|
|
expect(parseArgs(['--mode', 'always']).mode).toBe('always');
|
|
expect(parseArgs(['--mode', 'pain_triggered']).mode).toBe('pain_triggered');
|
|
expect(parseArgs(['--mode', 'off']).mode).toBe('off');
|
|
});
|
|
|
|
test('--migration and --host-dir parse values', () => {
|
|
const a = parseArgs(['--migration', '0.11.0', '--host-dir', '/tmp/abc']);
|
|
expect(a.specificMigration).toBe('0.11.0');
|
|
expect(a.hostDir).toBe('/tmp/abc');
|
|
});
|
|
|
|
test('--no-autopilot-install flips flag', () => {
|
|
expect(parseArgs(['--no-autopilot-install']).noAutopilotInstall).toBe(true);
|
|
});
|
|
|
|
test('--help sets help flag', () => {
|
|
expect(parseArgs(['--help']).help).toBe(true);
|
|
expect(parseArgs(['-h']).help).toBe(true);
|
|
});
|
|
});
|
|
|
|
describe('indexCompleted + statusForVersion', () => {
|
|
test('no entries → pending', () => {
|
|
const idx = indexCompleted([]);
|
|
expect(statusForVersion('0.11.0', idx)).toBe('pending');
|
|
});
|
|
|
|
test('one complete entry → complete', () => {
|
|
const entries: CompletedMigrationEntry[] = [
|
|
{ version: '0.11.0', status: 'complete', mode: 'always' },
|
|
];
|
|
const idx = indexCompleted(entries);
|
|
expect(statusForVersion('0.11.0', idx)).toBe('complete');
|
|
});
|
|
|
|
test('only partial entries → partial', () => {
|
|
const entries: CompletedMigrationEntry[] = [
|
|
{ version: '0.11.0', status: 'partial', apply_migrations_pending: true },
|
|
];
|
|
const idx = indexCompleted(entries);
|
|
expect(statusForVersion('0.11.0', idx)).toBe('partial');
|
|
});
|
|
|
|
test('partial then complete → complete (stopgap then v0.11.1 apply-migrations)', () => {
|
|
const entries: CompletedMigrationEntry[] = [
|
|
{ version: '0.11.0', status: 'partial', apply_migrations_pending: true },
|
|
{ version: '0.11.0', status: 'complete', mode: 'always' },
|
|
];
|
|
const idx = indexCompleted(entries);
|
|
expect(statusForVersion('0.11.0', idx)).toBe('complete');
|
|
});
|
|
|
|
test('only looks at the queried version', () => {
|
|
const entries: CompletedMigrationEntry[] = [
|
|
{ version: '0.10.0', status: 'complete' },
|
|
];
|
|
const idx = indexCompleted(entries);
|
|
expect(statusForVersion('0.11.0', idx)).toBe('pending');
|
|
expect(statusForVersion('0.10.0', idx)).toBe('complete');
|
|
});
|
|
});
|
|
|
|
describe('buildPlan — diff against completed + installed VERSION', () => {
|
|
test('fresh install (no entries) — v0.11.0 is pending when installed ≥ 0.11.0', () => {
|
|
const idx = indexCompleted([]);
|
|
const plan = buildPlan(idx, '0.11.1');
|
|
expect(plan.applied).toEqual([]);
|
|
expect(plan.partial).toEqual([]);
|
|
expect(plan.pending.map(m => m.version)).toContain('0.11.0');
|
|
// v0.12.0 (Knowledge Graph) and v0.12.2 (JSONB repair) are registered but
|
|
// installed VERSION is 0.11.1, so they land in skippedFuture until the
|
|
// binary catches up.
|
|
expect(plan.skippedFuture.map(m => m.version)).toEqual(['0.12.0', '0.12.2']);
|
|
});
|
|
|
|
test('already applied → v0.11.0 lands in `applied` bucket, not pending', () => {
|
|
const idx = indexCompleted([{ version: '0.11.0', status: 'complete' }]);
|
|
const plan = buildPlan(idx, '0.11.1');
|
|
expect(plan.applied.map(m => m.version)).toContain('0.11.0');
|
|
expect(plan.pending).toEqual([]);
|
|
});
|
|
|
|
test('stopgap wrote partial → v0.11.0 lands in `partial` bucket (resumable)', () => {
|
|
const idx = indexCompleted([
|
|
{ version: '0.11.0', status: 'partial', apply_migrations_pending: true },
|
|
]);
|
|
const plan = buildPlan(idx, '0.11.1');
|
|
expect(plan.partial.map(m => m.version)).toContain('0.11.0');
|
|
expect(plan.applied).toEqual([]);
|
|
expect(plan.pending).toEqual([]);
|
|
});
|
|
|
|
test('Codex H9 regression: installed older than migration → skippedFuture, not skipped silently', () => {
|
|
// Running a v0.10.x binary that somehow loaded a v0.11.0 migration registry:
|
|
// migration is skippedFuture (wait for a newer install), NOT ignored.
|
|
const idx = indexCompleted([]);
|
|
const plan = buildPlan(idx, '0.10.5');
|
|
expect(plan.skippedFuture.map(m => m.version)).toContain('0.11.0');
|
|
expect(plan.pending).toEqual([]);
|
|
});
|
|
|
|
test('Codex H9 regression: installed > migration version → still runs (not skipped)', () => {
|
|
// This is the critical bug Codex caught: the plan was "apply when version >
|
|
// installed", which would SKIP v0.11.0 when running v0.11.1. The correct
|
|
// rule is "apply when not in completed.jsonl AND version ≤ installed".
|
|
const idx = indexCompleted([]);
|
|
const plan = buildPlan(idx, '0.12.0');
|
|
expect(plan.pending.map(m => m.version)).toContain('0.11.0');
|
|
// v0.12.2 was added later (JSONB repair); installed=0.12.0 means it
|
|
// belongs in skippedFuture, not pending. v0.11.0 and v0.12.0 stay
|
|
// pending despite being ≤ installed — that is the H9 invariant.
|
|
expect(plan.skippedFuture.map(m => m.version)).toEqual(['0.12.2']);
|
|
});
|
|
|
|
test('--migration filter narrows to one version', () => {
|
|
const idx = indexCompleted([]);
|
|
const plan = buildPlan(idx, '0.11.1', '0.11.0');
|
|
expect(plan.pending.map(m => m.version)).toEqual(['0.11.0']);
|
|
});
|
|
|
|
test('--migration filter for unknown version → empty plan', () => {
|
|
const idx = indexCompleted([]);
|
|
const plan = buildPlan(idx, '0.11.1', '99.99.99');
|
|
expect(plan.applied).toEqual([]);
|
|
expect(plan.pending).toEqual([]);
|
|
expect(plan.partial).toEqual([]);
|
|
expect(plan.skippedFuture).toEqual([]);
|
|
});
|
|
});
|