mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
* fix: splitBody and inferType for wiki-style markdown content - splitBody now requires explicit timeline sentinel (<!-- timeline -->, --- timeline ---, or --- directly before ## Timeline / ## History). A bare --- in body text is a markdown horizontal rule, not a separator. This fixes the 83% content truncation @knee5 reported on a 1,991-article wiki where 4,856 of 6,680 wikilinks were lost. - serializeMarkdown emits <!-- timeline --> sentinel for round-trip stability. - inferType extended with /writing/, /wiki/analysis/, /wiki/guides/, /wiki/hardware/, /wiki/architecture/, /wiki/concepts/. Path order is most-specific-first so projects/blog/writing/essay.md → writing, not project. - PageType union extended: writing, analysis, guide, hardware, architecture. Updates test/import-file.test.ts to use the new sentinel. Co-Authored-By: @knee5 (PR #187) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * fix: JSONB double-encode bug on Postgres + parseEmbedding NaN scores Two related Postgres-string-typed-data bugs that PGLite hid: 1. JSONB double-encode (postgres-engine.ts:107,668,846 + files.ts:254): ${JSON.stringify(value)}::jsonb in postgres.js v3 stringified again on the wire, storing JSONB columns as quoted string literals. Every frontmatter->>'key' returned NULL on Postgres-backed brains; GIN indexes were inert. Switched to sql.json(value), which is the postgres.js-native JSONB encoder (Parameter with OID 3802). Affected columns: pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata. page_versions.frontmatter is downstream via INSERT...SELECT and propagates the fix. 2. pgvector embeddings returning as strings (utils.ts): getEmbeddingsByChunkIds returned "[0.1,0.2,...]" instead of Float32Array on Supabase, producing [NaN] cosine scores. Adds parseEmbedding() helper handling Float32Array, numeric arrays, and pgvector string format. Throws loud on malformed vectors (per Codex's no-silent-NaN requirement); returns null for non-vector strings (treated as "no embedding here"). rowToChunk delegates to parseEmbedding. E2E regression test at test/e2e/postgres-jsonb.test.ts asserts jsonb_typeof = 'object' AND col->>'k' returns expected scalar across all 5 affected columns — the test that should have caught the original bug. Runs in CI via the existing pgvector service. Co-Authored-By: @knee5 (PR #187 — JSONB triple-fix) Co-Authored-By: @leonardsellem (PR #175 — parseEmbedding) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: extract wikilink syntax with ancestor-search slug resolution extractMarkdownLinks now handles [[page]] and [[page|Display Text]] alongside standard [text](page.md). For wiki KBs where authors omit leading ../ (thinking in wiki-root-relative terms), resolveSlug walks ancestor directories until it finds a matching slug. Without this, wikilinks under tech/wiki/analysis/ targeting [[../../finance/wiki/concepts/foo]] silently dangled when the correct relative depth was 3 × ../ instead of 2. Co-Authored-By: @knee5 (PR #187) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * feat: gbrain repair-jsonb + v0.12.1 migration + CI grep guard - New gbrain repair-jsonb command. Detects rows where jsonb_typeof(col) = 'string' and rewrites them via (col #>> '{}')::jsonb across 5 affected columns: pages.frontmatter, raw_data.data, ingest_log.pages_updated, files.metadata, page_versions.frontmatter. Idempotent — re-running is a no-op. PGLite engines short-circuit cleanly (the bug never affected the parameterized encode path PGLite uses). --dry-run shows what would be repaired; --json for scripting. - New v0_12_1.ts migration orchestrator. Phases: schema → repair → verify. Modeled on v0_12_0 pattern, registered in migrations/index.ts. Runs automatically via gbrain upgrade / apply-migrations. - CI grep guard at scripts/check-jsonb-pattern.sh fails the build if anyone reintroduces the ${JSON.stringify(x)}::jsonb interpolation pattern. Wired into bun test via package.json. Best-effort static analysis (multi-line and helper-wrapped variants are caught by the E2E round-trip test instead). - Updates apply-migrations.test.ts expectations to account for the new v0.12.1 entry in the registry. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.12.1) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> * docs: update project documentation for v0.12.1 - CLAUDE.md: document repair-jsonb command, v0_12_1 migration, splitBody sentinel contract, inferType wiki subtypes, CI grep guard, new test files (repair-jsonb, migrations-v0_12_1, markdown) - README.md: add gbrain repair-jsonb to ADMIN command reference - INSTALL_FOR_AGENTS.md: fix verification count (6 -> 7), add v0.12.1 upgrade guidance for Postgres brains - docs/GBRAIN_VERIFY.md: add check #8 for JSONB integrity on Postgres-backed brains - docs/UPGRADING_DOWNSTREAM_AGENTS.md: add v0.12.1 section with migration steps, splitBody contract, wiki subtype inference - skills/migrate/SKILL.md: document native wikilink extraction via gbrain extract links (v0.12.1+) Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
60 lines
2.4 KiB
TypeScript
60 lines
2.4 KiB
TypeScript
/**
|
|
* Tests for the v0.12.2 JSONB-double-encode-repair orchestrator.
|
|
*
|
|
* Covers the contract that makes this migration safe to ship:
|
|
* - Registered in the TS registry (so apply-migrations sees it).
|
|
* - Phase functions exported via __testing for unit-level coverage.
|
|
* - Dry-run skips all side-effect phases.
|
|
* - Feature pitch explains what the user can NOW do that they couldn't.
|
|
*
|
|
* Idempotency, repair correctness, and PGLite-no-op behavior are exercised
|
|
* end-to-end against real Postgres in test/e2e/postgres-jsonb.test.ts.
|
|
*/
|
|
|
|
import { describe, test, expect } from 'bun:test';
|
|
|
|
describe('v0.12.2 — JSONB double-encode repair migration', () => {
|
|
test('registered in the TS migration registry', async () => {
|
|
const { migrations, getMigration } = await import('../src/commands/migrations/index.ts');
|
|
const versions = migrations.map(m => m.version);
|
|
expect(versions).toContain('0.12.2');
|
|
const m = getMigration('0.12.2');
|
|
expect(m).not.toBeNull();
|
|
expect(m!.featurePitch.headline).toContain('JSONB');
|
|
expect(typeof m!.orchestrator).toBe('function');
|
|
});
|
|
|
|
test('feature pitch lists the affected columns and the recovery path', async () => {
|
|
const { v0_12_2 } = await import('../src/commands/migrations/v0_12_2.ts');
|
|
const desc = v0_12_2.featurePitch.description ?? '';
|
|
expect(desc).toContain('pages.frontmatter');
|
|
expect(desc).toContain('raw_data.data');
|
|
expect(desc).toContain('ingest_log.pages_updated');
|
|
expect(desc).toContain('files.metadata');
|
|
expect(desc).toContain('page_versions.frontmatter');
|
|
expect(desc).toContain('gbrain sync --full');
|
|
});
|
|
|
|
test('phase functions exported for unit testing', async () => {
|
|
const { __testing } = await import('../src/commands/migrations/v0_12_2.ts');
|
|
expect(typeof __testing.phaseASchema).toBe('function');
|
|
expect(typeof __testing.phaseBRepair).toBe('function');
|
|
expect(typeof __testing.phaseCVerify).toBe('function');
|
|
});
|
|
|
|
test('dry-run skips all side-effect phases', async () => {
|
|
const { v0_12_2 } = await import('../src/commands/migrations/v0_12_2.ts');
|
|
const result = await v0_12_2.orchestrator({
|
|
yes: true,
|
|
dryRun: true,
|
|
noAutopilotInstall: true,
|
|
});
|
|
expect(result.version).toBe('0.12.2');
|
|
expect(result.phases.length).toBeGreaterThanOrEqual(3);
|
|
for (const p of result.phases) {
|
|
expect(p.status).toBe('skipped');
|
|
expect(p.detail).toContain('dry-run');
|
|
}
|
|
});
|
|
});
|