Files
gbrain/test/schema-pack-load-active.serial.test.ts
T
552ff4ed82 v0.41.11.1 ci: cut CI wallclock from 9min to 4.5min (#1457)
* feat(eval-longmemeval): RunOpts.engine seam for shared benchmark brain

Adds optional `engine?: PGLiteEngine` field to RunOpts. When set,
runEvalLongMemEval uses the caller-provided engine and skips the
withBenchmarkBrain wrapper (no fresh PGLite create, no disconnect on
exit). When unset, the production CLI path is unchanged: withBenchmarkBrain
creates and disposes a fresh engine per invocation.

Designed for the test seam that's about to land: one beforeAll-created
brain shared across all 13 runEvalLongMemEval calls in
test/eval-longmemeval-e2e.slow.test.ts, amortizing the ~1-3s PGLite
cold-create cost. runOneQuestion already calls resetTables() as its first
line so per-test isolation is preserved across the shared engine.

Pure additive seam — every existing caller (CLI, current tests that
already create engines via withBenchmarkBrain implicitly) keeps its
current behavior because opts.engine defaults to undefined.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* refactor(test): split eval-longmemeval slow tests + share engine across e2e half

The 884-line test/eval-longmemeval.slow.test.ts was the heaviest single
file in CI at ~359s on the matrix. Split by runEvalLongMemEval usage:

- test/eval-longmemeval.slow.test.ts (trimmed): 8 pure describes, 15 tests.
  Harness lifecycle, resetTables, schema-migration robustness, warm-create
  speed gate, adapter haystackToPages, source-boost guard, loadResumeSet,
  buildByTypeSummary. Local wall: 1.985s, projected CI ~42s.

- test/eval-longmemeval-e2e.slow.test.ts (NEW): 8 e2e describes, 11 tests.
  Every describe that calls runEvalLongMemEval — 13 call sites total.
  Threads a single beforeAll-created PGLite via the v0.41.10 RunOpts.engine
  seam. Local wall: 9.33s (was 15.09s without sharing); projected CI ~196s
  (was ~317s).

- test/helpers/longmemeval-stub.ts (NEW): shared makeStubClient + StubCall.
  Matches the existing test/helpers/ convention (with-env.ts,
  reset-pglite.ts). Single source of truth across the two split files.

- scripts/test-weights.json: replaced 359087ms entry with TWO entries
  (42000ms pure, 196000ms e2e). Projected linearly from local wall-clock
  × 21 CI scaling factor. First post-merge CI run will refine via
  scripts/mine-shard-weights.ts.

Test count is preserved: 15 pure + 11 e2e = 26, matches original file.
No production code changes in this commit — only test reorganization +
opt-in to the RunOpts.engine seam from the previous commit.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* ci(test): bump matrix 6→10, dedicate two slow files, cache bun-install

CI matrix wallclock: ~9 min → ~4.5 min. Three coordinated changes.

1. .github/workflows/test.yml matrix bumped from 6 → 10 shards. Per-shard
   total drops from 532s → 272s. Honest concurrency-budget call: total
   gated jobs go 13 → 18, so 2 concurrent PRs ≈ 36 queued, past the
   GH free-tier ~20 ceiling — single-PR runs unaffected, multi-PR days
   see queue pressure. Worth it for the 4-min CI saving.

2. Two slow files pulled out of the matrix and into their own dedicated
   jobs (sibling to verify, serial-tests):

   - slow-eval-longmemeval runs test/eval-longmemeval-e2e.slow.test.ts
     (~196s after the engine-sharing seam from the previous two commits).
   - slow-entity-resolve-perf runs test/entity-resolve-perf.slow.test.ts
     (~159s, single non-subdivisible perf test). The 60s default bun
     timeout is too tight for this file — bumped to 300000ms.

   scripts/test-shard.sh excludes both via -not -name clauses so the
   matrix sweep doesn't double-run them. Both new jobs wire into
   cache-write.needs and test-status.needs so CI gates on them.

3. actions/cache for ~/.bun/install/cache added to every job that runs
   bun install (test matrix, verify, serial-tests, slow-eval-longmemeval,
   slow-entity-resolve-perf). Keyed on bun.lock hash. Saves ~15s per job
   on cache hit; first-PR push pays full cost, subsequent runs hit cache.

Total CI wallclock now bounded by max(matrix ~4.5min, slow-eval ~3.3min,
slow-entity-resolve-perf ~2.6min) = ~4.5 min. The matrix is back to
being the floor; no single test file dominates a shard.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* chore: v0.41.10.0 — CI wallclock 9min → 4.5min

VERSION + package.json + CHANGELOG entry for the three preceding commits:

  feat(eval-longmemeval): RunOpts.engine seam for shared benchmark brain
  refactor(test): split eval-longmemeval slow tests + share engine across e2e half
  ci(test): bump matrix 6→10, dedicate two slow files, cache bun-install

Net user-visible: CI 'Test' check finishes in ~4.5 min instead of ~9 min.
Net contributor-visible: new RunOpts.engine seam on runEvalLongMemEval for
benchmark suites that want to amortize PGLite cold-create across many calls.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

* fix(test): quarantine hybrid-meta + schema-pack-load-active to serial

The 6→10 matrix shard bump in this branch re-shuffled file distribution
across shard processes. Two pre-existing tests with hidden cross-file
state dependencies surfaced as failures in CI run #77779498812/13:

- test/hybrid-meta.test.ts shard 7: gateway state (configured by some
  other test in the same shard process) survived past the test's
  `delete process.env.OPENAI_API_KEY` call, so the early-return for
  expansion didn't fire and `expansion_applied` stayed true.
- test/schema-pack-load-active.test.ts shard 8: the schema-pack module's
  test-injected locator state was left behind by an earlier file, so
  `loadActivePack` with the default config didn't fall through to the
  bundled gbrain-base path.

Both files pass cleanly solo (verified). The pollution sources are
unidentified — bun's reporter only printed 14 of 71 file headers per
shard log, hiding the polluters. Rather than spelunk for the source,
rename both files to *.serial.test.ts. The serial pass runs them at
--max-concurrency=1 in a process that doesn't share state with the
parallel matrix shards.

Same-wave bookkeeping:
- scripts/check-test-isolation.allowlist: drop test/hybrid-meta.test.ts
  entry (file is now serial, no longer R1-eligible).
- scripts/test-weights.json: rename both weight entries to match the
  new filenames so future matrix LPT runs don't fall back to median.

Companion to a7d029d0/2e1c269e/5a749acb of this branch.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-05-25 15:46:25 -07:00

139 lines
4.9 KiB
TypeScript

// v0.38 active-pack loader smoke tests (T-AP boundary helper).
//
// Covers: 7-tier resolution chain with config-driven inputs, tier-1
// trust gate (remote=true rejects per-call opt), gbrain-base loads from
// bundled path, custom pack via test-injected locator.
import { describe, expect, test, beforeAll, afterAll } from 'bun:test';
import {
loadActivePack,
resolveActivePackNameOnly,
__setPackLocatorForTests,
_resetPackLocatorForTests,
_resetPackCacheForTests,
} from '../src/core/schema-pack/index.ts';
import { withEnv } from './helpers/with-env.ts';
describe('loadActivePack boundary helper', () => {
beforeAll(() => {
// Tests use the real bundled gbrain-base for tier-7 default; per-test
// overrides via __setPackLocatorForTests when injecting synthetic packs.
});
afterAll(() => {
_resetPackLocatorForTests();
_resetPackCacheForTests();
});
test('default resolution loads gbrain-base from bundled path', async () => {
_resetPackCacheForTests();
const pack = await loadActivePack({ cfg: null, remote: false });
expect(pack.manifest.name).toBe('gbrain-base');
expect(pack.manifest.extends).toBeNull();
expect(pack.manifest.page_types.length).toBeGreaterThan(0);
});
test('tier-1 per-call wins when remote=false', async () => {
_resetPackCacheForTests();
const result = resolveActivePackNameOnly({
cfg: null,
remote: false,
perCall: 'custom-pack',
});
expect(result).toEqual({ pack_name: 'custom-pack', source: 'per-call' });
});
test('tier-1 per-call IGNORED when remote=true (D13 trust gate)', () => {
const result = resolveActivePackNameOnly({
cfg: { engine: 'pglite', schema_pack: 'config-pack' } as never,
remote: true,
perCall: 'malicious-pack',
});
// The resolver should land on config-pack (tier-6), not malicious-pack.
expect(result.pack_name).toBe('config-pack');
expect(result.source).toBe('home-config');
});
test('tier-2 env var GBRAIN_SCHEMA_PACK wins over tier-6 home config', async () => {
await withEnv({ GBRAIN_SCHEMA_PACK: 'env-pack' }, async () => {
const result = resolveActivePackNameOnly({
cfg: { engine: 'pglite', schema_pack: 'home-pack' } as never,
remote: false,
});
expect(result.pack_name).toBe('env-pack');
expect(result.source).toBe('env');
});
});
test('tier-3 per-source DB config beats tier-4 brain-wide', () => {
const result = resolveActivePackNameOnly({
cfg: null,
remote: false,
sourceId: 'zion',
perSourceDb: new Map([['zion', 'family-archive']]),
dbConfig: 'main-pack',
});
expect(result.pack_name).toBe('family-archive');
expect(result.source).toBe('per-source-db');
});
test('tier-7 default falls back to gbrain-base when nothing set', () => {
const result = resolveActivePackNameOnly({ cfg: null, remote: false });
expect(result.pack_name).toBe('gbrain-base');
expect(result.source).toBe('default');
});
test('UnknownPackError when configured pack is missing from disk', async () => {
_resetPackCacheForTests();
// Inject locator that returns null for everything (simulates missing pack)
__setPackLocatorForTests(() => null);
try {
await expect(loadActivePack({
cfg: { engine: 'pglite', schema_pack: 'nonexistent' } as never,
remote: false,
})).rejects.toThrow(/unknown schema pack: nonexistent/);
} finally {
_resetPackLocatorForTests();
}
});
test('injected locator overrides default disk path', async () => {
_resetPackCacheForTests();
// Build a minimal pack on the fly; write it to a temp file the
// locator can return.
const { mkdtempSync, writeFileSync } = await import('node:fs');
const { tmpdir } = await import('node:os');
const { join } = await import('node:path');
const dir = mkdtempSync(join(tmpdir(), 'gbrain-test-pack-'));
const yamlPath = join(dir, 'pack.yaml');
writeFileSync(yamlPath, `api_version: gbrain-schema-pack-v1
name: injected-pack
version: 0.1.0
description: test
extends: null
page_types: []
link_types: []
`);
__setPackLocatorForTests(name => name === 'injected-pack' ? yamlPath : null);
try {
const pack = await loadActivePack({
cfg: { engine: 'pglite', schema_pack: 'injected-pack' } as never,
remote: false,
});
expect(pack.manifest.name).toBe('injected-pack');
} finally {
_resetPackLocatorForTests();
}
});
test('pack identity is stable across loads of same manifest', async () => {
_resetPackCacheForTests();
const pack1 = await loadActivePack({ cfg: null, remote: false });
_resetPackCacheForTests();
const pack2 = await loadActivePack({ cfg: null, remote: false });
expect(pack1.identity).toBe(pack2.identity);
expect(pack1.manifest_sha8).toBe(pack2.manifest_sha8);
expect(pack1.alias_closure_hash).toBe(pack2.alias_closure_hash);
});
});