mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
v0.40.6.0 Phase 1 foundations — pack-lock + mutate-audit + cache invalidation + lint rules + best-effort
Six new primitives that Phase 2's withMutation skeleton (next commit) depends on.
No consumers yet; all callers wire up in Phase 4. Foundations ship first per
codex C1 phase-ordering finding from /plan-eng-review.
1.1 pack-lock.ts (18 cases)
Atomic acquire via openSync(path, 'wx') = O_CREAT|O_EXCL. Kernel-level
atomic, NO TOCTOU window. Codex C8 caught that page-lock.ts:79+96 has
existsSync+writeFileSync (TOCTOU) — we deliberately do NOT copy it.
Stale detection via TTL (60s default) + kill(pid, 0) liveness probe.
TTL refresh every 10s while withPackLock(fn) runs so long DB-aware
lint/stats on big brains don't go stale. --force = "steal stale lock"
(NOT "skip locking"). Lock path per-pack so two packs never block.
1.2 mutate-audit.ts (13 cases)
ISO-week JSONL at ~/.gbrain/audit/schema-mutations-YYYY-Www.jsonl.
Privacy redacted per D20: type names → sha8, prefixes → first slug
segment only. Matches candidate-audit.ts privacy posture. Both verbose
surfaces gate on GBRAIN_SCHEMA_AUDIT_VERBOSE=1 (same env). Logs BOTH
success AND failure events so Phase 9's schema_pack_writability doctor
check has signal to read (closes codex C11). summarizeMutations()
primitive shipped for cross-surface parity between doctor + future
audit CLI.
1.3 registry.ts cache invalidation + stat-mtime TTL (10 cases)
invalidatePackCache(name?) walks the extends-chain reverse-graph
(every cached entry whose chain contains name is evicted). This is the
codex C6 fix — pre-v0.40.6, editing a parent pack silently left
children stale because cache identity was child-bytes-only. New
per-name CacheEntry tracks the file-stat snapshot of every file in
the extends chain. tryCachedPack(name) is the TTL-gated fast path:
inside STAT_TTL_MS (1000ms default, env GBRAIN_PACK_STAT_TTL_MS)
returns cached without statting. Outside the window: stats every file
and cascade-invalidates on any mtime change (D11 cross-process
detection). resolvePack reference-equality preserved on byte-identical
re-build. ASCII state-machine diagram in file header (D9).
1.4 best-effort.ts (4 cases)
loadActivePackBestEffort(ctx) returns ResolvedPack | null. Single
source of truth for the 4 T1.5 wiring sites (Phase 8). null means
"EMPTY FILTER" semantics, NOT "fall back to hardcoded defaults" —
pack-load failure must be loud, per D4. Never throws.
1.5 lint-rules.ts (35 cases)
11 pure rule functions extracted from CLI handlers per codex C13/D16.
9 file-plane rules + 2 DB-aware rules (extractable_empty_corpus, mutation_count_anomaly).
Phase 2 withMutation pre-write gate composes file-plane subset.
runAllLintRules() returns {ok, errors, warnings} structured report
ready for CLI + MCP.
1.6 query-cache-invalidator.ts (4 cases)
invalidateQueryCache(engine, sourceId?) DELETEs query_cache rows so
cached search results bound to old page types don't survive a
schema mutation. Reuses SemanticQueryCache.clear() so we don't
reinvent the PGLite+Postgres parity. Codex C9 fix.
Tests: 84 new cases across 6 test files. All 153 schema-pack tests green.
Plan: ~/.claude/plans/system-instruction-you-are-working-recursive-thacker.md
Closes: half of T2-T7 from the plan's Implementation Tasks JSONL.
Successor to: closed PR #1321 (community PR; author garrytan-agents).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
a19ee8bafe
commit
e8ea179256
@@ -0,0 +1,59 @@
|
||||
// v0.40.6.0 Schema Cathedral v3 — best-effort active pack loader.
|
||||
//
|
||||
// Single source of truth for the T1.5 wiring sites (whoknows,
|
||||
// find-experts, facts/eligibility, enrichment-service). All four call
|
||||
// sites consume this helper so the empty-filter fallback contract lives
|
||||
// in ONE place. Without this helper, the four sites would each open-code
|
||||
// their own `try { load pack } catch { ... }` block, and one of them
|
||||
// WILL drift to silently use hardcoded defaults — the bug class D4
|
||||
// closed.
|
||||
//
|
||||
// Contract (D4 from /plan-eng-review):
|
||||
// - Pack load succeeds → return the ResolvedPack.
|
||||
// - Pack load fails (any reason: corrupt file, missing pack, federation
|
||||
// divergence, trust-gate reject) → return null.
|
||||
// - Caller MUST interpret null as "EMPTY FILTER" semantics. A null
|
||||
// return is NOT a license to fall back to hardcoded defaults like
|
||||
// ['person', 'company']; that silently re-introduces types the
|
||||
// user packed out.
|
||||
//
|
||||
// The empty-filter contract is the load-bearing design choice. Pack-load
|
||||
// failure should be loud (query returns empty results, agent debugs the
|
||||
// pack-load problem) — not silent (results look normal but contradict
|
||||
// user intent).
|
||||
|
||||
import { loadConfig } from '../config.ts';
|
||||
import type { OperationContext } from '../operations.ts';
|
||||
import { loadActivePack } from './load-active.ts';
|
||||
import type { ResolvedPack } from './registry.ts';
|
||||
|
||||
/**
|
||||
* Best-effort loader for the active schema pack. Returns null on any
|
||||
* failure path so callers can apply empty-filter semantics.
|
||||
*
|
||||
* NEVER throws. Never logs to stderr (callers don't need the noise on
|
||||
* routine queries; the underlying pack-load errors surface through
|
||||
* `gbrain doctor`'s schema_pack_coverage / schema_pack_writability
|
||||
* checks).
|
||||
*
|
||||
* @example
|
||||
* // In whoknows.ts (T1.5 wiring site):
|
||||
* const pack = await loadActivePackBestEffort(ctx);
|
||||
* const types = pack
|
||||
* ? expertTypesFromPack(pack)
|
||||
* : []; // EMPTY filter, NOT hardcoded defaults
|
||||
* const results = await search(query, { types });
|
||||
*/
|
||||
export async function loadActivePackBestEffort(
|
||||
ctx: OperationContext,
|
||||
): Promise<ResolvedPack | null> {
|
||||
try {
|
||||
return await loadActivePack({
|
||||
cfg: loadConfig(),
|
||||
remote: ctx.remote ?? true,
|
||||
sourceId: ctx.sourceId,
|
||||
});
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
@@ -68,6 +68,7 @@ export {
|
||||
export {
|
||||
EXTENDS_DEPTH_WARN,
|
||||
EXTENDS_DEPTH_HARD_CAP,
|
||||
STAT_TTL_MS_DEFAULT,
|
||||
ExtendsChainTooDeepError,
|
||||
UnknownPackError,
|
||||
type ResolvedPack,
|
||||
@@ -75,7 +76,11 @@ export {
|
||||
type ResolutionResult,
|
||||
resolveActivePackName,
|
||||
resolvePack,
|
||||
tryCachedPack,
|
||||
invalidatePackCache,
|
||||
_resetPackCacheForTests,
|
||||
_cacheSizeForTests,
|
||||
_cacheNamesForTests,
|
||||
} from './registry.ts';
|
||||
|
||||
export {
|
||||
@@ -112,3 +117,33 @@ export {
|
||||
enrichableTypesFromPack,
|
||||
rubricNameForType,
|
||||
} from './enrichable.ts';
|
||||
|
||||
// v0.40.6.0 Schema Cathedral v3 surface:
|
||||
export { loadActivePackBestEffort } from './best-effort.ts';
|
||||
|
||||
export {
|
||||
type MutationOp,
|
||||
type MutationActor,
|
||||
type MutationOutcome,
|
||||
type MutationAuditRecord,
|
||||
type LogMutationOpts,
|
||||
type LogMutationFailureOpts,
|
||||
type MutationSummary,
|
||||
computeMutateAuditPath,
|
||||
logMutationSuccess,
|
||||
logMutationFailure,
|
||||
readRecentMutations,
|
||||
summarizeMutations,
|
||||
} from './mutate-audit.ts';
|
||||
|
||||
export {
|
||||
DEFAULT_LOCK_TTL_MS,
|
||||
REFRESH_INTERVAL_MS,
|
||||
type LockOutcome,
|
||||
type PackLockOpts,
|
||||
type LockFileRecord,
|
||||
PackLockBusyError,
|
||||
isLockStale,
|
||||
acquirePackLock,
|
||||
withPackLock,
|
||||
} from './pack-lock.ts';
|
||||
|
||||
@@ -0,0 +1,388 @@
|
||||
// v0.40.6.0 Schema Cathedral v3 — pure lint rule functions.
|
||||
//
|
||||
// Extracted from CLI handlers per codex C13 / D16 so:
|
||||
// - Phase 2's `withMutation` validation gate can run pre-write checks.
|
||||
// - Phase 5's `gbrain schema lint` CLI verb wires to the same rules.
|
||||
// - Phase 7's `schema_lint` MCP op composes them without printing to stdout.
|
||||
//
|
||||
// Each rule is a pure function: takes a manifest (+ optional opts) and
|
||||
// returns an array of issues. No I/O except for the two DB-aware rules,
|
||||
// which gate on opts.engine being present (file-plane callers omit).
|
||||
//
|
||||
// Issue shape mirrors the StructuredAgentError envelope from
|
||||
// src/core/errors.ts so JSON output is consistent across CLI + MCP.
|
||||
|
||||
import type { SchemaPackManifest } from './manifest-v1.ts';
|
||||
import type { BrainEngine } from '../engine.ts';
|
||||
import { readRecentMutations } from './mutate-audit.ts';
|
||||
|
||||
export type LintSeverity = 'error' | 'warning';
|
||||
|
||||
export interface LintIssue {
|
||||
rule: string;
|
||||
severity: LintSeverity;
|
||||
message: string;
|
||||
/** Source pack name; useful when linting extends-chain in v0.41+. */
|
||||
pack: string;
|
||||
/** Affected type name when applicable. */
|
||||
type?: string;
|
||||
/** Affected link verb when applicable. */
|
||||
link?: string;
|
||||
/** Paste-ready hint command, when one exists. */
|
||||
hint?: string;
|
||||
}
|
||||
|
||||
export interface LintOpts {
|
||||
/** When set, DB-aware rules run. File-plane callers omit. */
|
||||
engine?: BrainEngine;
|
||||
/** Limit scan window for audit-aware rules. Default 7 days. */
|
||||
daysBack?: number;
|
||||
}
|
||||
|
||||
export type LintRule = (manifest: SchemaPackManifest, opts?: LintOpts) =>
|
||||
| LintIssue[]
|
||||
| Promise<LintIssue[]>;
|
||||
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
// File-plane rules (synchronous, no engine)
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
|
||||
export const aliasShadowsType: LintRule = (manifest) => {
|
||||
const issues: LintIssue[] = [];
|
||||
const typeNames = new Set(manifest.page_types.map((t) => t.name));
|
||||
for (const t of manifest.page_types) {
|
||||
for (const a of t.aliases) {
|
||||
if (typeNames.has(a) && a !== t.name) {
|
||||
issues.push({
|
||||
rule: 'alias_shadows_type',
|
||||
severity: 'error',
|
||||
message: `type '${t.name}' declares alias '${a}' which is also a declared page_type name; query closure collision`,
|
||||
pack: manifest.name,
|
||||
type: t.name,
|
||||
hint: `remove alias '${a}' from type '${t.name}' OR rename one of them`,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
return issues;
|
||||
};
|
||||
|
||||
export const aliasDeclaredByTwoTypes: LintRule = (manifest) => {
|
||||
const issues: LintIssue[] = [];
|
||||
const aliasToTypes = new Map<string, string[]>();
|
||||
for (const t of manifest.page_types) {
|
||||
for (const a of t.aliases) {
|
||||
const list = aliasToTypes.get(a) ?? [];
|
||||
list.push(t.name);
|
||||
aliasToTypes.set(a, list);
|
||||
}
|
||||
}
|
||||
for (const [alias, owners] of aliasToTypes) {
|
||||
if (owners.length > 1) {
|
||||
issues.push({
|
||||
rule: 'alias_declared_by_two_types',
|
||||
severity: 'error',
|
||||
message: `alias '${alias}' is declared by ${owners.length} types: ${owners.join(', ')}`,
|
||||
pack: manifest.name,
|
||||
hint: `keep the alias on the most-canonical type; remove from the others`,
|
||||
});
|
||||
}
|
||||
}
|
||||
return issues;
|
||||
};
|
||||
|
||||
export const aliasReferencesUndeclaredType: LintRule = (manifest) => {
|
||||
// codex C14 — alias should be a known type OR a known alias of another
|
||||
// type. For v0.40.6.0 we lint the simpler case: alias must match a
|
||||
// declared page_type name. Closure validation is a v0.41+ extension.
|
||||
const issues: LintIssue[] = [];
|
||||
const typeNames = new Set(manifest.page_types.map((t) => t.name));
|
||||
for (const t of manifest.page_types) {
|
||||
for (const a of t.aliases) {
|
||||
if (!typeNames.has(a)) {
|
||||
issues.push({
|
||||
rule: 'alias_references_undeclared_type',
|
||||
severity: 'warning',
|
||||
message: `type '${t.name}' aliases '${a}' which is not a declared page_type in this pack`,
|
||||
pack: manifest.name,
|
||||
type: t.name,
|
||||
hint: `add a page_type for '${a}' OR remove the alias`,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
return issues;
|
||||
};
|
||||
|
||||
export const enrichableTypesUndeclared: LintRule = (manifest) => {
|
||||
const issues: LintIssue[] = [];
|
||||
const typeNames = new Set(manifest.page_types.map((t) => t.name));
|
||||
for (const e of manifest.enrichable_types) {
|
||||
if (!typeNames.has(e.type)) {
|
||||
issues.push({
|
||||
rule: 'enrichable_types_undeclared',
|
||||
severity: 'error',
|
||||
message: `enrichable_types references '${e.type}' which is not a declared page_type`,
|
||||
pack: manifest.name,
|
||||
type: e.type,
|
||||
hint: `add a page_type for '${e.type}' OR remove the enrichable_types entry`,
|
||||
});
|
||||
}
|
||||
}
|
||||
return issues;
|
||||
};
|
||||
|
||||
export const linkTypesUndeclared: LintRule = (manifest) => {
|
||||
const issues: LintIssue[] = [];
|
||||
const typeNames = new Set(manifest.page_types.map((t) => t.name));
|
||||
for (const lt of manifest.link_types) {
|
||||
if (lt.inference?.page_type && !typeNames.has(lt.inference.page_type)) {
|
||||
issues.push({
|
||||
rule: 'link_types_undeclared_page_type',
|
||||
severity: 'error',
|
||||
message: `link_type '${lt.name}' inference.page_type='${lt.inference.page_type}' is not a declared page_type`,
|
||||
pack: manifest.name,
|
||||
link: lt.name,
|
||||
hint: `add a page_type for '${lt.inference.page_type}' OR remove the inference rule`,
|
||||
});
|
||||
}
|
||||
if (lt.inference?.target_type && !typeNames.has(lt.inference.target_type)) {
|
||||
issues.push({
|
||||
rule: 'link_types_undeclared_target_type',
|
||||
severity: 'error',
|
||||
message: `link_type '${lt.name}' inference.target_type='${lt.inference.target_type}' is not a declared page_type`,
|
||||
pack: manifest.name,
|
||||
link: lt.name,
|
||||
hint: `add a page_type for '${lt.inference.target_type}' OR remove inference.target_type`,
|
||||
});
|
||||
}
|
||||
}
|
||||
return issues;
|
||||
};
|
||||
|
||||
export const frontmatterLinksUndeclared: LintRule = (manifest) => {
|
||||
const issues: LintIssue[] = [];
|
||||
const typeNames = new Set(manifest.page_types.map((t) => t.name));
|
||||
const linkNames = new Set(manifest.link_types.map((l) => l.name));
|
||||
for (const fl of manifest.frontmatter_links) {
|
||||
if (!typeNames.has(fl.page_type)) {
|
||||
issues.push({
|
||||
rule: 'frontmatter_links_undeclared_page_type',
|
||||
severity: 'error',
|
||||
message: `frontmatter_links.page_type='${fl.page_type}' is not a declared page_type`,
|
||||
pack: manifest.name,
|
||||
type: fl.page_type,
|
||||
hint: `add a page_type for '${fl.page_type}' OR remove the frontmatter_links rule`,
|
||||
});
|
||||
}
|
||||
if (!linkNames.has(fl.link_type)) {
|
||||
issues.push({
|
||||
rule: 'frontmatter_links_undeclared_link_type',
|
||||
severity: 'error',
|
||||
message: `frontmatter_links.link_type='${fl.link_type}' is not a declared link_type`,
|
||||
pack: manifest.name,
|
||||
link: fl.link_type,
|
||||
hint: `add a link_type for '${fl.link_type}' OR remove the frontmatter_links rule`,
|
||||
});
|
||||
}
|
||||
}
|
||||
return issues;
|
||||
};
|
||||
|
||||
export const expertRoutingWithoutPrefix: LintRule = (manifest) => {
|
||||
const issues: LintIssue[] = [];
|
||||
for (const t of manifest.page_types) {
|
||||
if (t.expert_routing && t.path_prefixes.length === 0) {
|
||||
issues.push({
|
||||
rule: 'expert_routing_without_prefix',
|
||||
severity: 'warning',
|
||||
message: `type '${t.name}' is expert_routing:true but has no path_prefixes; expert routing will silently miss content`,
|
||||
pack: manifest.name,
|
||||
type: t.name,
|
||||
hint: `add a path_prefix so put_page can infer this type from disk paths`,
|
||||
});
|
||||
}
|
||||
}
|
||||
return issues;
|
||||
};
|
||||
|
||||
export const prefixCollision: LintRule = (manifest) => {
|
||||
const issues: LintIssue[] = [];
|
||||
const prefixToTypes = new Map<string, string[]>();
|
||||
for (const t of manifest.page_types) {
|
||||
for (const p of t.path_prefixes) {
|
||||
const list = prefixToTypes.get(p) ?? [];
|
||||
list.push(t.name);
|
||||
prefixToTypes.set(p, list);
|
||||
}
|
||||
}
|
||||
for (const [prefix, owners] of prefixToTypes) {
|
||||
if (owners.length > 1) {
|
||||
issues.push({
|
||||
rule: 'prefix_collision',
|
||||
severity: 'error',
|
||||
message: `path_prefix '${prefix}' is declared by ${owners.length} types: ${owners.join(', ')}; type inference is undefined`,
|
||||
pack: manifest.name,
|
||||
hint: `keep the prefix on one canonical type; remove from the others OR use distinct prefixes`,
|
||||
});
|
||||
}
|
||||
}
|
||||
return issues;
|
||||
};
|
||||
|
||||
export const prefixStrictSubsetOverlap: LintRule = (manifest) => {
|
||||
const issues: LintIssue[] = [];
|
||||
// Build (prefix, owningType) pairs.
|
||||
const pairs: Array<{ prefix: string; type: string }> = [];
|
||||
for (const t of manifest.page_types) {
|
||||
for (const p of t.path_prefixes) pairs.push({ prefix: p, type: t.name });
|
||||
}
|
||||
for (let i = 0; i < pairs.length; i++) {
|
||||
for (let j = 0; j < pairs.length; j++) {
|
||||
if (i === j) continue;
|
||||
const a = pairs[i]!;
|
||||
const b = pairs[j]!;
|
||||
// a is strict subset of b: a starts with b AND a !== b.
|
||||
if (a.prefix !== b.prefix && a.prefix.startsWith(b.prefix) && a.type !== b.type) {
|
||||
issues.push({
|
||||
rule: 'prefix_strict_subset_overlap',
|
||||
severity: 'warning',
|
||||
message: `path_prefix '${a.prefix}' (type ${a.type}) is a strict subset of '${b.prefix}' (type ${b.type}); inference precedence is first-match-wins`,
|
||||
pack: manifest.name,
|
||||
hint: `ensure '${a.type}' is declared BEFORE '${b.type}' in page_types[] so the specific prefix wins`,
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
return issues;
|
||||
};
|
||||
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
// DB-aware rules (engine required; file-plane callers see empty arrays)
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
|
||||
export const extractableEmptyCorpus: LintRule = async (manifest, opts) => {
|
||||
if (!opts?.engine) return [];
|
||||
const issues: LintIssue[] = [];
|
||||
for (const t of manifest.page_types) {
|
||||
if (!t.extractable || t.path_prefixes.length === 0) continue;
|
||||
// Check each prefix; if all return zero, warn.
|
||||
let totalPages = 0;
|
||||
for (const p of t.path_prefixes) {
|
||||
try {
|
||||
const rows = await opts.engine.executeRaw(
|
||||
`SELECT COUNT(*)::text AS cnt FROM pages WHERE deleted_at IS NULL AND source_path LIKE $1`,
|
||||
[`${p}%`],
|
||||
) as Array<{ cnt?: string }>;
|
||||
const cnt = rows[0]?.cnt ? parseInt(rows[0].cnt, 10) : 0;
|
||||
totalPages += cnt;
|
||||
} catch {
|
||||
// Engine call failed (no `pages` table on a fresh PGLite install?); skip.
|
||||
return [];
|
||||
}
|
||||
}
|
||||
if (totalPages === 0) {
|
||||
issues.push({
|
||||
rule: 'extractable_empty_corpus',
|
||||
severity: 'warning',
|
||||
message: `type '${t.name}' is extractable:true but its path_prefixes match 0 pages in the DB`,
|
||||
pack: manifest.name,
|
||||
type: t.name,
|
||||
hint: `either remove extractable:true OR check that path_prefixes match actual import paths`,
|
||||
});
|
||||
}
|
||||
}
|
||||
return issues;
|
||||
};
|
||||
|
||||
export const mutationCountAnomaly: LintRule = (manifest, opts) => {
|
||||
const daysBack = opts?.daysBack ?? 7;
|
||||
const issues: LintIssue[] = [];
|
||||
try {
|
||||
const recs = readRecentMutations(daysBack);
|
||||
const forPack = recs.filter((r) => r.pack === manifest.name);
|
||||
if (forPack.length > 50) {
|
||||
issues.push({
|
||||
rule: 'mutation_count_anomaly',
|
||||
severity: 'warning',
|
||||
message: `pack '${manifest.name}' has ${forPack.length} mutations in the last ${daysBack} days; consider committing pack.json to source control`,
|
||||
pack: manifest.name,
|
||||
hint: `cd to your brain repo, git add the pack file, commit + push so the changes survive across machines`,
|
||||
});
|
||||
}
|
||||
} catch {
|
||||
// Audit dir unreadable; skip silently.
|
||||
}
|
||||
return issues;
|
||||
};
|
||||
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
// Aggregator
|
||||
// ────────────────────────────────────────────────────────────────────────
|
||||
|
||||
/** All rules. File-plane callers can compose a subset via FILE_PLANE_RULES. */
|
||||
export const ALL_LINT_RULES: ReadonlyArray<{ name: string; rule: LintRule; planeAware: boolean }> = [
|
||||
{ name: 'alias_shadows_type', rule: aliasShadowsType, planeAware: false },
|
||||
{ name: 'alias_declared_by_two_types', rule: aliasDeclaredByTwoTypes, planeAware: false },
|
||||
{ name: 'alias_references_undeclared_type', rule: aliasReferencesUndeclaredType, planeAware: false },
|
||||
{ name: 'enrichable_types_undeclared', rule: enrichableTypesUndeclared, planeAware: false },
|
||||
{ name: 'link_types_undeclared', rule: linkTypesUndeclared, planeAware: false },
|
||||
{ name: 'frontmatter_links_undeclared', rule: frontmatterLinksUndeclared, planeAware: false },
|
||||
{ name: 'expert_routing_without_prefix', rule: expertRoutingWithoutPrefix, planeAware: false },
|
||||
{ name: 'prefix_collision', rule: prefixCollision, planeAware: false },
|
||||
{ name: 'prefix_strict_subset_overlap', rule: prefixStrictSubsetOverlap, planeAware: false },
|
||||
{ name: 'extractable_empty_corpus', rule: extractableEmptyCorpus, planeAware: true },
|
||||
{ name: 'mutation_count_anomaly', rule: mutationCountAnomaly, planeAware: true },
|
||||
];
|
||||
|
||||
/** File-plane subset: rules safe to run inside `withMutation`'s pre-write gate. */
|
||||
export const FILE_PLANE_LINT_RULES = ALL_LINT_RULES.filter((r) => !r.planeAware);
|
||||
|
||||
export interface LintReport {
|
||||
ok: boolean;
|
||||
errors: LintIssue[];
|
||||
warnings: LintIssue[];
|
||||
}
|
||||
|
||||
function classify(issues: LintIssue[]): LintReport {
|
||||
const errors = issues.filter((i) => i.severity === 'error');
|
||||
const warnings = issues.filter((i) => i.severity === 'warning');
|
||||
return { ok: errors.length === 0, errors, warnings };
|
||||
}
|
||||
|
||||
/**
|
||||
* Run every rule (file-plane + DB-aware when engine is provided).
|
||||
* Returns a structured report ready to render via CLI or wire-encode
|
||||
* via MCP.
|
||||
*/
|
||||
export async function runAllLintRules(
|
||||
manifest: SchemaPackManifest,
|
||||
opts?: LintOpts,
|
||||
): Promise<LintReport> {
|
||||
const issues: LintIssue[] = [];
|
||||
for (const { rule, planeAware } of ALL_LINT_RULES) {
|
||||
if (planeAware && !opts?.engine) continue;
|
||||
const out = await rule(manifest, opts);
|
||||
issues.push(...out);
|
||||
}
|
||||
return classify(issues);
|
||||
}
|
||||
|
||||
/**
|
||||
* Run only file-plane rules. Used by `withMutation`'s pre-write
|
||||
* validation gate so a mutation that creates a dangling ref fails
|
||||
* BEFORE the atomic write happens.
|
||||
*
|
||||
* Returns the same shape as runAllLintRules but skips DB-aware checks.
|
||||
*/
|
||||
export async function runFilePlaneLintRules(
|
||||
manifest: SchemaPackManifest,
|
||||
): Promise<LintReport> {
|
||||
const issues: LintIssue[] = [];
|
||||
for (const { rule, planeAware } of ALL_LINT_RULES) {
|
||||
if (planeAware) continue;
|
||||
const out = await rule(manifest);
|
||||
issues.push(...out);
|
||||
}
|
||||
return classify(issues);
|
||||
}
|
||||
@@ -31,6 +31,7 @@ import { loadPackFromFile } from './loader.ts';
|
||||
import {
|
||||
resolveActivePackName,
|
||||
resolvePack,
|
||||
tryCachedPack,
|
||||
UnknownPackError,
|
||||
type ResolvedPack,
|
||||
type ResolutionInput,
|
||||
@@ -143,8 +144,17 @@ async function loadPackManifestByName(name: string): Promise<SchemaPackManifest>
|
||||
export async function loadActivePack(input: LoadActivePackInput): Promise<ResolvedPack> {
|
||||
const resolutionInput = buildResolutionInput(input);
|
||||
const resolution: ResolutionResult = resolveActivePackName(resolutionInput);
|
||||
// v0.40.6.0: TTL-gated cache fast path. Inside STAT_TTL_MS (default 1s)
|
||||
// returns immediately (~10ns). Outside the window: stats files in the
|
||||
// extends chain; cascade-invalidates and falls through on mtime change.
|
||||
const cached = tryCachedPack(resolution.pack_name);
|
||||
if (cached) return cached;
|
||||
const manifest = await loadPackManifestByName(resolution.pack_name);
|
||||
return await resolvePack(manifest, loadPackManifestByName);
|
||||
// Thread the locator so resolvePack can snapshot file paths + mtimes
|
||||
// for the stat-TTL gate on subsequent calls (codex C6 + D11 + D13).
|
||||
return await resolvePack(manifest, loadPackManifestByName, {
|
||||
loadByPath: (name) => _packLocator(name),
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
|
||||
@@ -0,0 +1,224 @@
|
||||
// v0.40.6.0 Schema Cathedral v3 — pack mutation audit JSONL.
|
||||
//
|
||||
// Every `withMutation` call (and every refused mutation) emits one line
|
||||
// to `~/.gbrain/audit/schema-mutations-YYYY-Www.jsonl`. The audit lets:
|
||||
// - `gbrain doctor` detect anomalous mutation patterns (Phase 9)
|
||||
// - `schema_pack_writability` doctor check surface failure events
|
||||
// - operators forensically trace who mutated what and when
|
||||
//
|
||||
// Privacy posture (D20 + codex C10):
|
||||
// Type names are SHA-8 redacted by default. Path prefixes are
|
||||
// truncated to the first path segment only. Matches
|
||||
// `candidate-audit.ts:7-22` privacy contract — both files write under
|
||||
// `~/.gbrain/audit/` and a single leaked screenshot of either could
|
||||
// otherwise reveal sensitive taxonomy (mental_health_diagnosis,
|
||||
// patients/oncology/, contracts/litigation/, etc.).
|
||||
//
|
||||
// Opt out of redaction with `GBRAIN_SCHEMA_AUDIT_VERBOSE=1` (same env
|
||||
// gate as candidate-audit so operators flip both surfaces together).
|
||||
//
|
||||
// Failure logging: we log both success and failure events so the Phase 9
|
||||
// `schema_pack_writability` check has signal to read (codex C11 — the
|
||||
// pre-fix plan only audited successes, leaving the doctor check unable
|
||||
// to surface PACK_READONLY failures).
|
||||
//
|
||||
// Best-effort writes: stderr warn on disk failure, never throws.
|
||||
|
||||
import { appendFileSync, existsSync, mkdirSync, readdirSync, readFileSync } from 'node:fs';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { isoWeekFilename, resolveAuditDir } from '../audit-week-file.ts';
|
||||
import { isAuditVerbose } from './candidate-audit.ts';
|
||||
|
||||
export type MutationOp =
|
||||
| 'add_type'
|
||||
| 'remove_type'
|
||||
| 'update_type'
|
||||
| 'add_alias'
|
||||
| 'remove_alias'
|
||||
| 'add_prefix'
|
||||
| 'remove_prefix'
|
||||
| 'add_link_type'
|
||||
| 'remove_link_type'
|
||||
| 'set_extractable'
|
||||
| 'set_expert_routing';
|
||||
|
||||
export type MutationActor = 'cli' | `mcp:${string}` | 'autopilot' | 'test';
|
||||
|
||||
export type MutationOutcome = 'success' | 'failure';
|
||||
|
||||
export interface MutationAuditRecord {
|
||||
/** ISO 8601 timestamp. */
|
||||
ts: string;
|
||||
/** Which primitive ran. */
|
||||
op: MutationOp;
|
||||
/** Pack name (NEVER redacted — pack names are user-chosen and non-PII). */
|
||||
pack: string;
|
||||
/** SHA-8 of the affected type name by default; raw when verbose env set. */
|
||||
type_or_hash: string | null;
|
||||
/** Whether `type_or_hash` is the raw type (verbose) or a sha8 hash. */
|
||||
type_redacted: boolean;
|
||||
/** First path segment only, e.g. 'people'. Null when op didn't touch a prefix. */
|
||||
prefix_first_seg: string | null;
|
||||
/** 'cli' | 'mcp:<clientId8>' | 'autopilot' | 'test'. */
|
||||
actor: MutationActor;
|
||||
/** Outcome of the mutation attempt. */
|
||||
outcome: MutationOutcome;
|
||||
/** When outcome=failure: short error code (e.g. 'PACK_READONLY'). */
|
||||
reason: string | null;
|
||||
/** Pack identity sha8 before mutation (null on failures that never read). */
|
||||
prev_sha8: string | null;
|
||||
/** Pack identity sha8 after mutation (null on failures). */
|
||||
new_sha8: string | null;
|
||||
/** Atomic batch identity — set when the mutation is part of a batched
|
||||
* `schema_apply_mutations` call so an auditor can reconstruct the
|
||||
* whole transaction. Null for single-mutation calls. */
|
||||
batch_id: string | null;
|
||||
}
|
||||
|
||||
export interface LogMutationOpts {
|
||||
op: MutationOp;
|
||||
pack: string;
|
||||
/** Affected type name, if any. */
|
||||
type?: string;
|
||||
/** Affected prefix (full path, will be redacted to first segment). */
|
||||
prefix?: string;
|
||||
actor: MutationActor;
|
||||
prev_sha8?: string;
|
||||
new_sha8?: string;
|
||||
batch_id?: string;
|
||||
}
|
||||
|
||||
export interface LogMutationFailureOpts extends LogMutationOpts {
|
||||
/** Short error code, e.g. 'PACK_READONLY' | 'LOCK_BUSY' | 'INVALID_RESULT'. */
|
||||
reason: string;
|
||||
}
|
||||
|
||||
/** sha-256 → 8 hex chars. Matches candidate-audit.ts redaction. */
|
||||
async function sha8(value: string): Promise<string> {
|
||||
const hashBuffer = await crypto.subtle.digest('SHA-256', new TextEncoder().encode(value));
|
||||
return Array.from(new Uint8Array(hashBuffer))
|
||||
.slice(0, 4)
|
||||
.map((b) => b.toString(16).padStart(2, '0'))
|
||||
.join('');
|
||||
}
|
||||
|
||||
export function computeMutateAuditPath(now: Date = new Date()): string {
|
||||
return join(resolveAuditDir(), isoWeekFilename('schema-mutations', now));
|
||||
}
|
||||
|
||||
async function buildRecord(
|
||||
opts: LogMutationOpts,
|
||||
outcome: MutationOutcome,
|
||||
reason: string | null,
|
||||
): Promise<MutationAuditRecord> {
|
||||
const verbose = isAuditVerbose();
|
||||
let typeField: string | null = null;
|
||||
if (opts.type !== undefined) {
|
||||
typeField = verbose ? opts.type : await sha8(opts.type);
|
||||
}
|
||||
const prefixField =
|
||||
opts.prefix !== undefined && opts.prefix.length > 0
|
||||
? (opts.prefix.split('/')[0] ?? '')
|
||||
: null;
|
||||
return {
|
||||
ts: new Date().toISOString(),
|
||||
op: opts.op,
|
||||
pack: opts.pack,
|
||||
type_or_hash: typeField,
|
||||
type_redacted: !verbose,
|
||||
prefix_first_seg: prefixField,
|
||||
actor: opts.actor,
|
||||
outcome,
|
||||
reason,
|
||||
prev_sha8: opts.prev_sha8 ?? null,
|
||||
new_sha8: opts.new_sha8 ?? null,
|
||||
batch_id: opts.batch_id ?? null,
|
||||
};
|
||||
}
|
||||
|
||||
function appendBestEffort(path: string, record: MutationAuditRecord): void {
|
||||
try {
|
||||
mkdirSync(dirname(path), { recursive: true });
|
||||
appendFileSync(path, JSON.stringify(record) + '\n', 'utf-8');
|
||||
} catch (e) {
|
||||
process.stderr.write(`[schema-mutations-audit] write failed: ${(e as Error).message}\n`);
|
||||
}
|
||||
}
|
||||
|
||||
/** Log a successful mutation. Best-effort; never throws. */
|
||||
export async function logMutationSuccess(opts: LogMutationOpts): Promise<void> {
|
||||
const record = await buildRecord(opts, 'success', null);
|
||||
appendBestEffort(computeMutateAuditPath(), record);
|
||||
}
|
||||
|
||||
/** Log a failed mutation. Best-effort; never throws. */
|
||||
export async function logMutationFailure(opts: LogMutationFailureOpts): Promise<void> {
|
||||
const record = await buildRecord(opts, 'failure', opts.reason);
|
||||
appendBestEffort(computeMutateAuditPath(), record);
|
||||
}
|
||||
|
||||
/**
|
||||
* Read mutation audit entries across the last N days. Skips malformed
|
||||
* lines silently (audit is best-effort).
|
||||
*/
|
||||
export function readRecentMutations(daysBack = 30): MutationAuditRecord[] {
|
||||
const auditDir = resolveAuditDir();
|
||||
if (!existsSync(auditDir)) return [];
|
||||
const cutoffIso = new Date(Date.now() - daysBack * 86400 * 1000).toISOString();
|
||||
const records: MutationAuditRecord[] = [];
|
||||
for (const name of readdirSync(auditDir)) {
|
||||
if (!name.startsWith('schema-mutations-') || !name.endsWith('.jsonl')) continue;
|
||||
let content: string;
|
||||
try {
|
||||
content = readFileSync(join(auditDir, name), 'utf-8');
|
||||
} catch {
|
||||
continue;
|
||||
}
|
||||
for (const line of content.split('\n')) {
|
||||
if (!line.trim()) continue;
|
||||
try {
|
||||
const r = JSON.parse(line) as MutationAuditRecord;
|
||||
if (r.ts >= cutoffIso) records.push(r);
|
||||
} catch {
|
||||
// Skip malformed lines.
|
||||
}
|
||||
}
|
||||
}
|
||||
return records;
|
||||
}
|
||||
|
||||
export interface MutationSummary {
|
||||
total: number;
|
||||
by_op: Partial<Record<MutationOp, number>>;
|
||||
by_outcome: { success: number; failure: number };
|
||||
by_pack: Record<string, number>;
|
||||
by_reason: Record<string, number>;
|
||||
by_actor: Record<string, number>;
|
||||
}
|
||||
|
||||
/**
|
||||
* Aggregate mutation records for cross-surface parity. Both `gbrain
|
||||
* doctor` (Phase 9) and `gbrain schema audit` (future surface) MUST
|
||||
* consume from this helper so the two display sites never drift.
|
||||
*/
|
||||
export function summarizeMutations(records: MutationAuditRecord[]): MutationSummary {
|
||||
const summary: MutationSummary = {
|
||||
total: records.length,
|
||||
by_op: {},
|
||||
by_outcome: { success: 0, failure: 0 },
|
||||
by_pack: {},
|
||||
by_reason: {},
|
||||
by_actor: {},
|
||||
};
|
||||
for (const r of records) {
|
||||
summary.by_op[r.op] = (summary.by_op[r.op] ?? 0) + 1;
|
||||
if (r.outcome === 'success') summary.by_outcome.success++;
|
||||
else summary.by_outcome.failure++;
|
||||
summary.by_pack[r.pack] = (summary.by_pack[r.pack] ?? 0) + 1;
|
||||
if (r.reason) summary.by_reason[r.reason] = (summary.by_reason[r.reason] ?? 0) + 1;
|
||||
// Bucket actor classes: cli | mcp | autopilot | test
|
||||
const actorBucket = r.actor.startsWith('mcp:') ? 'mcp' : r.actor;
|
||||
summary.by_actor[actorBucket] = (summary.by_actor[actorBucket] ?? 0) + 1;
|
||||
}
|
||||
return summary;
|
||||
}
|
||||
@@ -0,0 +1,298 @@
|
||||
// v0.40.6.0 Schema Cathedral v3 — per-pack file lock primitive.
|
||||
//
|
||||
// `withPackLock(packName, opts, fn)` serializes concurrent mutations of
|
||||
// the same pack file across processes. Two `gbrain schema add-type foo`
|
||||
// invocations on the same pack are made safe: the second blocks until the
|
||||
// first releases (or refuses with `LOCK_BUSY` if a timeout is set).
|
||||
//
|
||||
// Design (codex C8 from /plan-eng-review):
|
||||
// - Atomic acquire via `openSync(lockPath, 'wx')` — the 'wx' flag is
|
||||
// POSIX `O_CREAT | O_EXCL`, kernel-level atomic. The acquire either
|
||||
// creates the file or throws EEXIST. There is NO check-then-write
|
||||
// window. Do NOT copy `src/core/page-lock.ts:79+96`'s
|
||||
// `existsSync` + `writeFileSync` shape — that's TOCTOU.
|
||||
// - Stale-lock detection: a holder process may crash without releasing.
|
||||
// On EEXIST, read the lockfile, check `kill(pid, 0)` for liveness,
|
||||
// and check `Date.now() - ts > ttlMs`. If either is true, steal.
|
||||
// - TTL refresh: long-running DB-aware lint/stats can outlive the
|
||||
// default 60s ttl. While `fn()` runs, a background `setInterval`
|
||||
// rewrites `ts` every 10s so the lock stays fresh.
|
||||
// - `--force` semantics: "steal stale lock", NOT "skip locking". Even
|
||||
// forced acquires go through the same atomic open path — the only
|
||||
// difference is that on EEXIST + non-stale, force succeeds by
|
||||
// stealing instead of throwing.
|
||||
// - Cleanup: `try/finally` unconditionally releases. Refresh interval
|
||||
// is cleared. Lockfile is unlinked.
|
||||
//
|
||||
// Lock path: `~/.gbrain/schema-packs/.locks/<packName>.lock`. Per-pack
|
||||
// so two different packs never block each other. Honors `GBRAIN_HOME`
|
||||
// via the shared `gbrainPath()` helper.
|
||||
|
||||
import { closeSync, mkdirSync, openSync, readFileSync, unlinkSync, writeFileSync } from 'node:fs';
|
||||
import { dirname, join } from 'node:path';
|
||||
import { gbrainPath } from '../config.ts';
|
||||
|
||||
export const DEFAULT_LOCK_TTL_MS = 60_000;
|
||||
export const REFRESH_INTERVAL_MS = 10_000;
|
||||
export const DEFAULT_WAIT_TIMEOUT_MS = 0; // 0 = fail-immediately on EEXIST
|
||||
|
||||
export type LockOutcome = 'acquired' | 'stolen_stale' | 'forced';
|
||||
|
||||
export interface PackLockOpts {
|
||||
/** TTL in ms before another acquirer considers the lock stale. Default 60s. */
|
||||
ttlMs?: number;
|
||||
/** Steal the lock even if non-stale + live PID. Default false. */
|
||||
force?: boolean;
|
||||
/** Override the lock directory for tests. */
|
||||
lockDir?: string;
|
||||
/**
|
||||
* Inject a clock for tests (returns ms since epoch).
|
||||
* Production callers leave undefined.
|
||||
*/
|
||||
now?: () => number;
|
||||
/**
|
||||
* Inject a PID-liveness probe for tests. Returns true if the PID is alive.
|
||||
* Production callers leave undefined (defaults to `kill(pid, 0)`).
|
||||
*/
|
||||
isPidAlive?: (pid: number) => boolean;
|
||||
}
|
||||
|
||||
export interface LockFileRecord {
|
||||
/** Holder process PID. */
|
||||
pid: number;
|
||||
/** Holder hostname (informational only). */
|
||||
hostname: string;
|
||||
/** Last refresh timestamp (ms since epoch). */
|
||||
ts: number;
|
||||
/** Holder's declared TTL in ms. */
|
||||
ttlMs: number;
|
||||
}
|
||||
|
||||
export class PackLockBusyError extends Error {
|
||||
readonly code = 'LOCK_BUSY' as const;
|
||||
readonly heldBy: number;
|
||||
readonly ageMs: number;
|
||||
readonly ttlMs: number;
|
||||
constructor(message: string, opts: { heldBy: number; ageMs: number; ttlMs: number }) {
|
||||
super(message);
|
||||
this.name = 'PackLockBusyError';
|
||||
this.heldBy = opts.heldBy;
|
||||
this.ageMs = opts.ageMs;
|
||||
this.ttlMs = opts.ttlMs;
|
||||
}
|
||||
}
|
||||
|
||||
function defaultIsPidAlive(pid: number): boolean {
|
||||
try {
|
||||
// Signal 0 doesn't deliver, but throws ESRCH if the process is dead.
|
||||
process.kill(pid, 0);
|
||||
return true;
|
||||
} catch (err) {
|
||||
const code = (err as NodeJS.ErrnoException).code;
|
||||
// EPERM means the process exists but we lack permission to signal.
|
||||
return code === 'EPERM';
|
||||
}
|
||||
}
|
||||
|
||||
function resolveLockPath(packName: string, lockDir?: string): string {
|
||||
const dir = lockDir ?? gbrainPath('schema-packs', '.locks');
|
||||
return join(dir, `${packName}.lock`);
|
||||
}
|
||||
|
||||
function readLockFile(path: string): LockFileRecord | null {
|
||||
try {
|
||||
const raw = readFileSync(path, 'utf-8');
|
||||
const parsed = JSON.parse(raw) as Partial<LockFileRecord>;
|
||||
if (
|
||||
typeof parsed.pid === 'number' &&
|
||||
typeof parsed.ts === 'number' &&
|
||||
typeof parsed.ttlMs === 'number' &&
|
||||
typeof parsed.hostname === 'string'
|
||||
) {
|
||||
return parsed as LockFileRecord;
|
||||
}
|
||||
return null;
|
||||
} catch {
|
||||
return null;
|
||||
}
|
||||
}
|
||||
|
||||
function writeLockRecord(path: string, fd: number, record: LockFileRecord): void {
|
||||
// We hold an open fd from the atomic create; truncate-and-write via fd.
|
||||
// Bun's fs.writeFileSync(path, ...) when the file exists is fine here
|
||||
// because we already own the lock (exclusive create succeeded).
|
||||
writeFileSync(path, JSON.stringify(record), 'utf-8');
|
||||
// Keep the fd open until release so file is held by this process; some
|
||||
// FS layers prefer this for crash detection. (Functionally a no-op on
|
||||
// POSIX where unlink() works regardless.)
|
||||
closeSync(fd);
|
||||
}
|
||||
|
||||
/**
|
||||
* Try to atomically acquire the lock. Returns the descriptor on success;
|
||||
* throws on EEXIST (caller decides whether to steal). Handles ENOENT on
|
||||
* the parent dir by creating it once.
|
||||
*/
|
||||
function atomicAcquire(lockPath: string): number {
|
||||
try {
|
||||
return openSync(lockPath, 'wx');
|
||||
} catch (err) {
|
||||
const code = (err as NodeJS.ErrnoException).code;
|
||||
if (code === 'ENOENT') {
|
||||
mkdirSync(dirname(lockPath), { recursive: true });
|
||||
return openSync(lockPath, 'wx');
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Decide whether a held lock is stale based on TTL + PID liveness.
|
||||
* Exported for unit testing the policy in isolation.
|
||||
*/
|
||||
export function isLockStale(
|
||||
record: LockFileRecord,
|
||||
now: number,
|
||||
isPidAlive: (pid: number) => boolean,
|
||||
): { stale: boolean; reason: 'ttl_expired' | 'pid_dead' | 'live' } {
|
||||
const ageMs = now - record.ts;
|
||||
if (ageMs > record.ttlMs) return { stale: true, reason: 'ttl_expired' };
|
||||
if (!isPidAlive(record.pid)) return { stale: true, reason: 'pid_dead' };
|
||||
return { stale: false, reason: 'live' };
|
||||
}
|
||||
|
||||
/**
|
||||
* Acquire the lock OR throw `PackLockBusyError`. Steals stale locks
|
||||
* (per TTL + PID liveness) or when `opts.force` is set. Returns the
|
||||
* outcome so callers can audit how the acquire resolved.
|
||||
*/
|
||||
export function acquirePackLock(
|
||||
packName: string,
|
||||
opts: PackLockOpts = {},
|
||||
): { lockPath: string; outcome: LockOutcome; record: LockFileRecord } {
|
||||
const ttlMs = opts.ttlMs ?? DEFAULT_LOCK_TTL_MS;
|
||||
const now = opts.now ?? Date.now;
|
||||
const isPidAlive = opts.isPidAlive ?? defaultIsPidAlive;
|
||||
const lockPath = resolveLockPath(packName, opts.lockDir);
|
||||
|
||||
const tryOnce = (): number | 'EEXIST' => {
|
||||
try {
|
||||
return atomicAcquire(lockPath);
|
||||
} catch (err) {
|
||||
if ((err as NodeJS.ErrnoException).code === 'EEXIST') return 'EEXIST';
|
||||
throw err;
|
||||
}
|
||||
};
|
||||
|
||||
const writeNew = (fd: number, outcome: LockOutcome): { lockPath: string; outcome: LockOutcome; record: LockFileRecord } => {
|
||||
const record: LockFileRecord = {
|
||||
pid: process.pid,
|
||||
hostname: process.env.HOSTNAME ?? 'unknown',
|
||||
ts: now(),
|
||||
ttlMs,
|
||||
};
|
||||
writeLockRecord(lockPath, fd, record);
|
||||
return { lockPath, outcome, record };
|
||||
};
|
||||
|
||||
// First attempt: clean acquire.
|
||||
const first = tryOnce();
|
||||
if (first !== 'EEXIST') return writeNew(first, 'acquired');
|
||||
|
||||
// EEXIST: inspect.
|
||||
const existing = readLockFile(lockPath);
|
||||
if (existing === null) {
|
||||
// Lockfile is corrupt — treat as stale and steal.
|
||||
try { unlinkSync(lockPath); } catch { /* race with another stealer; retry below */ }
|
||||
const retry = tryOnce();
|
||||
if (retry === 'EEXIST') {
|
||||
throw new PackLockBusyError(
|
||||
`pack ${packName} lock is corrupt and another process re-acquired it during recovery`,
|
||||
{ heldBy: -1, ageMs: 0, ttlMs },
|
||||
);
|
||||
}
|
||||
return writeNew(retry, 'stolen_stale');
|
||||
}
|
||||
|
||||
const staleness = isLockStale(existing, now(), isPidAlive);
|
||||
if (staleness.stale || opts.force) {
|
||||
try { unlinkSync(lockPath); } catch { /* race with another stealer; retry below */ }
|
||||
const retry = tryOnce();
|
||||
if (retry === 'EEXIST') {
|
||||
// Another stealer won. Surface as busy with the current holder.
|
||||
const current = readLockFile(lockPath);
|
||||
const ageMs = current ? now() - current.ts : 0;
|
||||
throw new PackLockBusyError(
|
||||
`pack ${packName} lock was stolen by another process during our recovery (pid=${current?.pid ?? '?'})`,
|
||||
{ heldBy: current?.pid ?? -1, ageMs, ttlMs: current?.ttlMs ?? ttlMs },
|
||||
);
|
||||
}
|
||||
return writeNew(retry, opts.force ? 'forced' : 'stolen_stale');
|
||||
}
|
||||
|
||||
// Live and non-stale: refuse.
|
||||
throw new PackLockBusyError(
|
||||
`pack ${packName} is locked by pid=${existing.pid} (held ${Math.round((now() - existing.ts) / 1000)}s, ttl=${Math.round(existing.ttlMs / 1000)}s; --force to steal)`,
|
||||
{ heldBy: existing.pid, ageMs: now() - existing.ts, ttlMs: existing.ttlMs },
|
||||
);
|
||||
}
|
||||
|
||||
/**
|
||||
* Refresh the lock's `ts` field so long-running operations don't appear
|
||||
* stale to a concurrent acquirer. Best-effort: write failures are
|
||||
* swallowed silently. Returns true on success, false if the lockfile is
|
||||
* gone (we lost the lock).
|
||||
*/
|
||||
function refreshLock(lockPath: string, record: LockFileRecord, now: number): boolean {
|
||||
try {
|
||||
const next: LockFileRecord = { ...record, ts: now };
|
||||
writeFileSync(lockPath, JSON.stringify(next), 'utf-8');
|
||||
return true;
|
||||
} catch {
|
||||
return false;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Release the lock by unlinking the file. Idempotent — missing file is
|
||||
* not an error (the holder may have crashed and another process stole).
|
||||
*/
|
||||
function releasePackLock(lockPath: string): void {
|
||||
try {
|
||||
unlinkSync(lockPath);
|
||||
} catch (err) {
|
||||
if ((err as NodeJS.ErrnoException).code !== 'ENOENT') {
|
||||
// Don't throw from a finally path — log to stderr and move on.
|
||||
process.stderr.write(`[pack-lock] release failed for ${lockPath}: ${(err as Error).message}\n`);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Run `fn()` with exclusive access to `packName`. Acquires the lock,
|
||||
* starts a TTL refresh timer, runs `fn`, then unconditionally releases.
|
||||
*
|
||||
* Throws `PackLockBusyError` if the lock is held by a live process and
|
||||
* neither stale nor forced.
|
||||
*/
|
||||
export async function withPackLock<T>(
|
||||
packName: string,
|
||||
opts: PackLockOpts,
|
||||
fn: () => Promise<T> | T,
|
||||
): Promise<T> {
|
||||
const acquired = acquirePackLock(packName, opts);
|
||||
const now = opts.now ?? Date.now;
|
||||
let currentRecord = acquired.record;
|
||||
const refresh = setInterval(() => {
|
||||
currentRecord = { ...currentRecord, ts: now() };
|
||||
refreshLock(acquired.lockPath, currentRecord, now());
|
||||
}, REFRESH_INTERVAL_MS);
|
||||
// Don't keep the process alive just for the refresh timer.
|
||||
if (typeof (refresh as NodeJS.Timer).unref === 'function') (refresh as NodeJS.Timer).unref();
|
||||
try {
|
||||
return await fn();
|
||||
} finally {
|
||||
clearInterval(refresh);
|
||||
releasePackLock(acquired.lockPath);
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
// v0.40.6.0 Schema Cathedral v3 — query-cache invalidation hook.
|
||||
//
|
||||
// Codex C9: `schema sync --apply` and `schema add-type` change page
|
||||
// types under cached search rows that were keyed by the OLD knobs_hash
|
||||
// (which doesn't include schema-pack identity yet — that's a v0.41+
|
||||
// design choice). Without invalidation, an agent who mutates the pack
|
||||
// AND immediately re-queries sees stale results from the pre-mutation
|
||||
// cache.
|
||||
//
|
||||
// The fix: after every successful withMutation, call
|
||||
// `invalidateQueryCache(engine, sourceId)` which DELETEs all rows for
|
||||
// the source. Cache rebuilds organically on next search — the only cost
|
||||
// is one extra LLM expansion / vector call per query for the first few
|
||||
// requests after a mutation. That's the right trade vs serving stale
|
||||
// page types.
|
||||
//
|
||||
// Reuses the existing SemanticQueryCache.clear() method (already
|
||||
// PGLite + Postgres parity-safe) rather than reinventing the SQL.
|
||||
|
||||
import type { BrainEngine } from '../engine.ts';
|
||||
import { SemanticQueryCache } from '../search/query-cache.ts';
|
||||
|
||||
export interface InvalidateQueryCacheResult {
|
||||
rows_invalidated: number;
|
||||
}
|
||||
|
||||
/**
|
||||
* Invalidate query_cache rows scoped to a source so search results
|
||||
* bound to the old knobs_hash don't serve stale page types after
|
||||
* schema mutations.
|
||||
*
|
||||
* Best-effort: failures (e.g. pre-v51 brain without the table) return
|
||||
* {rows_invalidated: 0} silently. Mutation hot-path must never break
|
||||
* because the cache invalidator fell over.
|
||||
*
|
||||
* `sourceId` omitted clears the whole table. Used by Phase 4 reload and
|
||||
* any cross-source mutation.
|
||||
*/
|
||||
export async function invalidateQueryCache(
|
||||
engine: BrainEngine,
|
||||
sourceId?: string,
|
||||
): Promise<InvalidateQueryCacheResult> {
|
||||
try {
|
||||
const cache = new SemanticQueryCache(engine);
|
||||
const rows_invalidated = await cache.clear(sourceId !== undefined ? { sourceId } : {});
|
||||
return { rows_invalidated };
|
||||
} catch {
|
||||
return { rows_invalidated: 0 };
|
||||
}
|
||||
}
|
||||
@@ -1,11 +1,31 @@
|
||||
// v0.38 schema pack registry — load, cache, resolve active pack.
|
||||
//
|
||||
// ┌──────────────────────────────────────────────────────┐
|
||||
// │ loadActivePack lifecycle (per process, v0.40.6.0) │
|
||||
// └──────────────────────────────────────────────────────┘
|
||||
// │
|
||||
// ┌────────────────────────┼────────────────────────┐
|
||||
// ▼ ▼ ▼
|
||||
// cache miss cache hit cache hit + TTL expired
|
||||
// │ │ │
|
||||
// fresh load STAT_TTL_MS gate statSync compare every file
|
||||
// (resolvePack) (~10ns fast return) in the extends chain
|
||||
// │ │ │
|
||||
// │ │ ┌──────────┴──────────┐
|
||||
// │ │ ▼ ▼
|
||||
// │ │ every mtime unchanged any mtime changed
|
||||
// │ │ │ │
|
||||
// │ │ refresh lastStatMs invalidate(name) +
|
||||
// │ │ return cached extends-chain cascade
|
||||
// │ │ (codex C6)
|
||||
// ▼ ▼ │
|
||||
// byName.set(name, entry) return cached fresh load
|
||||
//
|
||||
// Pack resolution chain (7 tiers per D13, tier-1 trust-gated):
|
||||
// 1. Per-call `schema_pack` opt — CLI only (`ctx.remote === false`).
|
||||
// Rejected for `ctx.remote === true` (D13 trust boundary).
|
||||
// 2. `GBRAIN_SCHEMA_PACK` env var
|
||||
// 3. Per-source DB config key `schema_pack.source.<id>` (dots, not
|
||||
// colons — codex F16, matches `models.tier.*` convention)
|
||||
// 3. Per-source DB config key `schema_pack.source.<id>`
|
||||
// 4. Brain-wide DB config key `schema_pack`
|
||||
// 5. `gbrain.yml schema:` section
|
||||
// 6. `~/.gbrain/config.json schema_pack` field
|
||||
@@ -14,20 +34,31 @@
|
||||
// Extends chain semantics (E4):
|
||||
// - Depth tracked via BFS during resolve.
|
||||
// - Soft warn to stderr at depth > 4.
|
||||
// - Hard reject at depth > 8 with paste-ready "shorten your extends
|
||||
// chain" hint.
|
||||
// - Hard reject at depth > 8.
|
||||
//
|
||||
// Cache: resolved packs are cached in-memory by pack-identity
|
||||
// (`<name>@<version>+<sha8>`). Mtime check via `loadPackFromFile` on
|
||||
// disk-backed packs invalidates the cache when a user edits the
|
||||
// manifest.
|
||||
// v0.40.6.0 cache invariants (codex C6 + D11 + D13):
|
||||
// - Cache key is the pack NAME (not identity sha8). Per-name cache entry
|
||||
// records the resolved pack PLUS every file path that fed it AND the
|
||||
// identities of every parent in the extends chain.
|
||||
// - Cache hits go through a stat-TTL gate (default 1000ms via
|
||||
// STAT_TTL_MS, env override GBRAIN_PACK_STAT_TTL_MS). Inside the
|
||||
// window: hot-path return (~10ns). Outside: statSync each file; if
|
||||
// any mtime changed, invalidate by name + cascade to every dependent.
|
||||
// - invalidatePackCache(name) walks the reverse extends-graph and
|
||||
// evicts every pack that has `name` in its chain. Without the cascade,
|
||||
// editing a parent silently leaves children stale (the codex C6 bug).
|
||||
// - The PUBLIC `ResolvedPack.identity` field is unchanged
|
||||
// (`<name>@<version>+<sha8>`); the composite cache key lives only
|
||||
// inside the registry.
|
||||
|
||||
import { statSync } from 'node:fs';
|
||||
import type { SchemaPackManifest } from './manifest-v1.ts';
|
||||
import { computeManifestSha8, packIdentity } from './manifest-v1.ts';
|
||||
import { computeAliasClosureHash, buildAliasGraph, type AliasGraph } from './closure.ts';
|
||||
|
||||
export const EXTENDS_DEPTH_WARN = 4 as const;
|
||||
export const EXTENDS_DEPTH_HARD_CAP = 8 as const;
|
||||
export const STAT_TTL_MS_DEFAULT = 1000 as const;
|
||||
|
||||
export class ExtendsChainTooDeepError extends Error {
|
||||
readonly depth: number;
|
||||
@@ -49,109 +80,188 @@ export class UnknownPackError extends Error {
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* The fully resolved pack — manifest + computed graph + identity hash.
|
||||
* Returned by `loadActivePack`; cached by registry until the source
|
||||
* manifest mtime changes.
|
||||
*/
|
||||
export interface ResolvedPack {
|
||||
manifest: SchemaPackManifest;
|
||||
identity: string; // `<name>@<version>+<sha8>`
|
||||
identity: string; // `<name>@<version>+<sha8>` (child only — wire-stable)
|
||||
manifest_sha8: string;
|
||||
alias_closure_hash: string;
|
||||
alias_graph: AliasGraph;
|
||||
}
|
||||
|
||||
/**
|
||||
* 7-tier resolution chain. Returns the pack NAME to load (resolved
|
||||
* packs are fetched separately via `loadPackByName`). Tier 1 (per-call)
|
||||
* is gated on `remote === false`; remote callers passing the opt get
|
||||
* `permission_denied` from operations.ts before reaching here.
|
||||
*/
|
||||
export interface ResolutionInput {
|
||||
/** Tier 1: per-call opt. ONLY honored when `remote === false`. */
|
||||
perCall?: string;
|
||||
/** Tier 1 trust gate. `true` = MCP/OAuth caller; rejects per-call opt. */
|
||||
remote: boolean;
|
||||
/** Tier 3: per-source DB config map (source_id → pack name). */
|
||||
perSourceDb?: ReadonlyMap<string, string>;
|
||||
/** Source ID the query targets (tier 3 lookup). */
|
||||
sourceId?: string;
|
||||
/** Tier 2: env var (`GBRAIN_SCHEMA_PACK`). */
|
||||
envVar?: string;
|
||||
/** Tier 4: brain-wide DB config. */
|
||||
dbConfig?: string;
|
||||
/** Tier 5: gbrain.yml schema.pack field. */
|
||||
gbrainYml?: string;
|
||||
/** Tier 6: ~/.gbrain/config.json schema_pack field. */
|
||||
homeConfig?: string;
|
||||
}
|
||||
|
||||
/** Resolved tier + pack name. `source` documents which tier won. */
|
||||
export interface ResolutionResult {
|
||||
pack_name: string;
|
||||
source: 'per-call' | 'env' | 'per-source-db' | 'db-config' | 'gbrain-yml' | 'home-config' | 'default';
|
||||
}
|
||||
|
||||
export function resolveActivePackName(input: ResolutionInput): ResolutionResult {
|
||||
// Tier 1: per-call opt (CLI only).
|
||||
if (input.perCall && input.remote === false) {
|
||||
return { pack_name: input.perCall, source: 'per-call' };
|
||||
}
|
||||
// Tier 2: env var.
|
||||
if (input.envVar) return { pack_name: input.envVar, source: 'env' };
|
||||
// Tier 3: per-source DB config.
|
||||
if (input.sourceId && input.perSourceDb?.has(input.sourceId)) {
|
||||
return { pack_name: input.perSourceDb.get(input.sourceId)!, source: 'per-source-db' };
|
||||
}
|
||||
// Tier 4: brain-wide DB.
|
||||
if (input.dbConfig) return { pack_name: input.dbConfig, source: 'db-config' };
|
||||
// Tier 5: gbrain.yml schema:
|
||||
if (input.gbrainYml) return { pack_name: input.gbrainYml, source: 'gbrain-yml' };
|
||||
// Tier 6: ~/.gbrain/config.json
|
||||
if (input.homeConfig) return { pack_name: input.homeConfig, source: 'home-config' };
|
||||
// Tier 7: default
|
||||
return { pack_name: 'gbrain-base', source: 'default' };
|
||||
}
|
||||
|
||||
/**
|
||||
* In-memory cache keyed by pack-identity. Resolved packs are immutable
|
||||
* once loaded; the cache is invalidated by mtime check in the
|
||||
* disk-loader callers.
|
||||
* Per-name cache entry. Tracks the resolved pack PLUS the file-stat
|
||||
* snapshot every file in the extends chain fed at resolve time. The
|
||||
* stat snapshot is what the cross-process stat-TTL gate compares
|
||||
* against on each loadActivePack call.
|
||||
*/
|
||||
const _packCache = new Map<string, ResolvedPack>();
|
||||
interface CacheEntry {
|
||||
resolved: ResolvedPack;
|
||||
/** Names that fed this entry (this pack + every parent transitively). */
|
||||
chain: ReadonlyArray<string>;
|
||||
/** Stat snapshot per file at resolve time. */
|
||||
files: ReadonlyArray<{ name: string; path: string; mtimeMs: number }>;
|
||||
/** Last time we stat()'d the files. Date.now() ms. */
|
||||
lastStatMs: number;
|
||||
}
|
||||
|
||||
const _byName = new Map<string, CacheEntry>();
|
||||
|
||||
/** Test seam — clears the in-process resolver cache. */
|
||||
export function _resetPackCacheForTests(): void {
|
||||
_packCache.clear();
|
||||
_byName.clear();
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve the effective STAT_TTL_MS, honoring the
|
||||
* `GBRAIN_PACK_STAT_TTL_MS` env override. Invalid values fall back to
|
||||
* the default with no warning (this is a power-user knob).
|
||||
*/
|
||||
function resolveStatTtlMs(): number {
|
||||
const raw = process.env.GBRAIN_PACK_STAT_TTL_MS;
|
||||
if (!raw) return STAT_TTL_MS_DEFAULT;
|
||||
const parsed = Number.parseInt(raw, 10);
|
||||
if (Number.isFinite(parsed) && parsed >= 0) return parsed;
|
||||
return STAT_TTL_MS_DEFAULT;
|
||||
}
|
||||
|
||||
/**
|
||||
* Cheap statSync that returns Infinity on error so callers treat
|
||||
* disappearing files as "changed" (forcing reload).
|
||||
*/
|
||||
function safeMtimeMs(path: string): number {
|
||||
try {
|
||||
return statSync(path).mtimeMs;
|
||||
} catch {
|
||||
return Number.POSITIVE_INFINITY;
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Check whether a cached entry's file snapshot is still fresh on disk.
|
||||
* Returns true when EVERY file's mtime matches the snapshot.
|
||||
*/
|
||||
function snapshotMatches(files: ReadonlyArray<{ path: string; mtimeMs: number }>): boolean {
|
||||
for (const f of files) {
|
||||
if (safeMtimeMs(f.path) !== f.mtimeMs) return false;
|
||||
}
|
||||
return true;
|
||||
}
|
||||
|
||||
/**
|
||||
* Walk the reverse extends-graph: every cached entry whose `chain`
|
||||
* contains `name`. The set is unbounded in principle but bounded in
|
||||
* practice by EXTENDS_DEPTH_HARD_CAP × installed packs (typically <50).
|
||||
*/
|
||||
function findDependents(name: string): string[] {
|
||||
const out: string[] = [];
|
||||
for (const [cachedName, entry] of _byName) {
|
||||
if (entry.chain.includes(name)) out.push(cachedName);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* Invalidate the cache for a pack name AND every pack that extends it
|
||||
* (transitive — the codex C6 fix). When called with no argument,
|
||||
* invalidates everything.
|
||||
*
|
||||
* Called automatically by `withMutation` (Phase 2) after every
|
||||
* successful pack mutation; also exposed via `gbrain schema reload`.
|
||||
*/
|
||||
export function invalidatePackCache(name?: string): { invalidated: string[] } {
|
||||
if (name === undefined) {
|
||||
const all = [..._byName.keys()];
|
||||
_byName.clear();
|
||||
return { invalidated: all };
|
||||
}
|
||||
const dependents = findDependents(name);
|
||||
// The pack itself + all dependents.
|
||||
const toEvict = Array.from(new Set([name, ...dependents]));
|
||||
for (const n of toEvict) _byName.delete(n);
|
||||
return { invalidated: toEvict };
|
||||
}
|
||||
|
||||
/** Test-only access for assertions on the cache shape. */
|
||||
export function _cacheSizeForTests(): number {
|
||||
return _byName.size;
|
||||
}
|
||||
|
||||
/** Test-only access for assertions on which names are cached. */
|
||||
export function _cacheNamesForTests(): string[] {
|
||||
return [..._byName.keys()];
|
||||
}
|
||||
|
||||
/**
|
||||
* Resolve + cache a manifest. Loads parent packs via the `loadByName`
|
||||
* dependency, tracks extends-chain depth, applies the E4 cap.
|
||||
*
|
||||
* Pass `loadByName` as a dependency so the registry doesn't have to
|
||||
* own filesystem layout (tests inject a mock; production wires the
|
||||
* disk loader).
|
||||
* v0.40.6.0: cache is name-keyed and tracks file-stat snapshots so the
|
||||
* stat-TTL gate (inside `loadActivePack`) can detect cross-process
|
||||
* mutations without re-reading the bytes.
|
||||
*
|
||||
* `loadByPath` is the disk path resolver for each name in the extends
|
||||
* chain (used for the file-stat snapshot). Optional — when omitted, the
|
||||
* snapshot is empty and stat-TTL becomes a no-op for this entry (used
|
||||
* by tests that drive synthetic manifests with no disk backing).
|
||||
*/
|
||||
export async function resolvePack(
|
||||
manifest: SchemaPackManifest,
|
||||
loadByName: (name: string) => Promise<SchemaPackManifest>,
|
||||
opts: { onDepthWarn?: (depth: number, chain: string[]) => void } = {},
|
||||
opts: {
|
||||
onDepthWarn?: (depth: number, chain: string[]) => void;
|
||||
loadByPath?: (name: string) => string | null;
|
||||
} = {},
|
||||
): Promise<ResolvedPack> {
|
||||
const sha8 = await computeManifestSha8(manifest);
|
||||
const id = packIdentity(manifest, sha8);
|
||||
const cached = _packCache.get(id);
|
||||
if (cached) return cached;
|
||||
|
||||
// Walk extends chain to enforce depth cap.
|
||||
// Reference-equality fast path: if a previous resolvePack(manifest, ...)
|
||||
// produced the SAME identity, return the cached resolved object. This
|
||||
// preserves the v0.38 contract that two calls with the same manifest
|
||||
// bytes return the same JS object reference.
|
||||
const existing = _byName.get(manifest.name);
|
||||
if (existing && existing.resolved.identity === id) {
|
||||
return existing.resolved;
|
||||
}
|
||||
|
||||
// Walk extends chain to enforce depth cap AND collect names for the
|
||||
// cache snapshot (codex C6 — child cache entry must remember every
|
||||
// parent so invalidatePackCache(parentName) can cascade).
|
||||
const chain: string[] = [manifest.name];
|
||||
let cursor: SchemaPackManifest | null = manifest;
|
||||
while (cursor?.extends) {
|
||||
const parentName = cursor.extends;
|
||||
if (chain.includes(parentName)) {
|
||||
// Cycle in extends graph — should be impossible for legit packs;
|
||||
// safety net.
|
||||
throw new ExtendsChainTooDeepError(chain.length, [...chain, parentName]);
|
||||
}
|
||||
chain.push(parentName);
|
||||
@@ -164,10 +274,8 @@ export async function resolvePack(
|
||||
cursor = await loadByName(parentName);
|
||||
}
|
||||
|
||||
// For v0.38 skeleton: the closure is computed on the manifest itself.
|
||||
// T7 Phase B will add full extends-merging (child-wins) here. For now,
|
||||
// we treat manifest.page_types as the effective set (gbrain-base
|
||||
// codegen places everything in the seed manifest directly).
|
||||
// For v0.38 skeleton: closure is computed on the manifest itself.
|
||||
// Full extends-merging (child-wins) is the v0.41+ T20 follow-up.
|
||||
const alias_graph = buildAliasGraph(manifest);
|
||||
const alias_closure_hash = await computeAliasClosureHash(manifest);
|
||||
|
||||
@@ -178,6 +286,51 @@ export async function resolvePack(
|
||||
alias_closure_hash,
|
||||
alias_graph,
|
||||
};
|
||||
_packCache.set(id, resolved);
|
||||
|
||||
// Capture file-stat snapshot for the stat-TTL gate. Skip names that
|
||||
// the locator can't resolve (synthetic manifests in tests).
|
||||
const files: Array<{ name: string; path: string; mtimeMs: number }> = [];
|
||||
if (opts.loadByPath) {
|
||||
for (const n of chain) {
|
||||
const path = opts.loadByPath(n);
|
||||
if (path === null) continue;
|
||||
files.push({ name: n, path, mtimeMs: safeMtimeMs(path) });
|
||||
}
|
||||
}
|
||||
|
||||
_byName.set(manifest.name, {
|
||||
resolved,
|
||||
chain: [...chain],
|
||||
files,
|
||||
lastStatMs: Date.now(),
|
||||
});
|
||||
return resolved;
|
||||
}
|
||||
|
||||
/**
|
||||
* Try to return a cached resolved pack for `name` without re-reading the
|
||||
* manifest from disk. Returns null on cache miss OR when the stat-TTL
|
||||
* gate detects a file change (which triggers eviction + cascade).
|
||||
*
|
||||
* The TTL gate keeps the hot path cheap: most calls inside the 1-second
|
||||
* window return immediately (~10ns) without statting. Outside the
|
||||
* window: one statSync per file in the extends chain (~50µs per file).
|
||||
* Worst-case latency for a daemon picking up an operator's mutation:
|
||||
* 1 second.
|
||||
*/
|
||||
export function tryCachedPack(name: string): ResolvedPack | null {
|
||||
const entry = _byName.get(name);
|
||||
if (!entry) return null;
|
||||
const ttl = resolveStatTtlMs();
|
||||
const ageMs = Date.now() - entry.lastStatMs;
|
||||
if (ageMs < ttl) return entry.resolved;
|
||||
// TTL expired: stat all files. If any changed, cascade-invalidate.
|
||||
if (!snapshotMatches(entry.files)) {
|
||||
invalidatePackCache(name);
|
||||
return null;
|
||||
}
|
||||
// Snapshot still fresh: refresh lastStatMs so the next hot-path return
|
||||
// is cheap again.
|
||||
_byName.set(name, { ...entry, lastStatMs: Date.now() });
|
||||
return entry.resolved;
|
||||
}
|
||||
|
||||
@@ -0,0 +1,82 @@
|
||||
// v0.40.6.0 — best-effort.ts contract tests.
|
||||
//
|
||||
// Pins the empty-filter contract: pack-load failure returns null, NOT
|
||||
// hardcoded defaults. Four call sites in Phase 8 (whoknows, find-experts,
|
||||
// facts/eligibility, enrichment-service) depend on this contract.
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'bun:test';
|
||||
import { mkdirSync, mkdtempSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import { loadActivePackBestEffort } from '../src/core/schema-pack/best-effort.ts';
|
||||
import {
|
||||
__setPackLocatorForTests,
|
||||
_resetPackLocatorForTests,
|
||||
} from '../src/core/schema-pack/load-active.ts';
|
||||
import { _resetPackCacheForTests } from '../src/core/schema-pack/registry.ts';
|
||||
import type { OperationContext } from '../src/core/operations.ts';
|
||||
import { withEnv } from './helpers/with-env.ts';
|
||||
|
||||
let tmpDir: string;
|
||||
|
||||
beforeEach(() => {
|
||||
_resetPackCacheForTests();
|
||||
_resetPackLocatorForTests();
|
||||
tmpDir = mkdtempSync(join(tmpdir(), 'gbrain-best-effort-test-'));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
_resetPackCacheForTests();
|
||||
_resetPackLocatorForTests();
|
||||
try { rmSync(tmpDir, { recursive: true, force: true }); } catch { /* swallow */ }
|
||||
});
|
||||
|
||||
function fakeCtx(remote = false): OperationContext {
|
||||
return {
|
||||
engine: null as never,
|
||||
config: {},
|
||||
logger: { info: () => {}, warn: () => {}, error: () => {} } as never,
|
||||
dryRun: false,
|
||||
remote,
|
||||
} as OperationContext;
|
||||
}
|
||||
|
||||
describe('loadActivePackBestEffort', () => {
|
||||
it('returns ResolvedPack when load succeeds (default bundled gbrain-base)', async () => {
|
||||
// No locator override → defaults to bundled gbrain-base resolution.
|
||||
await withEnv({ GBRAIN_HOME: tmpDir, GBRAIN_SCHEMA_PACK: undefined }, async () => {
|
||||
const result = await loadActivePackBestEffort(fakeCtx());
|
||||
expect(result).not.toBeNull();
|
||||
expect(result?.manifest.name).toBe('gbrain-base');
|
||||
});
|
||||
});
|
||||
|
||||
it('returns null when the resolved pack is not on disk', async () => {
|
||||
__setPackLocatorForTests(() => null);
|
||||
await withEnv({ GBRAIN_HOME: tmpDir, GBRAIN_SCHEMA_PACK: 'never-installed' }, async () => {
|
||||
const result = await loadActivePackBestEffort(fakeCtx());
|
||||
expect(result).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
it('returns null when the pack file is corrupt', async () => {
|
||||
const packDir = join(tmpDir, 'schema-packs', 'corrupt-pack');
|
||||
mkdirSync(packDir, { recursive: true });
|
||||
const packPath = join(packDir, 'pack.yaml');
|
||||
writeFileSync(packPath, 'this: is: not: valid: yaml: at all: \n}}{', 'utf-8');
|
||||
__setPackLocatorForTests((name) => (name === 'corrupt-pack' ? packPath : null));
|
||||
await withEnv({ GBRAIN_HOME: tmpDir, GBRAIN_SCHEMA_PACK: 'corrupt-pack' }, async () => {
|
||||
const result = await loadActivePackBestEffort(fakeCtx());
|
||||
expect(result).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
it('NEVER throws on any failure path (the load-bearing best-effort contract)', async () => {
|
||||
// Force load to throw via a locator that returns garbage.
|
||||
__setPackLocatorForTests(() => { throw new Error('synthetic disk failure'); });
|
||||
await withEnv({ GBRAIN_HOME: tmpDir }, async () => {
|
||||
// resolves, doesn't throw.
|
||||
await expect(loadActivePackBestEffort(fakeCtx())).resolves.toBeNull();
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,348 @@
|
||||
// v0.40.6.0 — lint-rules.ts unit tests. 36 cases (11 rules covering each
|
||||
// of clean / single-violation / multi-violation paths plus the audit-aware
|
||||
// rule's empty-DB and audit-best-effort paths).
|
||||
|
||||
import { describe, expect, it } from 'bun:test';
|
||||
import type { SchemaPackManifest } from '../src/core/schema-pack/manifest-v1.ts';
|
||||
import {
|
||||
aliasShadowsType,
|
||||
aliasDeclaredByTwoTypes,
|
||||
aliasReferencesUndeclaredType,
|
||||
enrichableTypesUndeclared,
|
||||
linkTypesUndeclared,
|
||||
frontmatterLinksUndeclared,
|
||||
expertRoutingWithoutPrefix,
|
||||
prefixCollision,
|
||||
prefixStrictSubsetOverlap,
|
||||
runAllLintRules,
|
||||
runFilePlaneLintRules,
|
||||
FILE_PLANE_LINT_RULES,
|
||||
ALL_LINT_RULES,
|
||||
} from '../src/core/schema-pack/lint-rules.ts';
|
||||
|
||||
function mk(opts: Partial<SchemaPackManifest>): SchemaPackManifest {
|
||||
const baseTypes = opts.page_types ?? [];
|
||||
return {
|
||||
api_version: 'gbrain-schema-pack-v1',
|
||||
name: opts.name ?? 'p',
|
||||
version: '1.0.0',
|
||||
description: '',
|
||||
gbrain_min_version: '0.38.0',
|
||||
extends: null,
|
||||
borrow_from: [],
|
||||
page_types: baseTypes,
|
||||
link_types: opts.link_types ?? [],
|
||||
frontmatter_links: opts.frontmatter_links ?? [],
|
||||
takes_kinds: ['fact', 'take', 'bet', 'hunch'],
|
||||
enrichable_types: opts.enrichable_types ?? [],
|
||||
filing_rules: [],
|
||||
} as SchemaPackManifest;
|
||||
}
|
||||
|
||||
const baseType = (over: { name: string; aliases?: string[]; extractable?: boolean; expert?: boolean; prefixes?: string[] }) => ({
|
||||
name: over.name,
|
||||
primitive: 'entity' as const,
|
||||
path_prefixes: over.prefixes ?? [],
|
||||
aliases: over.aliases ?? [],
|
||||
extractable: over.extractable ?? false,
|
||||
expert_routing: over.expert ?? false,
|
||||
});
|
||||
|
||||
describe('aliasShadowsType', () => {
|
||||
it('clean: no aliases shadow type names', async () => {
|
||||
const m = mk({ page_types: [baseType({ name: 'person', aliases: ['alias-only'] })] });
|
||||
expect(await aliasShadowsType(m)).toEqual([]);
|
||||
});
|
||||
|
||||
it('single: alias matches another declared type', async () => {
|
||||
const m = mk({ page_types: [
|
||||
baseType({ name: 'person' }),
|
||||
baseType({ name: 'researcher', aliases: ['person'] }),
|
||||
] });
|
||||
const issues = await aliasShadowsType(m);
|
||||
expect(issues.length).toBe(1);
|
||||
expect(issues[0]!.rule).toBe('alias_shadows_type');
|
||||
expect(issues[0]!.severity).toBe('error');
|
||||
});
|
||||
|
||||
it('skips when alias matches self (self-alias is degenerate but not shadow)', async () => {
|
||||
const m = mk({ page_types: [baseType({ name: 'person', aliases: ['person'] })] });
|
||||
expect(await aliasShadowsType(m)).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('aliasDeclaredByTwoTypes', () => {
|
||||
it('clean: each alias declared by at most one type', async () => {
|
||||
expect(await aliasDeclaredByTwoTypes(mk({ page_types: [baseType({ name: 'p', aliases: ['x'] })] }))).toEqual([]);
|
||||
});
|
||||
|
||||
it('flags alias claimed by two types', async () => {
|
||||
const m = mk({ page_types: [
|
||||
baseType({ name: 'a', aliases: ['shared'] }),
|
||||
baseType({ name: 'b', aliases: ['shared'] }),
|
||||
] });
|
||||
const issues = await aliasDeclaredByTwoTypes(m);
|
||||
expect(issues.length).toBe(1);
|
||||
expect(issues[0]!.severity).toBe('error');
|
||||
expect(issues[0]!.message).toContain('shared');
|
||||
expect(issues[0]!.message).toContain('a');
|
||||
expect(issues[0]!.message).toContain('b');
|
||||
});
|
||||
|
||||
it('flags multiple distinct duplicate-alias collisions', async () => {
|
||||
const m = mk({ page_types: [
|
||||
baseType({ name: 'a', aliases: ['x', 'y'] }),
|
||||
baseType({ name: 'b', aliases: ['x', 'y'] }),
|
||||
] });
|
||||
const issues = await aliasDeclaredByTwoTypes(m);
|
||||
expect(issues.length).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('aliasReferencesUndeclaredType', () => {
|
||||
it('clean: aliases all match declared types', async () => {
|
||||
const m = mk({ page_types: [
|
||||
baseType({ name: 'person' }),
|
||||
baseType({ name: 'researcher', aliases: ['person'] }),
|
||||
] });
|
||||
expect(await aliasReferencesUndeclaredType(m)).toEqual([]);
|
||||
});
|
||||
|
||||
it('flags alias pointing at undeclared type', async () => {
|
||||
const m = mk({ page_types: [baseType({ name: 'r', aliases: ['ghost'] })] });
|
||||
const issues = await aliasReferencesUndeclaredType(m);
|
||||
expect(issues.length).toBe(1);
|
||||
expect(issues[0]!.severity).toBe('warning');
|
||||
expect(issues[0]!.message).toContain('ghost');
|
||||
});
|
||||
|
||||
it('flags multiple undeclared references separately', async () => {
|
||||
const m = mk({ page_types: [baseType({ name: 'r', aliases: ['g1', 'g2'] })] });
|
||||
expect((await aliasReferencesUndeclaredType(m)).length).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('enrichableTypesUndeclared', () => {
|
||||
it('clean: all enrichable_types are declared page_types', async () => {
|
||||
const m = mk({
|
||||
page_types: [baseType({ name: 'person' })],
|
||||
enrichable_types: [{ type: 'person', rubric: 'r' }],
|
||||
});
|
||||
expect(await enrichableTypesUndeclared(m)).toEqual([]);
|
||||
});
|
||||
|
||||
it('flags enrichable that names a ghost type', async () => {
|
||||
const m = mk({
|
||||
page_types: [baseType({ name: 'person' })],
|
||||
enrichable_types: [{ type: 'ghost', rubric: 'r' }],
|
||||
});
|
||||
const issues = await enrichableTypesUndeclared(m);
|
||||
expect(issues.length).toBe(1);
|
||||
expect(issues[0]!.severity).toBe('error');
|
||||
});
|
||||
|
||||
it('aggregates multiple ghost references', async () => {
|
||||
const m = mk({
|
||||
page_types: [baseType({ name: 'person' })],
|
||||
enrichable_types: [
|
||||
{ type: 'ghost1', rubric: 'r' },
|
||||
{ type: 'ghost2', rubric: 'r' },
|
||||
],
|
||||
});
|
||||
expect((await enrichableTypesUndeclared(m)).length).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('linkTypesUndeclared', () => {
|
||||
it('clean: inference targets resolve', async () => {
|
||||
const m = mk({
|
||||
page_types: [baseType({ name: 'person' }), baseType({ name: 'company' })],
|
||||
link_types: [{ name: 'works_at', inference: { page_type: 'person', target_type: 'company' } }],
|
||||
});
|
||||
expect(await linkTypesUndeclared(m)).toEqual([]);
|
||||
});
|
||||
|
||||
it('flags inference.page_type referencing ghost', async () => {
|
||||
const m = mk({
|
||||
page_types: [baseType({ name: 'company' })],
|
||||
link_types: [{ name: 'works_at', inference: { page_type: 'ghost', target_type: 'company' } }],
|
||||
});
|
||||
expect((await linkTypesUndeclared(m)).length).toBe(1);
|
||||
});
|
||||
|
||||
it('flags both page_type AND target_type independently', async () => {
|
||||
const m = mk({
|
||||
page_types: [baseType({ name: 'person' })],
|
||||
link_types: [{ name: 'l', inference: { page_type: 'g1', target_type: 'g2' } }],
|
||||
});
|
||||
expect((await linkTypesUndeclared(m)).length).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('frontmatterLinksUndeclared', () => {
|
||||
it('clean: page_type + link_type both resolve', async () => {
|
||||
const m = mk({
|
||||
page_types: [baseType({ name: 'meeting' })],
|
||||
link_types: [{ name: 'attended' }],
|
||||
frontmatter_links: [{ page_type: 'meeting', fields: ['attendees'], link_type: 'attended' }],
|
||||
});
|
||||
expect(await frontmatterLinksUndeclared(m)).toEqual([]);
|
||||
});
|
||||
|
||||
it('flags unknown page_type', async () => {
|
||||
const m = mk({
|
||||
page_types: [],
|
||||
link_types: [{ name: 'attended' }],
|
||||
frontmatter_links: [{ page_type: 'ghost', fields: ['x'], link_type: 'attended' }],
|
||||
});
|
||||
const issues = await frontmatterLinksUndeclared(m);
|
||||
expect(issues.length).toBe(1);
|
||||
expect(issues[0]!.rule).toBe('frontmatter_links_undeclared_page_type');
|
||||
});
|
||||
|
||||
it('flags unknown link_type', async () => {
|
||||
const m = mk({
|
||||
page_types: [baseType({ name: 'meeting' })],
|
||||
link_types: [],
|
||||
frontmatter_links: [{ page_type: 'meeting', fields: ['x'], link_type: 'ghost' }],
|
||||
});
|
||||
const issues = await frontmatterLinksUndeclared(m);
|
||||
expect(issues.length).toBe(1);
|
||||
expect(issues[0]!.rule).toBe('frontmatter_links_undeclared_link_type');
|
||||
});
|
||||
});
|
||||
|
||||
describe('expertRoutingWithoutPrefix', () => {
|
||||
it('clean: expert types have prefixes', async () => {
|
||||
const m = mk({ page_types: [baseType({ name: 'r', expert: true, prefixes: ['people/'] })] });
|
||||
expect(await expertRoutingWithoutPrefix(m)).toEqual([]);
|
||||
});
|
||||
|
||||
it('warns: expert-routed type lacks prefix', async () => {
|
||||
const m = mk({ page_types: [baseType({ name: 'r', expert: true })] });
|
||||
const issues = await expertRoutingWithoutPrefix(m);
|
||||
expect(issues.length).toBe(1);
|
||||
expect(issues[0]!.severity).toBe('warning');
|
||||
});
|
||||
|
||||
it('skips non-expert types without prefix (legitimate concept-only types)', async () => {
|
||||
const m = mk({ page_types: [baseType({ name: 'concept' })] });
|
||||
expect(await expertRoutingWithoutPrefix(m)).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('prefixCollision', () => {
|
||||
it('clean: each prefix declared by only one type', async () => {
|
||||
const m = mk({ page_types: [
|
||||
baseType({ name: 'a', prefixes: ['a/'] }),
|
||||
baseType({ name: 'b', prefixes: ['b/'] }),
|
||||
] });
|
||||
expect(await prefixCollision(m)).toEqual([]);
|
||||
});
|
||||
|
||||
it('flags two types declaring same prefix', async () => {
|
||||
const m = mk({ page_types: [
|
||||
baseType({ name: 'a', prefixes: ['shared/'] }),
|
||||
baseType({ name: 'b', prefixes: ['shared/'] }),
|
||||
] });
|
||||
const issues = await prefixCollision(m);
|
||||
expect(issues.length).toBe(1);
|
||||
expect(issues[0]!.severity).toBe('error');
|
||||
});
|
||||
|
||||
it('aggregates multiple prefix collisions', async () => {
|
||||
const m = mk({ page_types: [
|
||||
baseType({ name: 'a', prefixes: ['x/', 'y/'] }),
|
||||
baseType({ name: 'b', prefixes: ['x/', 'y/'] }),
|
||||
] });
|
||||
expect((await prefixCollision(m)).length).toBe(2);
|
||||
});
|
||||
});
|
||||
|
||||
describe('prefixStrictSubsetOverlap', () => {
|
||||
it('clean: prefixes are unrelated', async () => {
|
||||
const m = mk({ page_types: [
|
||||
baseType({ name: 'a', prefixes: ['people/'] }),
|
||||
baseType({ name: 'b', prefixes: ['companies/'] }),
|
||||
] });
|
||||
expect(await prefixStrictSubsetOverlap(m)).toEqual([]);
|
||||
});
|
||||
|
||||
it('flags one type prefix that is a strict subset of another', async () => {
|
||||
const m = mk({ page_types: [
|
||||
baseType({ name: 'researcher', prefixes: ['people/researchers/'] }),
|
||||
baseType({ name: 'person', prefixes: ['people/'] }),
|
||||
] });
|
||||
const issues = await prefixStrictSubsetOverlap(m);
|
||||
// strict-subset detection fires for the researcher prefix.
|
||||
expect(issues.length).toBeGreaterThanOrEqual(1);
|
||||
expect(issues[0]!.severity).toBe('warning');
|
||||
});
|
||||
|
||||
it('does not flag identical prefixes (that is prefixCollision territory)', async () => {
|
||||
const m = mk({ page_types: [
|
||||
baseType({ name: 'a', prefixes: ['x/'] }),
|
||||
baseType({ name: 'b', prefixes: ['x/'] }),
|
||||
] });
|
||||
// prefixStrictSubsetOverlap only fires on STRICT subsets; identical is collision's job.
|
||||
expect(await prefixStrictSubsetOverlap(m)).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
describe('runFilePlaneLintRules — composition', () => {
|
||||
it('returns ok:true for a clean manifest', async () => {
|
||||
const m = mk({ page_types: [baseType({ name: 'person', prefixes: ['people/'] })] });
|
||||
const report = await runFilePlaneLintRules(m);
|
||||
expect(report.ok).toBe(true);
|
||||
expect(report.errors).toEqual([]);
|
||||
});
|
||||
|
||||
it('returns ok:false when any error fires', async () => {
|
||||
const m = mk({ page_types: [
|
||||
baseType({ name: 'a', prefixes: ['x/'] }),
|
||||
baseType({ name: 'b', prefixes: ['x/'] }),
|
||||
] });
|
||||
const report = await runFilePlaneLintRules(m);
|
||||
expect(report.ok).toBe(false);
|
||||
expect(report.errors.length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('separates warnings from errors', async () => {
|
||||
const m = mk({ page_types: [baseType({ name: 'r', expert: true })] });
|
||||
const report = await runFilePlaneLintRules(m);
|
||||
expect(report.ok).toBe(true);
|
||||
expect(report.errors).toEqual([]);
|
||||
expect(report.warnings.length).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('skips DB-aware rules (file-plane only)', async () => {
|
||||
const m = mk({ page_types: [baseType({ name: 'r', extractable: true, prefixes: ['ghost/'] })] });
|
||||
const report = await runFilePlaneLintRules(m);
|
||||
// extractable_empty_corpus needs an engine; this should NOT fire here.
|
||||
expect(report.warnings.find((w) => w.rule === 'extractable_empty_corpus')).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('runAllLintRules — composition', () => {
|
||||
it('without engine, behaves like runFilePlaneLintRules', async () => {
|
||||
const m = mk({ page_types: [baseType({ name: 'r', extractable: true })] });
|
||||
const report = await runAllLintRules(m);
|
||||
expect(report.warnings.find((w) => w.rule === 'extractable_empty_corpus')).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe('rule registry shape', () => {
|
||||
it('ALL_LINT_RULES contains 11 rules', () => {
|
||||
expect(ALL_LINT_RULES.length).toBe(11);
|
||||
});
|
||||
|
||||
it('FILE_PLANE_LINT_RULES excludes the 2 DB-aware rules', () => {
|
||||
expect(FILE_PLANE_LINT_RULES.length).toBe(9);
|
||||
expect(FILE_PLANE_LINT_RULES.every((r) => !r.planeAware)).toBe(true);
|
||||
});
|
||||
|
||||
it('all rule names are unique', () => {
|
||||
const names = ALL_LINT_RULES.map((r) => r.name);
|
||||
expect(new Set(names).size).toBe(names.length);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,209 @@
|
||||
// v0.40.6.0 — mutate-audit.ts contract tests.
|
||||
//
|
||||
// Pins privacy redaction (sha8 + first-slug-only), success+failure
|
||||
// logging, ISO-week rotation, GBRAIN_AUDIT_DIR honoring, and the
|
||||
// summarizeMutations() shape that doctor + a future audit CLI both bind to.
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'bun:test';
|
||||
import { existsSync, mkdtempSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import {
|
||||
computeMutateAuditPath,
|
||||
logMutationFailure,
|
||||
logMutationSuccess,
|
||||
readRecentMutations,
|
||||
summarizeMutations,
|
||||
type MutationAuditRecord,
|
||||
} from '../src/core/schema-pack/mutate-audit.ts';
|
||||
import { withEnv } from './helpers/with-env.ts';
|
||||
|
||||
let auditDir: string;
|
||||
|
||||
beforeEach(() => {
|
||||
auditDir = mkdtempSync(join(tmpdir(), 'gbrain-mutate-audit-test-'));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
try { rmSync(auditDir, { recursive: true, force: true }); } catch { /* swallow */ }
|
||||
});
|
||||
|
||||
describe('privacy posture', () => {
|
||||
it('redacts type name to sha8 by default', async () => {
|
||||
await withEnv({ GBRAIN_AUDIT_DIR: auditDir, GBRAIN_SCHEMA_AUDIT_VERBOSE: undefined }, async () => {
|
||||
await logMutationSuccess({
|
||||
op: 'add_type',
|
||||
pack: 'my-pack',
|
||||
type: 'mental_health_diagnosis',
|
||||
prefix: 'personal/health/oncology/2026-05-23.md',
|
||||
actor: 'cli',
|
||||
});
|
||||
const path = computeMutateAuditPath();
|
||||
const raw = readFileSync(path, 'utf-8');
|
||||
const record = JSON.parse(raw.trim()) as MutationAuditRecord;
|
||||
expect(record.type_redacted).toBe(true);
|
||||
expect(record.type_or_hash).toMatch(/^[0-9a-f]{8}$/);
|
||||
expect(record.type_or_hash).not.toBe('mental_health_diagnosis');
|
||||
expect(record.prefix_first_seg).toBe('personal');
|
||||
expect(raw).not.toContain('oncology');
|
||||
expect(raw).not.toContain('mental_health');
|
||||
});
|
||||
});
|
||||
|
||||
it('writes raw type name when GBRAIN_SCHEMA_AUDIT_VERBOSE=1', async () => {
|
||||
await withEnv({ GBRAIN_AUDIT_DIR: auditDir, GBRAIN_SCHEMA_AUDIT_VERBOSE: '1' }, async () => {
|
||||
await logMutationSuccess({
|
||||
op: 'add_type',
|
||||
pack: 'my-pack',
|
||||
type: 'researcher',
|
||||
prefix: 'people/researchers/',
|
||||
actor: 'cli',
|
||||
});
|
||||
const record = JSON.parse(
|
||||
readFileSync(computeMutateAuditPath(), 'utf-8').trim(),
|
||||
) as MutationAuditRecord;
|
||||
expect(record.type_redacted).toBe(false);
|
||||
expect(record.type_or_hash).toBe('researcher');
|
||||
});
|
||||
});
|
||||
|
||||
it('pack name is NEVER redacted (it is user-chosen and non-PII)', async () => {
|
||||
await withEnv({ GBRAIN_AUDIT_DIR: auditDir }, async () => {
|
||||
await logMutationSuccess({ op: 'add_type', pack: 'my-pack', actor: 'cli' });
|
||||
const record = JSON.parse(
|
||||
readFileSync(computeMutateAuditPath(), 'utf-8').trim(),
|
||||
) as MutationAuditRecord;
|
||||
expect(record.pack).toBe('my-pack');
|
||||
});
|
||||
});
|
||||
|
||||
it('omits type_or_hash when op did not involve a type', async () => {
|
||||
await withEnv({ GBRAIN_AUDIT_DIR: auditDir }, async () => {
|
||||
// Hypothetical pack-level op (none today, but the shape must accept it).
|
||||
await logMutationSuccess({ op: 'add_type', pack: 'p', actor: 'cli' });
|
||||
const r = JSON.parse(readFileSync(computeMutateAuditPath(), 'utf-8').trim()) as MutationAuditRecord;
|
||||
expect(r.type_or_hash).toBeNull();
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('success + failure logging', () => {
|
||||
it('logs success with outcome=success and reason=null', async () => {
|
||||
await withEnv({ GBRAIN_AUDIT_DIR: auditDir }, async () => {
|
||||
await logMutationSuccess({
|
||||
op: 'add_type',
|
||||
pack: 'my-pack',
|
||||
type: 'researcher',
|
||||
actor: 'cli',
|
||||
prev_sha8: 'aaaaaaaa',
|
||||
new_sha8: 'bbbbbbbb',
|
||||
});
|
||||
const r = JSON.parse(readFileSync(computeMutateAuditPath(), 'utf-8').trim()) as MutationAuditRecord;
|
||||
expect(r.outcome).toBe('success');
|
||||
expect(r.reason).toBeNull();
|
||||
expect(r.prev_sha8).toBe('aaaaaaaa');
|
||||
expect(r.new_sha8).toBe('bbbbbbbb');
|
||||
});
|
||||
});
|
||||
|
||||
it('logs failure with outcome=failure + reason code (the C11 signal doctor reads)', async () => {
|
||||
await withEnv({ GBRAIN_AUDIT_DIR: auditDir }, async () => {
|
||||
await logMutationFailure({
|
||||
op: 'add_type',
|
||||
pack: 'gbrain-base',
|
||||
type: 'researcher',
|
||||
actor: 'cli',
|
||||
reason: 'PACK_READONLY',
|
||||
});
|
||||
const r = JSON.parse(readFileSync(computeMutateAuditPath(), 'utf-8').trim()) as MutationAuditRecord;
|
||||
expect(r.outcome).toBe('failure');
|
||||
expect(r.reason).toBe('PACK_READONLY');
|
||||
});
|
||||
});
|
||||
|
||||
it('actor field surfaces mcp:<clientId8> shape', async () => {
|
||||
await withEnv({ GBRAIN_AUDIT_DIR: auditDir }, async () => {
|
||||
await logMutationSuccess({
|
||||
op: 'add_type', pack: 'p', type: 't',
|
||||
actor: 'mcp:abc12345',
|
||||
});
|
||||
const r = JSON.parse(readFileSync(computeMutateAuditPath(), 'utf-8').trim()) as MutationAuditRecord;
|
||||
expect(r.actor).toBe('mcp:abc12345');
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('ISO-week rotation + GBRAIN_AUDIT_DIR', () => {
|
||||
it('writes filename in the schema-mutations-YYYY-Www.jsonl shape', async () => {
|
||||
await withEnv({ GBRAIN_AUDIT_DIR: auditDir }, async () => {
|
||||
const path = computeMutateAuditPath(new Date('2026-05-23T12:00:00Z'));
|
||||
expect(path).toMatch(/schema-mutations-2026-W\d{2}\.jsonl$/);
|
||||
expect(path.startsWith(auditDir)).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
it('honors GBRAIN_AUDIT_DIR override', async () => {
|
||||
const customDir = mkdtempSync(join(tmpdir(), 'gbrain-mutate-custom-'));
|
||||
try {
|
||||
await withEnv({ GBRAIN_AUDIT_DIR: customDir }, async () => {
|
||||
await logMutationSuccess({ op: 'add_type', pack: 'p', actor: 'cli' });
|
||||
const path = computeMutateAuditPath();
|
||||
expect(path.startsWith(customDir)).toBe(true);
|
||||
expect(existsSync(path)).toBe(true);
|
||||
});
|
||||
} finally {
|
||||
rmSync(customDir, { recursive: true, force: true });
|
||||
}
|
||||
});
|
||||
|
||||
it('readRecentMutations returns records sorted within file, skips malformed lines', async () => {
|
||||
await withEnv({ GBRAIN_AUDIT_DIR: auditDir }, async () => {
|
||||
await logMutationSuccess({ op: 'add_type', pack: 'a', actor: 'cli' });
|
||||
await logMutationFailure({ op: 'remove_type', pack: 'a', actor: 'cli', reason: 'TYPE_NOT_FOUND' });
|
||||
// Inject malformed line.
|
||||
const path = computeMutateAuditPath();
|
||||
writeFileSync(path, readFileSync(path, 'utf-8') + 'not-json{{{\n' + '{}\n');
|
||||
const recs = readRecentMutations(7);
|
||||
expect(recs.length).toBeGreaterThanOrEqual(2);
|
||||
expect(recs.some((r) => r.outcome === 'success')).toBe(true);
|
||||
expect(recs.some((r) => r.outcome === 'failure')).toBe(true);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('summarizeMutations — cross-surface parity primitive', () => {
|
||||
it('aggregates by op, outcome, pack, reason, actor', () => {
|
||||
const recs: MutationAuditRecord[] = [
|
||||
{ ts: '2026-01-01T00:00:00Z', op: 'add_type', pack: 'a', type_or_hash: null, type_redacted: true, prefix_first_seg: null, actor: 'cli', outcome: 'success', reason: null, prev_sha8: null, new_sha8: null, batch_id: null },
|
||||
{ ts: '2026-01-01T00:00:01Z', op: 'add_type', pack: 'a', type_or_hash: null, type_redacted: true, prefix_first_seg: null, actor: 'mcp:abc12345', outcome: 'failure', reason: 'PACK_READONLY', prev_sha8: null, new_sha8: null, batch_id: 'batch-1' },
|
||||
{ ts: '2026-01-01T00:00:02Z', op: 'remove_type', pack: 'b', type_or_hash: null, type_redacted: true, prefix_first_seg: null, actor: 'autopilot', outcome: 'success', reason: null, prev_sha8: null, new_sha8: null, batch_id: null },
|
||||
];
|
||||
const s = summarizeMutations(recs);
|
||||
expect(s.total).toBe(3);
|
||||
expect(s.by_op.add_type).toBe(2);
|
||||
expect(s.by_op.remove_type).toBe(1);
|
||||
expect(s.by_outcome).toEqual({ success: 2, failure: 1 });
|
||||
expect(s.by_pack).toEqual({ a: 2, b: 1 });
|
||||
expect(s.by_reason).toEqual({ PACK_READONLY: 1 });
|
||||
// Actor bucketing collapses mcp:* to 'mcp'
|
||||
expect(s.by_actor).toEqual({ cli: 1, mcp: 1, autopilot: 1 });
|
||||
});
|
||||
|
||||
it('returns empty summary for empty input', () => {
|
||||
const s = summarizeMutations([]);
|
||||
expect(s.total).toBe(0);
|
||||
expect(s.by_outcome).toEqual({ success: 0, failure: 0 });
|
||||
});
|
||||
});
|
||||
|
||||
describe('best-effort behavior', () => {
|
||||
it('does not throw when the audit dir is unwritable', async () => {
|
||||
// Point at a path that fs.mkdirSync will fail on (e.g. a regular file).
|
||||
const fakeDir = join(mkdtempSync(join(tmpdir(), 'gbrain-mutate-baddir-')), 'not-a-dir');
|
||||
writeFileSync(fakeDir, 'this-is-a-file-not-a-dir', 'utf-8');
|
||||
await withEnv({ GBRAIN_AUDIT_DIR: fakeDir }, async () => {
|
||||
// Should NOT throw — best-effort posture.
|
||||
await expect(logMutationSuccess({ op: 'add_type', pack: 'p', actor: 'cli' })).resolves.toBeUndefined();
|
||||
});
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,227 @@
|
||||
// v0.40.6.0 — pack-lock.ts contract tests.
|
||||
//
|
||||
// Pins the atomic-acquire, stale-detection, refresh, and cleanup behavior
|
||||
// that the schema cathedral v3 mutation skeleton depends on.
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it, mock } from 'bun:test';
|
||||
import { existsSync, mkdtempSync, openSync, readFileSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import {
|
||||
acquirePackLock,
|
||||
isLockStale,
|
||||
PackLockBusyError,
|
||||
withPackLock,
|
||||
type LockFileRecord,
|
||||
} from '../src/core/schema-pack/pack-lock.ts';
|
||||
|
||||
let lockDir: string;
|
||||
|
||||
beforeEach(() => {
|
||||
lockDir = mkdtempSync(join(tmpdir(), 'gbrain-pack-lock-test-'));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
try { rmSync(lockDir, { recursive: true, force: true }); } catch { /* swallow */ }
|
||||
});
|
||||
|
||||
const liveAlways = (_pid: number): boolean => true;
|
||||
const deadAlways = (_pid: number): boolean => false;
|
||||
|
||||
describe('acquirePackLock — clean acquire', () => {
|
||||
it('atomically creates the lockfile on first call', () => {
|
||||
const result = acquirePackLock('foo', { lockDir });
|
||||
expect(result.outcome).toBe('acquired');
|
||||
expect(existsSync(join(lockDir, 'foo.lock'))).toBe(true);
|
||||
expect(result.record.pid).toBe(process.pid);
|
||||
expect(result.record.ttlMs).toBeGreaterThan(0);
|
||||
});
|
||||
|
||||
it('writes valid JSON record with pid, ts, ttlMs, hostname', () => {
|
||||
acquirePackLock('foo', { lockDir, ttlMs: 12345 });
|
||||
const raw = readFileSync(join(lockDir, 'foo.lock'), 'utf-8');
|
||||
const parsed = JSON.parse(raw) as LockFileRecord;
|
||||
expect(parsed.pid).toBe(process.pid);
|
||||
expect(parsed.ttlMs).toBe(12345);
|
||||
expect(typeof parsed.ts).toBe('number');
|
||||
expect(typeof parsed.hostname).toBe('string');
|
||||
});
|
||||
|
||||
it('auto-creates the parent directory on first acquire', () => {
|
||||
const deepDir = join(lockDir, 'a', 'b', 'c');
|
||||
const result = acquirePackLock('bar', { lockDir: deepDir });
|
||||
expect(result.outcome).toBe('acquired');
|
||||
expect(existsSync(join(deepDir, 'bar.lock'))).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('acquirePackLock — contention', () => {
|
||||
it('refuses when lock is held by live process with non-expired TTL', () => {
|
||||
// Hand-craft a live, fresh lock.
|
||||
const lockPath = join(lockDir, 'foo.lock');
|
||||
const record: LockFileRecord = {
|
||||
pid: 99999,
|
||||
hostname: 'test',
|
||||
ts: Date.now(),
|
||||
ttlMs: 60_000,
|
||||
};
|
||||
writeFileSync(lockPath, JSON.stringify(record), 'utf-8');
|
||||
|
||||
expect(() =>
|
||||
acquirePackLock('foo', { lockDir, isPidAlive: liveAlways }),
|
||||
).toThrow(PackLockBusyError);
|
||||
});
|
||||
|
||||
it('steals lock when holder PID is dead', () => {
|
||||
const lockPath = join(lockDir, 'foo.lock');
|
||||
writeFileSync(lockPath, JSON.stringify({
|
||||
pid: 99999, hostname: 'test', ts: Date.now(), ttlMs: 60_000,
|
||||
}), 'utf-8');
|
||||
|
||||
const result = acquirePackLock('foo', { lockDir, isPidAlive: deadAlways });
|
||||
expect(result.outcome).toBe('stolen_stale');
|
||||
expect(result.record.pid).toBe(process.pid);
|
||||
});
|
||||
|
||||
it('steals lock when TTL is expired even if PID is alive', () => {
|
||||
const lockPath = join(lockDir, 'foo.lock');
|
||||
writeFileSync(lockPath, JSON.stringify({
|
||||
pid: 99999, hostname: 'test', ts: Date.now() - 120_000, ttlMs: 60_000,
|
||||
}), 'utf-8');
|
||||
|
||||
const result = acquirePackLock('foo', { lockDir, isPidAlive: liveAlways });
|
||||
expect(result.outcome).toBe('stolen_stale');
|
||||
});
|
||||
|
||||
it('steals lock with --force even when live + non-stale', () => {
|
||||
const lockPath = join(lockDir, 'foo.lock');
|
||||
writeFileSync(lockPath, JSON.stringify({
|
||||
pid: 99999, hostname: 'test', ts: Date.now(), ttlMs: 60_000,
|
||||
}), 'utf-8');
|
||||
|
||||
const result = acquirePackLock('foo', { lockDir, force: true, isPidAlive: liveAlways });
|
||||
expect(result.outcome).toBe('forced');
|
||||
expect(result.record.pid).toBe(process.pid);
|
||||
});
|
||||
|
||||
it('PackLockBusyError carries heldBy + ageMs + ttlMs', () => {
|
||||
const lockPath = join(lockDir, 'foo.lock');
|
||||
const past = Date.now() - 1500;
|
||||
writeFileSync(lockPath, JSON.stringify({
|
||||
pid: 88888, hostname: 'test', ts: past, ttlMs: 60_000,
|
||||
}), 'utf-8');
|
||||
|
||||
try {
|
||||
acquirePackLock('foo', { lockDir, isPidAlive: liveAlways });
|
||||
throw new Error('expected throw');
|
||||
} catch (err) {
|
||||
expect(err).toBeInstanceOf(PackLockBusyError);
|
||||
const lockErr = err as PackLockBusyError;
|
||||
expect(lockErr.heldBy).toBe(88888);
|
||||
expect(lockErr.ageMs).toBeGreaterThanOrEqual(1500);
|
||||
expect(lockErr.ttlMs).toBe(60_000);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe('acquirePackLock — corruption recovery', () => {
|
||||
it('steals when lockfile content is unparseable', () => {
|
||||
const lockPath = join(lockDir, 'foo.lock');
|
||||
writeFileSync(lockPath, 'not-valid-json{{{', 'utf-8');
|
||||
const result = acquirePackLock('foo', { lockDir, isPidAlive: liveAlways });
|
||||
expect(result.outcome).toBe('stolen_stale');
|
||||
});
|
||||
|
||||
it('steals when lockfile is empty', () => {
|
||||
const lockPath = join(lockDir, 'foo.lock');
|
||||
writeFileSync(lockPath, '', 'utf-8');
|
||||
const result = acquirePackLock('foo', { lockDir, isPidAlive: liveAlways });
|
||||
expect(result.outcome).toBe('stolen_stale');
|
||||
});
|
||||
|
||||
it('steals when lockfile shape is missing required fields', () => {
|
||||
const lockPath = join(lockDir, 'foo.lock');
|
||||
writeFileSync(lockPath, JSON.stringify({ pid: 'not-a-number' }), 'utf-8');
|
||||
const result = acquirePackLock('foo', { lockDir, isPidAlive: liveAlways });
|
||||
expect(result.outcome).toBe('stolen_stale');
|
||||
});
|
||||
});
|
||||
|
||||
describe('isLockStale — policy unit tests', () => {
|
||||
it('returns live when ts is fresh and pid is alive', () => {
|
||||
const rec: LockFileRecord = { pid: 1, hostname: 'h', ts: 1000, ttlMs: 60_000 };
|
||||
expect(isLockStale(rec, 30_000, liveAlways)).toEqual({ stale: false, reason: 'live' });
|
||||
});
|
||||
|
||||
it('returns ttl_expired when age > ttl, regardless of PID', () => {
|
||||
const rec: LockFileRecord = { pid: 1, hostname: 'h', ts: 1000, ttlMs: 1000 };
|
||||
expect(isLockStale(rec, 5000, liveAlways)).toEqual({ stale: true, reason: 'ttl_expired' });
|
||||
});
|
||||
|
||||
it('returns pid_dead when age <= ttl but PID is dead', () => {
|
||||
const rec: LockFileRecord = { pid: 1, hostname: 'h', ts: 1000, ttlMs: 60_000 };
|
||||
expect(isLockStale(rec, 2000, deadAlways)).toEqual({ stale: true, reason: 'pid_dead' });
|
||||
});
|
||||
|
||||
it('checks ttl BEFORE pid (avoids unnecessary kill syscall)', () => {
|
||||
const rec: LockFileRecord = { pid: 1, hostname: 'h', ts: 1000, ttlMs: 1000 };
|
||||
let pidProbed = false;
|
||||
const probe = (_pid: number) => { pidProbed = true; return true; };
|
||||
isLockStale(rec, 5000, probe);
|
||||
expect(pidProbed).toBe(false);
|
||||
});
|
||||
});
|
||||
|
||||
describe('withPackLock — wrapper contract', () => {
|
||||
it('runs the callback and releases lock on success', async () => {
|
||||
let ran = false;
|
||||
await withPackLock('foo', { lockDir }, async () => {
|
||||
ran = true;
|
||||
expect(existsSync(join(lockDir, 'foo.lock'))).toBe(true);
|
||||
});
|
||||
expect(ran).toBe(true);
|
||||
expect(existsSync(join(lockDir, 'foo.lock'))).toBe(false);
|
||||
});
|
||||
|
||||
it('releases lock even when callback throws', async () => {
|
||||
await expect(
|
||||
withPackLock('foo', { lockDir }, async () => {
|
||||
throw new Error('boom');
|
||||
}),
|
||||
).rejects.toThrow('boom');
|
||||
expect(existsSync(join(lockDir, 'foo.lock'))).toBe(false);
|
||||
});
|
||||
|
||||
it('returns the callback value', async () => {
|
||||
const result = await withPackLock('foo', { lockDir }, async () => 42);
|
||||
expect(result).toBe(42);
|
||||
});
|
||||
|
||||
it('serializes two concurrent withPackLock calls (second throws BUSY)', async () => {
|
||||
let firstReleased = false;
|
||||
const first = withPackLock('foo', { lockDir }, async () => {
|
||||
// Hold for 50ms.
|
||||
await new Promise((r) => setTimeout(r, 50));
|
||||
firstReleased = true;
|
||||
});
|
||||
// Give first a moment to acquire.
|
||||
await new Promise((r) => setTimeout(r, 10));
|
||||
await expect(
|
||||
withPackLock('foo', { lockDir, isPidAlive: liveAlways }, async () => 'second'),
|
||||
).rejects.toThrow(PackLockBusyError);
|
||||
await first;
|
||||
expect(firstReleased).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
describe('cleanup invariants', () => {
|
||||
it('does not leak file descriptors across many acquire/release cycles', async () => {
|
||||
// Smoke test — 100 cycles. If we leaked fds, EMFILE would eventually fire.
|
||||
for (let i = 0; i < 100; i++) {
|
||||
await withPackLock('foo', { lockDir }, async () => {
|
||||
return i;
|
||||
});
|
||||
}
|
||||
expect(existsSync(join(lockDir, 'foo.lock'))).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,83 @@
|
||||
// v0.40.6.0 — query-cache-invalidator.ts contract tests.
|
||||
//
|
||||
// Pins the C9 fix: schema mutations DELETE the query_cache for the
|
||||
// affected source so cached search results bound to old page types
|
||||
// don't survive a `sync --apply`.
|
||||
|
||||
import { afterAll, beforeAll, beforeEach, describe, expect, it } from 'bun:test';
|
||||
import { PGLiteEngine } from '../src/core/pglite-engine.ts';
|
||||
import { resetPgliteState } from './helpers/reset-pglite.ts';
|
||||
import { invalidateQueryCache } from '../src/core/schema-pack/query-cache-invalidator.ts';
|
||||
|
||||
let engine: PGLiteEngine;
|
||||
|
||||
beforeAll(async () => {
|
||||
engine = new PGLiteEngine();
|
||||
await engine.connect({});
|
||||
await engine.initSchema();
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
await engine.disconnect();
|
||||
});
|
||||
|
||||
beforeEach(async () => {
|
||||
await resetPgliteState(engine);
|
||||
});
|
||||
|
||||
async function seedCacheRow(sourceId: string, queryText: string): Promise<void> {
|
||||
// Use raw INSERT to bypass the semantic-similarity path. We're testing the
|
||||
// CLEAR behavior, not the LOOKUP behavior.
|
||||
await engine.executeRaw(
|
||||
`INSERT INTO query_cache (id, query_text, source_id, knobs_hash, embedding, results, meta, ttl_seconds, created_at)
|
||||
VALUES ($1, $2, $3, 'v3:test', NULL, '[]'::jsonb, '{}'::jsonb, 3600, now())`,
|
||||
[`${sourceId}-${queryText}-id`, queryText, sourceId],
|
||||
);
|
||||
}
|
||||
|
||||
async function countCacheRows(sourceId?: string): Promise<number> {
|
||||
const sql = sourceId
|
||||
? `SELECT COUNT(*)::int AS n FROM query_cache WHERE source_id = $1`
|
||||
: `SELECT COUNT(*)::int AS n FROM query_cache`;
|
||||
const rows = await engine.executeRaw<{ n: number }>(sql, sourceId ? [sourceId] : []);
|
||||
return rows[0]?.n ?? 0;
|
||||
}
|
||||
|
||||
describe('invalidateQueryCache', () => {
|
||||
it('clears all rows for a given source_id', async () => {
|
||||
await seedCacheRow('source-a', 'q1');
|
||||
await seedCacheRow('source-a', 'q2');
|
||||
await seedCacheRow('source-b', 'q1');
|
||||
expect(await countCacheRows('source-a')).toBe(2);
|
||||
|
||||
const result = await invalidateQueryCache(engine, 'source-a');
|
||||
expect(result.rows_invalidated).toBe(2);
|
||||
expect(await countCacheRows('source-a')).toBe(0);
|
||||
expect(await countCacheRows('source-b')).toBe(1);
|
||||
});
|
||||
|
||||
it('clears all rows when sourceId is omitted', async () => {
|
||||
await seedCacheRow('source-a', 'q1');
|
||||
await seedCacheRow('source-b', 'q2');
|
||||
const result = await invalidateQueryCache(engine);
|
||||
expect(result.rows_invalidated).toBe(2);
|
||||
expect(await countCacheRows()).toBe(0);
|
||||
});
|
||||
|
||||
it('is idempotent on empty cache', async () => {
|
||||
const r1 = await invalidateQueryCache(engine, 'source-a');
|
||||
const r2 = await invalidateQueryCache(engine, 'source-a');
|
||||
expect(r1.rows_invalidated).toBe(0);
|
||||
expect(r2.rows_invalidated).toBe(0);
|
||||
});
|
||||
|
||||
it('returns {rows_invalidated: 0} silently if engine call fails (never throws)', async () => {
|
||||
// Build a stub engine whose executeRaw always throws.
|
||||
const broken = {
|
||||
kind: 'pglite',
|
||||
executeRaw: async () => { throw new Error('synthetic'); },
|
||||
} as unknown as PGLiteEngine;
|
||||
const result = await invalidateQueryCache(broken, 'source-a');
|
||||
expect(result.rows_invalidated).toBe(0);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,235 @@
|
||||
// v0.40.6.0 — registry.ts cache invalidation + stat-mtime TTL tests.
|
||||
//
|
||||
// Pins codex C6 fix (parent-pack edits cascade-invalidate children) and
|
||||
// D11 + D13 (stat-mtime TTL gate keeps hot path cheap; cross-process
|
||||
// mutations get picked up within STAT_TTL_MS).
|
||||
|
||||
import { afterEach, beforeEach, describe, expect, it } from 'bun:test';
|
||||
import { mkdtempSync, rmSync, writeFileSync } from 'node:fs';
|
||||
import { tmpdir } from 'node:os';
|
||||
import { join } from 'node:path';
|
||||
import {
|
||||
invalidatePackCache,
|
||||
resolvePack,
|
||||
tryCachedPack,
|
||||
_cacheNamesForTests,
|
||||
_cacheSizeForTests,
|
||||
_resetPackCacheForTests,
|
||||
STAT_TTL_MS_DEFAULT,
|
||||
} from '../src/core/schema-pack/registry.ts';
|
||||
import type { SchemaPackManifest } from '../src/core/schema-pack/manifest-v1.ts';
|
||||
import { withEnv } from './helpers/with-env.ts';
|
||||
|
||||
let tmpDir: string;
|
||||
|
||||
function fakeManifest(name: string, opts: { extends?: string; version?: string } = {}): SchemaPackManifest {
|
||||
return {
|
||||
api_version: 'gbrain-schema-pack-v1',
|
||||
name,
|
||||
version: opts.version ?? '1.0.0',
|
||||
description: '',
|
||||
gbrain_min_version: '0.38.0',
|
||||
extends: opts.extends ?? null,
|
||||
borrow_from: [],
|
||||
page_types: [
|
||||
{
|
||||
name: 'person',
|
||||
primitive: 'entity',
|
||||
path_prefixes: ['people/'],
|
||||
aliases: [],
|
||||
extractable: false,
|
||||
expert_routing: false,
|
||||
},
|
||||
],
|
||||
link_types: [],
|
||||
frontmatter_links: [],
|
||||
takes_kinds: ['fact', 'take', 'bet', 'hunch'],
|
||||
enrichable_types: [],
|
||||
filing_rules: [],
|
||||
} as SchemaPackManifest;
|
||||
}
|
||||
|
||||
beforeEach(() => {
|
||||
_resetPackCacheForTests();
|
||||
tmpDir = mkdtempSync(join(tmpdir(), 'gbrain-registry-test-'));
|
||||
});
|
||||
|
||||
afterEach(() => {
|
||||
_resetPackCacheForTests();
|
||||
try { rmSync(tmpDir, { recursive: true, force: true }); } catch { /* swallow */ }
|
||||
});
|
||||
|
||||
describe('invalidatePackCache — basic', () => {
|
||||
it('returns invalidated: [] when no entries exist', () => {
|
||||
const r = invalidatePackCache('nonexistent');
|
||||
expect(r.invalidated).toEqual(['nonexistent']);
|
||||
expect(_cacheSizeForTests()).toBe(0);
|
||||
});
|
||||
|
||||
it('invalidates a single cached entry by name', async () => {
|
||||
await resolvePack(fakeManifest('a'), async () => { throw new Error('no parent'); });
|
||||
expect(_cacheSizeForTests()).toBe(1);
|
||||
invalidatePackCache('a');
|
||||
expect(_cacheSizeForTests()).toBe(0);
|
||||
});
|
||||
|
||||
it('invalidates all entries when called with no argument', async () => {
|
||||
await resolvePack(fakeManifest('a'), async () => { throw new Error('no parent'); });
|
||||
await resolvePack(fakeManifest('b'), async () => { throw new Error('no parent'); });
|
||||
expect(_cacheSizeForTests()).toBe(2);
|
||||
const r = invalidatePackCache();
|
||||
expect(r.invalidated.sort()).toEqual(['a', 'b']);
|
||||
expect(_cacheSizeForTests()).toBe(0);
|
||||
});
|
||||
});
|
||||
|
||||
describe('invalidatePackCache — extends-chain cascade (codex C6 fix)', () => {
|
||||
it('invalidating a parent cascades to every child that extends it', async () => {
|
||||
const parentManifest = fakeManifest('p');
|
||||
const child1Manifest = fakeManifest('c1', { extends: 'p' });
|
||||
const child2Manifest = fakeManifest('c2', { extends: 'p' });
|
||||
const grandchildManifest = fakeManifest('g', { extends: 'c1' });
|
||||
|
||||
const loadByName = async (name: string): Promise<SchemaPackManifest> => {
|
||||
if (name === 'p') return parentManifest;
|
||||
if (name === 'c1') return child1Manifest;
|
||||
if (name === 'c2') return child2Manifest;
|
||||
throw new Error('unknown parent in test');
|
||||
};
|
||||
await resolvePack(parentManifest, loadByName);
|
||||
await resolvePack(child1Manifest, loadByName);
|
||||
await resolvePack(child2Manifest, loadByName);
|
||||
await resolvePack(grandchildManifest, loadByName);
|
||||
expect(_cacheSizeForTests()).toBe(4);
|
||||
|
||||
const result = invalidatePackCache('p');
|
||||
// p, c1, c2 directly contain 'p' in their chain; g contains c1 which
|
||||
// contains p. Cascade evicts p, c1, c2 (one-hop). g has 'c1' + 'p' in
|
||||
// its chain (via the extends walk during resolve), so it's also
|
||||
// evicted. The dependent set is built from cached entries' chain arrays.
|
||||
expect(new Set(result.invalidated)).toEqual(new Set(['p', 'c1', 'c2', 'g']));
|
||||
expect(_cacheSizeForTests()).toBe(0);
|
||||
});
|
||||
|
||||
it('invalidating a leaf does NOT touch siblings or parent', async () => {
|
||||
const p = fakeManifest('p');
|
||||
const c1 = fakeManifest('c1', { extends: 'p' });
|
||||
const c2 = fakeManifest('c2', { extends: 'p' });
|
||||
const loadByName = async (n: string) => (n === 'p' ? p : n === 'c1' ? c1 : c2);
|
||||
await resolvePack(p, loadByName);
|
||||
await resolvePack(c1, loadByName);
|
||||
await resolvePack(c2, loadByName);
|
||||
|
||||
invalidatePackCache('c1');
|
||||
expect(_cacheNamesForTests().sort()).toEqual(['c2', 'p']);
|
||||
});
|
||||
});
|
||||
|
||||
describe('tryCachedPack — TTL gate', () => {
|
||||
it('returns null when name is not cached', () => {
|
||||
expect(tryCachedPack('never-seen')).toBeNull();
|
||||
});
|
||||
|
||||
it('returns the cached resolved pack on hot path', async () => {
|
||||
const m = fakeManifest('foo');
|
||||
const resolved = await resolvePack(m, async () => { throw new Error('no parent'); });
|
||||
const hit = tryCachedPack('foo');
|
||||
expect(hit).toBe(resolved);
|
||||
expect(hit?.manifest.name).toBe('foo');
|
||||
});
|
||||
|
||||
it('respects GBRAIN_PACK_STAT_TTL_MS env override', async () => {
|
||||
await withEnv({ GBRAIN_PACK_STAT_TTL_MS: '0' }, async () => {
|
||||
// Cache + a file snapshot on disk.
|
||||
const packPath = join(tmpDir, 'foo-pack.yaml');
|
||||
writeFileSync(packPath, 'placeholder', 'utf-8');
|
||||
const m = fakeManifest('foo');
|
||||
await resolvePack(m, async () => { throw new Error('no parent'); }, {
|
||||
loadByPath: (n) => (n === 'foo' ? packPath : null),
|
||||
});
|
||||
// TTL=0 forces a stat on every call. Touch the file → mtime changes.
|
||||
// (small sleep ensures mtimeMs is different)
|
||||
await new Promise((r) => setTimeout(r, 5));
|
||||
writeFileSync(packPath, 'updated', 'utf-8');
|
||||
const hit = tryCachedPack('foo');
|
||||
expect(hit).toBeNull();
|
||||
expect(_cacheNamesForTests()).not.toContain('foo');
|
||||
});
|
||||
});
|
||||
|
||||
it('falls back to default TTL when env override is invalid', async () => {
|
||||
await withEnv({ GBRAIN_PACK_STAT_TTL_MS: 'not-a-number' }, async () => {
|
||||
// Default TTL is 1000ms; just check it doesn't crash + returns hit.
|
||||
const m = fakeManifest('foo');
|
||||
await resolvePack(m, async () => { throw new Error('no parent'); });
|
||||
expect(tryCachedPack('foo')).not.toBeNull();
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('stat-snapshot cross-process invalidation (D11)', () => {
|
||||
it('detects mtime change on the pack file and invalidates', async () => {
|
||||
await withEnv({ GBRAIN_PACK_STAT_TTL_MS: '0' }, async () => {
|
||||
const packPath = join(tmpDir, 'p.yaml');
|
||||
writeFileSync(packPath, 'v1', 'utf-8');
|
||||
const m = fakeManifest('p');
|
||||
await resolvePack(m, async () => { throw new Error('no parent'); }, {
|
||||
loadByPath: (n) => (n === 'p' ? packPath : null),
|
||||
});
|
||||
expect(tryCachedPack('p')).not.toBeNull();
|
||||
|
||||
// Mutate the file mtime.
|
||||
await new Promise((r) => setTimeout(r, 10));
|
||||
writeFileSync(packPath, 'v2', 'utf-8');
|
||||
|
||||
// Stat-TTL gate fires (TTL=0 = always stat), detects change, evicts.
|
||||
expect(tryCachedPack('p')).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
it('detects file deletion and evicts (cross-process delete)', async () => {
|
||||
await withEnv({ GBRAIN_PACK_STAT_TTL_MS: '0' }, async () => {
|
||||
const packPath = join(tmpDir, 'p.yaml');
|
||||
writeFileSync(packPath, 'v1', 'utf-8');
|
||||
const m = fakeManifest('p');
|
||||
await resolvePack(m, async () => { throw new Error('no parent'); }, {
|
||||
loadByPath: (n) => (n === 'p' ? packPath : null),
|
||||
});
|
||||
expect(tryCachedPack('p')).not.toBeNull();
|
||||
|
||||
rmSync(packPath);
|
||||
expect(tryCachedPack('p')).toBeNull();
|
||||
});
|
||||
});
|
||||
|
||||
it('cascades when parent file mtime changes (codex C6 fix at file level)', async () => {
|
||||
await withEnv({ GBRAIN_PACK_STAT_TTL_MS: '0' }, async () => {
|
||||
const parentPath = join(tmpDir, 'parent.yaml');
|
||||
const childPath = join(tmpDir, 'child.yaml');
|
||||
writeFileSync(parentPath, 'parent v1', 'utf-8');
|
||||
writeFileSync(childPath, 'child v1', 'utf-8');
|
||||
|
||||
const parentM = fakeManifest('parent');
|
||||
const childM = fakeManifest('child', { extends: 'parent' });
|
||||
const loadByName = async (n: string) => (n === 'parent' ? parentM : childM);
|
||||
const loadByPath = (n: string) => (n === 'parent' ? parentPath : n === 'child' ? childPath : null);
|
||||
|
||||
await resolvePack(parentM, loadByName, { loadByPath });
|
||||
await resolvePack(childM, loadByName, { loadByPath });
|
||||
|
||||
// Mutate ONLY the parent file.
|
||||
await new Promise((r) => setTimeout(r, 10));
|
||||
writeFileSync(parentPath, 'parent v2', 'utf-8');
|
||||
|
||||
// tryCachedPack on the CHILD must detect parent's mtime change
|
||||
// (parent is in child's chain + files snapshot).
|
||||
expect(tryCachedPack('child')).toBeNull();
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('STAT_TTL_MS_DEFAULT export', () => {
|
||||
it('exports the default constant', () => {
|
||||
expect(STAT_TTL_MS_DEFAULT).toBe(1000);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user