Files
gbrain/src/core/write-through.ts
T
dde1132a29 v0.42.55.0 fix(security): dotfile/skills/slug confinement + DCR consent default + schema-lint migration (#418 #419 #245 #1353 #1647 #171 #1385) (#2399)
* fix(security): confine routing dotfiles, skills dir, slugs, and transcription exec

Shared src/core/path-confine.ts consolidates the realpath-containment idiom
(moved from sources-ops.ts) and adds isTrustedDotfile + isWriteTargetContained.

- .gbrain-source (source-resolver) and .gbrain-mount (brain-resolver) walk-up
  dotfiles are now lstat trust-gated: a symlink, foreign-owned, or world-writable
  file is refused on multi-user hosts (#418), fail-closed on stat error.
- resolveWorkspaceSkillsDir + every skills-dir tier (env, cwd_walk_up, repo_root,
  cwd_skills, install_path) route through realpath containment so a symlinked
  workspace/skills can't escape the declared workspace (#419).
- resolveSourceId/resolveBrainId realpath both sides of the registered local_path
  / mount prefix match so a symlinked cwd can't misattribute source/brain.
- validateSlug rejects NUL/control, bidi/RTL overrides, backslashes, and
  URL-encoded path separators at the shared putPage/updateSlug chokepoint;
  write-through confirms the file path stays within the source tree.
- transcribeLargeFile uses execFileSync arg-arrays + fs.rmSync (no shell), so a
  path with shell metacharacters is never parsed by a shell (#245).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(security): default dynamic-registration clients to authorization_code

Self-registered DCR clients (the unauthenticated network registration path)
previously defaulted to the client_credentials grant, which bypasses the
/authorize consent screen. They now default to authorization_code; an explicit
client_credentials request is rejected with invalid_client_metadata unless the
operator opts in with the new --enable-dcr-insecure flag. A loud stderr WARNING
prints at startup whenever DCR is enabled (#1353). Manual CLI/admin client
registration is unchanged (operator-trusted).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(security): schema-lint hardening migration (search_path + view security_invoker)

Migration v120 brings existing brains to the same posture as fresh installs:

- ALTER VIEW page_links SET (security_invoker = on) on Postgres so the view
  honors the caller's RLS instead of the owner's (the view-through-RLS bypass).
- ALTER FUNCTION ... SET search_path on the gbrain-owned trigger/event functions
  (both engines, IF EXISTS so engine-only functions are skipped; body untouched,
  so the load-bearing auto_enable_rls event trigger is unchanged). Closes #171.
- Broaden the BYPASSRLS preflight in the historical RLS migration gates to honor
  superuser and inherited-role BYPASSRLS, so a superuser-connected fresh install
  no longer aborts (#1385).

Fresh-install function definitions in schema.sql / pglite-schema.ts are
born-correct (regenerated schema-embedded.ts). scripts/check-search-path.sh is a
new CI guard (wired into verify) that fails if a trigger function in the schema
base files is added without SET search_path. Postgres-only assertions live in the
bootstrap E2E; the PGLite path is covered by test/migration-v120.test.ts.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* v0.42.55.0 fix(security): dotfile/skills/slug confinement, DCR consent default, schema-lint migration

Bump VERSION + package.json to 0.42.55.0 and add the CHANGELOG entry for the
security-hardening wave (#418 #419 #245 #1353 #1647 #171 #1385).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(security): note the DCR consent default in SECURITY.md (#1353)

The "disable client_credentials, only allow authorization_code" guidance is now
the built-in DCR default; document the new --enable-dcr-insecure escape hatch.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* docs(todos): add takes_search + code_def to the federated by-slug P1 (#2200)

The v0.42.55.0 eng-review codex pass flagged takes_search (holder-allowlist only)
and code_def (brain-wide raw SQL over content_chunks) as remaining same-class
surfaces. Noted on the existing #2200 P1 follow-up, with the caveat that the
#2399 close-list deliberately keeps #1371/#2200 open until this lands.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

* fix(security): correct plpgsql alias collision in #1385 BYPASSRLS gate (real-PG)

The broadened BYPASSRLS preflight aliased `pg_roles r`, but several RLS DO-blocks
already declare `r record` for their backfill FOR loop, so plpgsql resolved
`r.oid`/`r.rolbypassrls` to the unassigned record variable → "record \"r\" is
not assigned yet" on real Postgres (PGLite tolerated it; the DATABASE_URL-gated
e2e jobs are the backstop). Renamed the subquery alias to `pr` at all 10
migrate.ts sites; also broadened the schema.sql base RLS gate the same way (with
the `pr` alias) for #1385 consistency on superuser fresh installs, and
regenerated schema-embedded.ts.

Also fixes a PRE-EXISTING engine-parity bug (confirmed failing on clean
origin/master): the relationalFanout shape compared `canonical_chunk_id`, a
serial id that diverges between a fresh PGLite engine and a shared Postgres DB
(setupDB TRUNCATEs without RESTART IDENTITY). Compare its presence, not the
exact value.

Validated on real Postgres (pgvector/pg16): migration v120 applies, the v35 RLS
backfill runs, and engine-parity + postgres-bootstrap + jsonb-parity are green.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>

---------

Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-02 08:00:58 -07:00

167 lines
7.5 KiB
TypeScript

/**
* Shared disk write-through for the canonical ingestion path.
*
* After a page row lands in the DB (via importFromContent / putPage), this
* renders the row to markdown via `serializePageToMarkdown` and writes it to
* `sync.repo_path` so the brain repo has a committable `.md` artifact that
* round-trips cleanly through `gbrain sync`. The file is rendered FROM the DB
* row, so the two sinks cannot diverge.
*
* Extracted from the v0.38 `put_page` write-through (operations.ts) so the
* `put_page` op AND `gbrain brainstorm/lsd --save` share one implementation
* instead of hand-rolling parallel (and divergent) copies. The extraction also
* upgraded the write to be ATOMIC — the original used a bare `writeFileSync`
* into a live git tree that `gbrain sync` / autopilot actively walk, so a crash
* mid-write left a partial `.md` that sync would fail to parse. We now write to
* a unique temp sibling and `rename` into place (rename is atomic on the same
* filesystem), matching the `.tmp + rename` convention used by
* import-checkpoint.ts / op-checkpoint.ts.
*
* Trust gating (subagent sandbox, dry-run) stays at the CALLER — this helper
* only does "row exists + repo is a real dir → render + atomic write".
*/
import { existsSync, statSync, mkdirSync, writeFileSync, renameSync, unlinkSync } from 'fs';
import { dirname, join } from 'path';
import { randomBytes } from 'crypto';
import type { BrainEngine } from './engine.ts';
import { serializePageToMarkdown, resolvePageFilePath } from './markdown.ts';
import { isWriteTargetContained } from './path-confine.ts';
/** Minimal logger surface — structurally compatible with operations.ts `Logger`. */
export interface WriteThroughLogger {
warn(msg: string): void;
}
export interface WriteThroughResult {
written: boolean;
path?: string;
/**
* Non-error reasons the file was not written:
* - no_repo_configured: the resolved target (source `local_path` or, for a
* sole-source brain, `sync.repo_path`) is unset (DB-only by design).
* - repo_not_found: target set but missing / not a directory.
* - source_repo_belongs_to_other_source: the assigned source has no
* `local_path`, and `sync.repo_path` is another source's own working tree
* — #2018: writing here would pollute that sibling's repo, so we skip.
* - page_not_found_after_write: the DB row isn't readable back (the caller's
* DB write failed or targeted a different source).
* - path_escapes_source_root: the computed file path resolves outside the
* source's working tree (hostile slug row / symlinked subtree) — refused.
*/
skipped?: 'no_repo_configured' | 'repo_not_found' | 'source_repo_belongs_to_other_source' | 'page_not_found_after_write' | 'path_escapes_source_root';
/** Set when the render/write/rename itself threw (EACCES, ENOTDIR, disk full). */
error?: string;
}
export interface WritePageThroughOpts {
sourceId?: string;
/** Merged over the page's own frontmatter at render time (e.g. provenance). */
frontmatterOverrides?: Record<string, unknown>;
logger?: WriteThroughLogger;
}
/**
* Render the DB row for `slug` to markdown and atomically write it under
* `sync.repo_path`. Never throws — failures are reported via the result's
* `skipped` / `error` fields (the DB write is the durable sink; the file is
* best-effort and reconciled by the next `gbrain sync`).
*/
export async function writePageThrough(
engine: BrainEngine,
slug: string,
opts: WritePageThroughOpts = {},
): Promise<WriteThroughResult> {
const sourceId = opts.sourceId ?? 'default';
try {
// #2018: pick the disk target so a page is NEVER written into a different
// source's working tree. Two legitimate topologies, plus the leak guard:
// 1. The assigned source has its OWN `local_path` (a separate working
// tree) → write at that tree's root (matches how `scanOneSource` reads
// it back; never nested under `.sources/`).
// 2. No per-source `local_path` → nest under the host repo
// (`sync.repo_path`): default at the root, non-default under
// `.sources/<id>/` (the established multi-source layout).
// 3. LEAK GUARD: if `sync.repo_path` is literally ANOTHER source's own
// `local_path`, nesting this page there would pollute that sibling's
// git repo (the reported bug). Skip instead.
let filePath: string;
let writeRoot: string;
const srcRows = await engine.executeRaw<{ local_path: string | null }>(
`SELECT local_path FROM sources WHERE id = $1`,
[sourceId],
);
const sourceLocalPath = srcRows[0]?.local_path ?? null;
if (sourceLocalPath) {
if (!existsSync(sourceLocalPath) || !statSync(sourceLocalPath).isDirectory()) {
return { written: false, skipped: 'repo_not_found' };
}
filePath = join(sourceLocalPath, `${slug}.md`);
writeRoot = sourceLocalPath;
} else {
const repoPath = await engine.getConfig('sync.repo_path');
if (!repoPath) {
return { written: false, skipped: 'no_repo_configured' };
}
if (!existsSync(repoPath) || !statSync(repoPath).isDirectory()) {
return { written: false, skipped: 'repo_not_found' };
}
// Leak guard: refuse to write into a path that is some OTHER source's
// own working tree (#2018).
const collide = await engine.executeRaw<{ one: number }>(
`SELECT 1 AS one FROM sources WHERE id <> $1 AND local_path = $2 LIMIT 1`,
[sourceId, repoPath],
);
if (collide.length > 0) {
return { written: false, skipped: 'source_repo_belongs_to_other_source' };
}
filePath = resolvePageFilePath(repoPath, slug, sourceId);
writeRoot = repoPath;
}
// Defense-in-depth (#1647-slug / codex #6): confirm the computed file path
// stays within the source's working tree before any mkdir/write. validateSlug
// already rejects `..`/backslash/control/%2e in the slug at write time, so
// this guards a pre-existing hostile row or a symlinked intermediate dir
// under the source tree from escaping to an arbitrary filesystem location.
if (!isWriteTargetContained(filePath, writeRoot)) {
return { written: false, skipped: 'path_escapes_source_root' };
}
const writtenPage = await engine.getPage(slug, { sourceId });
if (!writtenPage) {
return { written: false, skipped: 'page_not_found_after_write' };
}
const tags = await engine.getTags(slug, { sourceId });
const md = serializePageToMarkdown(writtenPage, tags, {
frontmatterOverrides: opts.frontmatterOverrides,
});
mkdirSync(dirname(filePath), { recursive: true });
// Atomic write: unique temp sibling + rename. Unique name (pid + random)
// so two concurrent saves to the same target can't clobber each other's
// temp file. Clean up the temp on any failure so we never leak a stray
// `.tmp` next to the real file.
const tmpPath = `${filePath}.tmp.${process.pid}.${randomBytes(4).toString('hex')}`;
try {
writeFileSync(tmpPath, md, 'utf8');
renameSync(tmpPath, filePath);
} catch (writeErr) {
try {
if (existsSync(tmpPath)) unlinkSync(tmpPath);
} catch {
// best-effort cleanup; surface the original write error below
}
throw writeErr;
}
return { written: true, path: filePath };
} catch (e) {
const msg = e instanceof Error ? e.message : String(e);
opts.logger?.warn(`[write-through] failed for ${slug}: ${msg}`);
return { written: false, error: msg };
}
}