mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-30 03:12:32 +00:00
* fix(security): confine routing dotfiles, skills dir, slugs, and transcription exec Shared src/core/path-confine.ts consolidates the realpath-containment idiom (moved from sources-ops.ts) and adds isTrustedDotfile + isWriteTargetContained. - .gbrain-source (source-resolver) and .gbrain-mount (brain-resolver) walk-up dotfiles are now lstat trust-gated: a symlink, foreign-owned, or world-writable file is refused on multi-user hosts (#418), fail-closed on stat error. - resolveWorkspaceSkillsDir + every skills-dir tier (env, cwd_walk_up, repo_root, cwd_skills, install_path) route through realpath containment so a symlinked workspace/skills can't escape the declared workspace (#419). - resolveSourceId/resolveBrainId realpath both sides of the registered local_path / mount prefix match so a symlinked cwd can't misattribute source/brain. - validateSlug rejects NUL/control, bidi/RTL overrides, backslashes, and URL-encoded path separators at the shared putPage/updateSlug chokepoint; write-through confirms the file path stays within the source tree. - transcribeLargeFile uses execFileSync arg-arrays + fs.rmSync (no shell), so a path with shell metacharacters is never parsed by a shell (#245). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(security): default dynamic-registration clients to authorization_code Self-registered DCR clients (the unauthenticated network registration path) previously defaulted to the client_credentials grant, which bypasses the /authorize consent screen. They now default to authorization_code; an explicit client_credentials request is rejected with invalid_client_metadata unless the operator opts in with the new --enable-dcr-insecure flag. A loud stderr WARNING prints at startup whenever DCR is enabled (#1353). Manual CLI/admin client registration is unchanged (operator-trusted). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(security): schema-lint hardening migration (search_path + view security_invoker) Migration v120 brings existing brains to the same posture as fresh installs: - ALTER VIEW page_links SET (security_invoker = on) on Postgres so the view honors the caller's RLS instead of the owner's (the view-through-RLS bypass). - ALTER FUNCTION ... SET search_path on the gbrain-owned trigger/event functions (both engines, IF EXISTS so engine-only functions are skipped; body untouched, so the load-bearing auto_enable_rls event trigger is unchanged). Closes #171. - Broaden the BYPASSRLS preflight in the historical RLS migration gates to honor superuser and inherited-role BYPASSRLS, so a superuser-connected fresh install no longer aborts (#1385). Fresh-install function definitions in schema.sql / pglite-schema.ts are born-correct (regenerated schema-embedded.ts). scripts/check-search-path.sh is a new CI guard (wired into verify) that fails if a trigger function in the schema base files is added without SET search_path. Postgres-only assertions live in the bootstrap E2E; the PGLite path is covered by test/migration-v120.test.ts. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * v0.42.55.0 fix(security): dotfile/skills/slug confinement, DCR consent default, schema-lint migration Bump VERSION + package.json to 0.42.55.0 and add the CHANGELOG entry for the security-hardening wave (#418 #419 #245 #1353 #1647 #171 #1385). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(security): note the DCR consent default in SECURITY.md (#1353) The "disable client_credentials, only allow authorization_code" guidance is now the built-in DCR default; document the new --enable-dcr-insecure escape hatch. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs(todos): add takes_search + code_def to the federated by-slug P1 (#2200) The v0.42.55.0 eng-review codex pass flagged takes_search (holder-allowlist only) and code_def (brain-wide raw SQL over content_chunks) as remaining same-class surfaces. Noted on the existing #2200 P1 follow-up, with the caveat that the #2399 close-list deliberately keeps #1371/#2200 open until this lands. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * fix(security): correct plpgsql alias collision in #1385 BYPASSRLS gate (real-PG) The broadened BYPASSRLS preflight aliased `pg_roles r`, but several RLS DO-blocks already declare `r record` for their backfill FOR loop, so plpgsql resolved `r.oid`/`r.rolbypassrls` to the unassigned record variable → "record \"r\" is not assigned yet" on real Postgres (PGLite tolerated it; the DATABASE_URL-gated e2e jobs are the backstop). Renamed the subquery alias to `pr` at all 10 migrate.ts sites; also broadened the schema.sql base RLS gate the same way (with the `pr` alias) for #1385 consistency on superuser fresh installs, and regenerated schema-embedded.ts. Also fixes a PRE-EXISTING engine-parity bug (confirmed failing on clean origin/master): the relationalFanout shape compared `canonical_chunk_id`, a serial id that diverges between a fresh PGLite engine and a shared Postgres DB (setupDB TRUNCATEs without RESTART IDENTITY). Compare its presence, not the exact value. Validated on real Postgres (pgvector/pg16): migration v120 applies, the v35 RLS backfill runs, and engine-parity + postgres-bootstrap + jsonb-parity are green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
167 lines
7.5 KiB
TypeScript
167 lines
7.5 KiB
TypeScript
/**
|
|
* Shared disk write-through for the canonical ingestion path.
|
|
*
|
|
* After a page row lands in the DB (via importFromContent / putPage), this
|
|
* renders the row to markdown via `serializePageToMarkdown` and writes it to
|
|
* `sync.repo_path` so the brain repo has a committable `.md` artifact that
|
|
* round-trips cleanly through `gbrain sync`. The file is rendered FROM the DB
|
|
* row, so the two sinks cannot diverge.
|
|
*
|
|
* Extracted from the v0.38 `put_page` write-through (operations.ts) so the
|
|
* `put_page` op AND `gbrain brainstorm/lsd --save` share one implementation
|
|
* instead of hand-rolling parallel (and divergent) copies. The extraction also
|
|
* upgraded the write to be ATOMIC — the original used a bare `writeFileSync`
|
|
* into a live git tree that `gbrain sync` / autopilot actively walk, so a crash
|
|
* mid-write left a partial `.md` that sync would fail to parse. We now write to
|
|
* a unique temp sibling and `rename` into place (rename is atomic on the same
|
|
* filesystem), matching the `.tmp + rename` convention used by
|
|
* import-checkpoint.ts / op-checkpoint.ts.
|
|
*
|
|
* Trust gating (subagent sandbox, dry-run) stays at the CALLER — this helper
|
|
* only does "row exists + repo is a real dir → render + atomic write".
|
|
*/
|
|
|
|
import { existsSync, statSync, mkdirSync, writeFileSync, renameSync, unlinkSync } from 'fs';
|
|
import { dirname, join } from 'path';
|
|
import { randomBytes } from 'crypto';
|
|
import type { BrainEngine } from './engine.ts';
|
|
import { serializePageToMarkdown, resolvePageFilePath } from './markdown.ts';
|
|
import { isWriteTargetContained } from './path-confine.ts';
|
|
|
|
/** Minimal logger surface — structurally compatible with operations.ts `Logger`. */
|
|
export interface WriteThroughLogger {
|
|
warn(msg: string): void;
|
|
}
|
|
|
|
export interface WriteThroughResult {
|
|
written: boolean;
|
|
path?: string;
|
|
/**
|
|
* Non-error reasons the file was not written:
|
|
* - no_repo_configured: the resolved target (source `local_path` or, for a
|
|
* sole-source brain, `sync.repo_path`) is unset (DB-only by design).
|
|
* - repo_not_found: target set but missing / not a directory.
|
|
* - source_repo_belongs_to_other_source: the assigned source has no
|
|
* `local_path`, and `sync.repo_path` is another source's own working tree
|
|
* — #2018: writing here would pollute that sibling's repo, so we skip.
|
|
* - page_not_found_after_write: the DB row isn't readable back (the caller's
|
|
* DB write failed or targeted a different source).
|
|
* - path_escapes_source_root: the computed file path resolves outside the
|
|
* source's working tree (hostile slug row / symlinked subtree) — refused.
|
|
*/
|
|
skipped?: 'no_repo_configured' | 'repo_not_found' | 'source_repo_belongs_to_other_source' | 'page_not_found_after_write' | 'path_escapes_source_root';
|
|
/** Set when the render/write/rename itself threw (EACCES, ENOTDIR, disk full). */
|
|
error?: string;
|
|
}
|
|
|
|
export interface WritePageThroughOpts {
|
|
sourceId?: string;
|
|
/** Merged over the page's own frontmatter at render time (e.g. provenance). */
|
|
frontmatterOverrides?: Record<string, unknown>;
|
|
logger?: WriteThroughLogger;
|
|
}
|
|
|
|
/**
|
|
* Render the DB row for `slug` to markdown and atomically write it under
|
|
* `sync.repo_path`. Never throws — failures are reported via the result's
|
|
* `skipped` / `error` fields (the DB write is the durable sink; the file is
|
|
* best-effort and reconciled by the next `gbrain sync`).
|
|
*/
|
|
export async function writePageThrough(
|
|
engine: BrainEngine,
|
|
slug: string,
|
|
opts: WritePageThroughOpts = {},
|
|
): Promise<WriteThroughResult> {
|
|
const sourceId = opts.sourceId ?? 'default';
|
|
try {
|
|
// #2018: pick the disk target so a page is NEVER written into a different
|
|
// source's working tree. Two legitimate topologies, plus the leak guard:
|
|
// 1. The assigned source has its OWN `local_path` (a separate working
|
|
// tree) → write at that tree's root (matches how `scanOneSource` reads
|
|
// it back; never nested under `.sources/`).
|
|
// 2. No per-source `local_path` → nest under the host repo
|
|
// (`sync.repo_path`): default at the root, non-default under
|
|
// `.sources/<id>/` (the established multi-source layout).
|
|
// 3. LEAK GUARD: if `sync.repo_path` is literally ANOTHER source's own
|
|
// `local_path`, nesting this page there would pollute that sibling's
|
|
// git repo (the reported bug). Skip instead.
|
|
let filePath: string;
|
|
let writeRoot: string;
|
|
const srcRows = await engine.executeRaw<{ local_path: string | null }>(
|
|
`SELECT local_path FROM sources WHERE id = $1`,
|
|
[sourceId],
|
|
);
|
|
const sourceLocalPath = srcRows[0]?.local_path ?? null;
|
|
if (sourceLocalPath) {
|
|
if (!existsSync(sourceLocalPath) || !statSync(sourceLocalPath).isDirectory()) {
|
|
return { written: false, skipped: 'repo_not_found' };
|
|
}
|
|
filePath = join(sourceLocalPath, `${slug}.md`);
|
|
writeRoot = sourceLocalPath;
|
|
} else {
|
|
const repoPath = await engine.getConfig('sync.repo_path');
|
|
if (!repoPath) {
|
|
return { written: false, skipped: 'no_repo_configured' };
|
|
}
|
|
if (!existsSync(repoPath) || !statSync(repoPath).isDirectory()) {
|
|
return { written: false, skipped: 'repo_not_found' };
|
|
}
|
|
// Leak guard: refuse to write into a path that is some OTHER source's
|
|
// own working tree (#2018).
|
|
const collide = await engine.executeRaw<{ one: number }>(
|
|
`SELECT 1 AS one FROM sources WHERE id <> $1 AND local_path = $2 LIMIT 1`,
|
|
[sourceId, repoPath],
|
|
);
|
|
if (collide.length > 0) {
|
|
return { written: false, skipped: 'source_repo_belongs_to_other_source' };
|
|
}
|
|
filePath = resolvePageFilePath(repoPath, slug, sourceId);
|
|
writeRoot = repoPath;
|
|
}
|
|
|
|
// Defense-in-depth (#1647-slug / codex #6): confirm the computed file path
|
|
// stays within the source's working tree before any mkdir/write. validateSlug
|
|
// already rejects `..`/backslash/control/%2e in the slug at write time, so
|
|
// this guards a pre-existing hostile row or a symlinked intermediate dir
|
|
// under the source tree from escaping to an arbitrary filesystem location.
|
|
if (!isWriteTargetContained(filePath, writeRoot)) {
|
|
return { written: false, skipped: 'path_escapes_source_root' };
|
|
}
|
|
|
|
const writtenPage = await engine.getPage(slug, { sourceId });
|
|
if (!writtenPage) {
|
|
return { written: false, skipped: 'page_not_found_after_write' };
|
|
}
|
|
|
|
const tags = await engine.getTags(slug, { sourceId });
|
|
const md = serializePageToMarkdown(writtenPage, tags, {
|
|
frontmatterOverrides: opts.frontmatterOverrides,
|
|
});
|
|
|
|
mkdirSync(dirname(filePath), { recursive: true });
|
|
|
|
// Atomic write: unique temp sibling + rename. Unique name (pid + random)
|
|
// so two concurrent saves to the same target can't clobber each other's
|
|
// temp file. Clean up the temp on any failure so we never leak a stray
|
|
// `.tmp` next to the real file.
|
|
const tmpPath = `${filePath}.tmp.${process.pid}.${randomBytes(4).toString('hex')}`;
|
|
try {
|
|
writeFileSync(tmpPath, md, 'utf8');
|
|
renameSync(tmpPath, filePath);
|
|
} catch (writeErr) {
|
|
try {
|
|
if (existsSync(tmpPath)) unlinkSync(tmpPath);
|
|
} catch {
|
|
// best-effort cleanup; surface the original write error below
|
|
}
|
|
throw writeErr;
|
|
}
|
|
|
|
return { written: true, path: filePath };
|
|
} catch (e) {
|
|
const msg = e instanceof Error ? e.message : String(e);
|
|
opts.logger?.warn(`[write-through] failed for ${slug}: ${msg}`);
|
|
return { written: false, error: msg };
|
|
}
|
|
}
|