mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-28 14:59:47 +00:00
7968f84077
* feat(pace): composable DB-contention pacer primitive createDbPacer/createNoopPacer (concurrency permit + in-band EWMA + jittered cooperative sleep, abort-throws, fail-open) + named pace-mode bundles (env>config>bundle, default off) + shared embed-backfill lock key. * feat(embed): wire DB-pacing into embed paths + single-flight + bounded keyset re-entry embedStaleForSource + CLI embedAllStale/embedAll lower worker count to the resolved cap and observe()/pace() their DB ops; embed job + embed-backfill handler resolve env>config>bundle; CLI --pace flags; --background carries overrides into the job payload; single-flight via shared per-source lock; budget-timer re-arm around paced sleeps; EmbedResult.pacing telemetry. * feat(sync): shared DB-pacer permit across parallel worker engines One pacer spans the per-worker PostgresEngines (the multi-pool permit case); observe() import writes, pace() between files, dispose on all exit paths. * docs(pace): CLAUDE.md Pace Mode section + regenerated llms bundle * fix(pace): pre-landing review fixes (Codex P1/P2) - never unref() the cooperative-sleep timer (could exit mid-sleep) - pace() excludes the wall-clock budget + re-arm after each sleep - pacing only lowers concurrency, never raises above an operator cap - serialized job pace resolves at config tier so GBRAIN_PACE_* still wins - --pace-max-concurrency consumes its value token * chore: bump version and changelog (v0.42.48.0) Native DB-contention pacing for embed/sync backfills. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: bump version to v0.42.49.0 Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
18 lines
718 B
TypeScript
18 lines
718 B
TypeScript
/**
|
|
* Shared lock identity for embed backfills (paced-backfill E-2).
|
|
*
|
|
* The per-source lock key + TTL live here, in a zero-dependency module, so BOTH
|
|
* the `embed-backfill` minion handler AND the CLI `embed --stale` single-flight
|
|
* take the SAME key — a hand-run backfill and a queued job are then mutually
|
|
* exclusive per source. Kept dependency-free to avoid an import cycle between
|
|
* embed.ts, embed-stale.ts, and the handler.
|
|
*/
|
|
|
|
/** Per-source embed-backfill lock id, namespaced like sync's. */
|
|
export function embedBackfillLockId(sourceId: string): string {
|
|
return `gbrain-embed-backfill:${sourceId}`;
|
|
}
|
|
|
|
/** Lock TTL (minutes) for embed backfills. */
|
|
export const EMBED_BACKFILL_LOCK_TTL_MIN = 60;
|