fix(webhook): extract links for incremental push syncs (#2849)

GitHub webhook + sync-trigger 'sync' jobs never opted into extraction:
the standalone sync handler defaults noExtract to TRUE, so webhook-
imported pages advanced the source bookmark with links_extracted_at
unstamped — and nothing behind a standalone job ever swept them (the
autopilot cycle's extract phase only walks that cycle's changedSlugs).

Fix, both halves of the issue:
- webhook + sync-trigger payloads pass noExtract:false so normal
  incremental pushes get sync's inline link/timeline extraction.
- >100-file deferral branch: performSync surfaces extractDeferred, and
  the sync job handler queues a source-scoped extract{stale:true}
  follow-up job (idempotent sweep; extract handler gains a stale mode
  backed by the same extractStaleFromDB core as `gbrain extract --stale`).

Tests: new sync-large-deferral-extract.serial.test.ts drives a real
101-file incremental sync end-to-end (deferral flag → queued follow-up →
stale sweep recovers links + watermark); webhook payload contract test +
sync-trigger noExtract assertion salvaged from #2850.

Fixes #2849
Takeover of #2850

Co-authored-by: patentsong <patentsong@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-07-21 14:38:43 -07:00
co-authored by patentsong Claude Fable 5
parent 0612b0daa8
commit 08f2397615
7 changed files with 224 additions and 5 deletions
+4
View File
@@ -100,6 +100,10 @@ describe('runSyncTrigger', () => {
const job = jobs[0];
expect(job.priority).toBe(-10);
expect((job.data as { sourceId: string }).sourceId).toBe('default');
// #2849: opt in to inline extraction — the standalone sync handler
// defaults noExtract to TRUE, which would leave triggered syncs
// extraction-stale.
expect((job.data as { noExtract: boolean }).noExtract).toBe(false);
expect((job.data as { auto_embed_backfill: boolean }).auto_embed_backfill).toBe(true);
});