mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-31 04:07:52 +00:00
fix(webhook): extract links for incremental push syncs (#2849)
GitHub webhook + sync-trigger 'sync' jobs never opted into extraction:
the standalone sync handler defaults noExtract to TRUE, so webhook-
imported pages advanced the source bookmark with links_extracted_at
unstamped — and nothing behind a standalone job ever swept them (the
autopilot cycle's extract phase only walks that cycle's changedSlugs).
Fix, both halves of the issue:
- webhook + sync-trigger payloads pass noExtract:false so normal
incremental pushes get sync's inline link/timeline extraction.
- >100-file deferral branch: performSync surfaces extractDeferred, and
the sync job handler queues a source-scoped extract{stale:true}
follow-up job (idempotent sweep; extract handler gains a stale mode
backed by the same extractStaleFromDB core as `gbrain extract --stale`).
Tests: new sync-large-deferral-extract.serial.test.ts drives a real
101-file incremental sync end-to-end (deferral flag → queued follow-up →
stale sweep recovers links + watermark); webhook payload contract test +
sync-trigger noExtract assertion salvaged from #2850.
Fixes #2849
Takeover of #2850
Co-authored-by: patentsong <patentsong@users.noreply.github.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
This commit is contained in:
co-authored by
patentsong
Claude Fable 5
parent
0612b0daa8
commit
08f2397615
@@ -100,6 +100,10 @@ describe('runSyncTrigger', () => {
|
||||
const job = jobs[0];
|
||||
expect(job.priority).toBe(-10);
|
||||
expect((job.data as { sourceId: string }).sourceId).toBe('default');
|
||||
// #2849: opt in to inline extraction — the standalone sync handler
|
||||
// defaults noExtract to TRUE, which would leave triggered syncs
|
||||
// extraction-stale.
|
||||
expect((job.data as { noExtract: boolean }).noExtract).toBe(false);
|
||||
expect((job.data as { auto_embed_backfill: boolean }).auto_embed_backfill).toBe(true);
|
||||
});
|
||||
|
||||
|
||||
Reference in New Issue
Block a user