mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-30 11:22:34 +00:00
Matt GunninandGarry Tan
e17ea13745
OpenAI's /v1/embeddings endpoint hard-caps a single request at 300k tokens
total across all input items. When the cap is exceeded it returns:
Invalid 'input': maximum request size is 300000 tokens per request.
None of the three existing regexes in isTokenLimitError matched this
phrasing, so the recursive-halving safety net in embedSubBatch never
engaged for OpenAI. The same fat page (a token-dense markdown export,
e.g. a Discord transcript) would re-fail every pass, blocking forward
progress on the whole batch indefinitely.
Locally reproduced on a 31,129-chunk Postgres brain: 2,125 chunks
stuck at 'remaining' across 30+ embed --stale passes with retry
loops + sleep delays. Adding the two new patterns lets halving fire;
the same backlog cleared in one pass after the regex change (the
companion max_batch_tokens recipe fix from PR #924 caps fresh batches,
but existing oversize pages still need halving to recover).
Adds:
- /maximum request size.*tokens/i — OpenAI verbatim
- /max.*tokens.*per.*request/i — defensive against minor rewording
Tests:
- Regression test for the exact OpenAI error string
- Coverage for the generic 'max tokens per request' variant
- All 25 tests in adaptive-embed-batch.test.ts pass
No behavior change for providers whose errors already matched.
(cherry picked from commit b834e84c56)