fix(recipes/openai): add max_batch_tokens to embedding touchpoint

OpenAI is the only recipe in the codebase without a max_batch_tokens cap.
Every other provider declares one (voyage=120K, azure-openai=8K, dashscope=8K,
zhipu=8K, minimax=4K). Without it, gbrain's recursive-halving safety net never
engages — batches dispatched purely on the char/4 estimator window will trip
OpenAI's 1M-token TPM ceiling on token-dense pages (Discord exports, JSON
dumps, code-heavy markdown), then retry storm and block the queue head.

Setting cap to 100_000:
- gbrain's batcher estimates tokens as chars/4
- Token-dense markdown+JSON tokenizes at ~chars/2.7
- 100K estimated = ~150K real worst-case, safely under OpenAI's 300K
  per-request hard cap and the 1M/min TPM ceiling
- Leaves headroom for recursive-halving on outlier chunks
This commit is contained in:
Matt Gunnin
2026-05-12 13:41:41 -05:00
parent 17b190e227
commit 40536aace5
+6
View File
@@ -17,6 +17,12 @@ export const openai: Recipe = {
dims_options: [256, 512, 768, 1024, 1536, 3072],
cost_per_1m_tokens_usd: 0.13,
price_last_verified: '2026-04-20',
// OpenAI per-request hard cap is 300K tokens. Free/Tier-1 TPM is 1M.
// Cap batches conservatively at 100K to handle token-dense content
// (Discord/Slack markdown+JSON tokenizes at ~chars/2.7, not the chars/4
// estimate the batcher uses). 100K estimated = ~150K real tokens worst-case,
// safely under both the 300K per-request and 1M TPM ceilings.
max_batch_tokens: 100_000,
},
expansion: {
models: ['gpt-5.2', 'gpt-4o-mini'],