mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-28 14:59:47 +00:00
292b8b1637
llama.cpp's llama-server rejects /v1/embeddings requests with more inputs than its launch --batch-size (default 32): "batch size 100 > maximum allowed batch size 32". gbrain sends batches of 100, so any page with >32 chunks fails to embed, and embed --stale then trips the Postgres statement_timeout retrying the doomed batches. The existing token-based protection (max_batch_tokens) can't bound item count — N tiny chunks fit under any token budget. Add an optional max_batch_items count cap to EmbeddingTouchpoint, enforced as a hard re-split after the token split in embed(), and set it to 32 on the llama-server recipe (replacing no_batch_cap: true, which wrongly assumed llama.cpp has no per-request item cap). A declared item cap also suppresses the missing-max_batch_tokens startup warning. Co-authored-by: Time Attakc <89218912+time-attack@users.noreply.github.com>