mirror of
https://github.com/open-jarvis/OpenJarvis.git
synced 2026-07-28 05:12:26 +00:00
0b140110d8
The OpenAI-compat and Ollama engines exposed stream()/stream_full() as async def but iterated a synchronous httpx.Client.iter_lines() internally, blocking the single event loop on every inter-token read (serializing concurrent chats; one wedged upstream read froze the whole API). Convert both to a shared AsyncHTTPEngineMixin using httpx.AsyncClient + aiter_lines() with a per-event-loop pooled client and the configured timeout applied; map mid-stream transport errors (RemoteProtocolError/ReadError) to EngineConnectionError via a deliberately narrow set that keeps CancelledError/GeneratorExit propagating; handle non-2xx explicitly (incl. 3xx and a typed EngineContextLengthError for context-window overflow 400s); switch litellm streaming to acompletion; and offload the blocking non-streaming handlers and websocket generate() to asyncio.to_thread. No public API change. Strong MockTransport-based tests, including a pin that the async path never touches the sync client. Complements #618. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>