mirror of
https://github.com/tinyhumansai/openhuman.git
synced 2026-07-27 21:08:00 +00:00
fix(chat): drop "local provider" misdirect from empty-response copy + log empty 2xx streams (#3335) (#3415)
## Summary - Replace the misleading **"Try a different model or check your local provider in Settings → AI → LLM"** copy on the empty-response classifier — that sent **Managed-route users toward a remedy that doesn't exist for them**, because there is no local provider on the Managed route. The new copy explicitly names the credits / billing remedy alongside the model / provider-config remedies (chat surface), and a tighter ≤120-char form for the cron notification drawer. - Add a forensic `warn`-level log in `stream_native_chat` for the exact upstream shape that triggers `AgentError::EmptyProviderResponse` — a streaming chat call that returns **HTTP 2xx with zero text, zero thinking, and zero tool calls**. Captures `elapsed_ms`, `sse_chunks_parsed`, `raw_bytes_received`, `has_usage`, `has_openhuman_meta` so production data can disambiguate the three plausible upstream root causes (credit exhaustion served as 200 + empty body, upstream provider stall, real degenerate model output). - Strictly additive — no behavioural change to request paths, no new Sentry suppression. The diagnostic log fires only on the empty-2xx case (otherwise the existing aggregated log carries the same fields at `info`). ## Problem Issue #3335 (`iamrhn`): on Managed settings, every chat turn surfaces: > *"The model returned an empty response. Try a different model or check your local provider in Settings → AI → LLM."* The "local provider" remedy is nonsensical for Managed users. The Sentry data confirms this is real and broad: - **TAURI-RUST-4JX**: 2,633 events, 55 distinct clients in 14 days. Releases include current `0.57.13` (PR #2790's Sentry suppression silenced the events but the underlying user-visible bug remains). - Latest event breadcrumbs (Managed route, `reasoning-v1`, web channel): ``` [provider:OpenHuman] outbound chat/completions -> https://api.tinyhumans.ai/openai/v1/chat/completions [stream] OpenHuman POST … (stream=true, tools=21) …~7 s gap, no [stream] error breadcrumb, no [llm_provider] api_error breadcrumb… [agent] provider responded — parsed tool_calls=0 text_chars=0 [agent_loop] provider returned an empty final response (i=1, no text, no tool calls) — surfacing as error ``` HTTP status was 2xx (no `streaming API error` line), so none of the existing classifiers (`is_budget_exhausted_http_400`, `is_provider_access_policy_denied_http_403`, …) at `compatible.rs:1028-1060` fire. The streaming aggregator builds a `ChatResponse` with `content: None`, the agent loop returns `AgentError::EmptyProviderResponse`, and the chat surface renders the wrong-remediation copy. This is exactly the shape #3386 (team-filed) identifies as path **(b)**: the OpenHuman managed backend returns 200 + empty SSE under credit exhaustion / upstream stall instead of a typed 400 + `Insufficient budget`. The deeper fix lives in `tinyhumansai/backend`; this PR addresses the **two client-side wins that don't require the server change**. ## Solution **(1) Copy fix — chat surface (`web_errors.rs::classify_inference_error` empty_response arm).** Replace the `"Try a different model or check your local provider"` text with a three-remedy form that's accurate for **all** providers: *"This usually means your inference credits are exhausted (Settings → Billing), the upstream model is temporarily unhealthy, or your provider configuration is rejecting the request (Settings → AI → LLM). Try one of those, or pick a different model."* New regression test `classify_inference_error_empty_response_copy_names_billing_remedy_and_drops_local_provider_misdirect` locks in: contains `Settings → Billing`, contains `Settings → AI → LLM`, contains `different model`, does NOT contain `local provider`, and `provider` stays `None`. Existing test `classify_inference_error_empty_response_is_actionable_and_retryable` continues to pass. **(2) Copy fix — cron drawer (`cron/scheduler.rs::agent_error_to_user_message`).** Shorter form for the drawer's ≤120-char convention: *"Empty model response. Out of credits (Settings → Billing) or try a different model in Settings → AI → LLM."* New regression test `agent_error_to_user_message_classifies_empty_provider_response_for_3335` mirrors the chat-side assertions; `EmptyProviderResponse` is added to the existing `agent_error_to_user_message_canned_strings_are_short` variants list to lock in the ≤120-char contract (was previously absent — fit incidentally, but nothing enforced it). **(3) Diagnostic log — `compatible.rs::stream_native_chat`.** Right after the existing aggregated `info` log, emit a `warn` when `text_accum.is_empty() && thinking_accum.is_empty() && tool_call_count == 0`: ``` [stream] {name} empty 2xx stream — model={..} elapsed_ms={..} sse_chunks={..} raw_bytes={..} has_usage={..} has_openhuman_meta={..} ``` `stream_started_at: Instant`, `sse_chunks_parsed: usize`, and `raw_bytes_received: usize` are added as append-only counters (never read by the request path). The signal disambiguates: - `elapsed_ms` small + `sse_chunks=0` + `raw_bytes` small → backend closed the SSE near-immediately (credit reject or auth) - `elapsed_ms` large + `sse_chunks=0` + `raw_bytes=0` → upstream stall / timeout - `sse_chunks>0` + `has_usage=true` + accumulators empty → backend streamed metadata-only chunks (tokens counted, no content delivered) **(4) Deliberately deferred — the in-session budget correlator from #3386.** `EmptyProviderResponse`'s flattened Display carries no `" API error"` infix for `extract_provider_name` to anchor on, so the classifier here cannot tell which provider was used without plumbing the typed `provider: Option<String>` through the AgentError. That plumbing affects ~10 string-match test/observability sites and balloons the scope. The diagnostic log is the cheapest path to the production data that decides whether the plumbing is worth doing — once we have evidence on which path is dominant (200+empty under credit exhaustion vs. upstream stall vs. true degenerate output), the next iteration of the empty-response arm can be provider-aware. ## Submission Checklist > If a section does not apply to this change, mark the item as `N/A` with a one-line reason. Do not delete items. - [x] Tests added or updated (happy path + at least one failure / edge case) per [Testing Strategy](../gitbooks/developing/testing-strategy.md#failure-path-requirement) — three new tests: `classify_inference_error_empty_response_copy_names_billing_remedy_and_drops_local_provider_misdirect` (web chat copy), `agent_error_to_user_message_classifies_empty_provider_response_for_3335` (cron drawer copy), plus `EmptyProviderResponse` added to `agent_error_to_user_message_canned_strings_are_short` variants (locks in the ≤120-char contract that was previously unenforced). - [x] **Diff coverage ≥ 80%** — every new/changed line in `web_errors.rs` and `cron/scheduler.rs` is exercised by the three new regression tests; the `compatible.rs` diagnostic log is observability-only (no behavioural branching) and covered by the existing streaming-chat test surface that drives it. Run `pnpm test:rust` locally to verify. - [x] Coverage matrix updated — `N/A: behaviour-only change` (error-message copy + observability log, no new feature ID). - [x] All affected feature IDs from the matrix are listed in the PR description under `## Related` — `N/A: behaviour-only change, no matrix rows touched`. - [x] No new external network dependencies introduced (mock backend used per [Testing Strategy](../gitbooks/developing/testing-strategy.md#mock-policy)) — `N/A: tests are pure unit-level over the classifier and the AgentError variant; no provider HTTP exercised`. - [x] Manual smoke checklist updated if this touches release-cut surfaces ([`docs/RELEASE-MANUAL-SMOKE.md`](../docs/RELEASE-MANUAL-SMOKE.md)) — `N/A: error-copy + log only, no UI flow or release-cut surface affected`. - [x] Linked issue closed via `Closes #NNN` in the `## Related` section. ## Impact - **Desktop (chat surface)**: Managed-route users hitting empty-response failures now see an actionable, non-misleading message that names `Settings → Billing` as the most likely fix (per Sentry / #3386 evidence) without losing the existing `Settings → AI → LLM` and model-switch remedies. Self-hosted users see the same three-remedy framing — strictly more accurate than the prior "local provider" claim for users on cloud providers. - **Desktop (cron notifications)**: Same behavioural change, shorter copy to fit the ≤120-char drawer-render budget. Now enforced by the existing length test. - **Observability**: Net +1 `warn` log per streaming chat turn that returns 2xx-with-empty-body. This is the failure case being investigated; non-empty streams are unaffected. Lands in Sentry breadcrumbs even after `AgentError::skips_sentry()` (#2790) silences the parent event, unblocking the data needed to decide whether the in-session budget correlator from #3386 is the right next move or whether the fix has to be server-side. - **Performance / security**: No request-path behaviour change. The added counters are stack-local `usize` increments. The diagnostic log redacts nothing new — provider name, model name, and counts are all already in the existing aggregated `info` log; only the byte total and elapsed ms are net-new and both are content-free. - **Migration / compatibility**: None. Pure string + observability change. ## Related - Closes: #3335 - Related: #3386 (the team-filed deep-dive that this PR addresses without requiring the per-session budget correlator), #3104 (cascade-error reports likely sharing the same empty-stream root cause), #2790 (the Sentry suppression that silenced TAURI-RUST-4JX without fixing the user-visible bug), #3199 (the empty-response arm being modified), #3121 (the budget-exhausted copy whose remedy this copy now points users toward), #3092 (closed parent issue for the chat-error cluster). - Follow-up PR(s)/TODOs: Once the diagnostic log produces 24–48 h of production data, decide between (a) opening a backend issue against `tinyhumansai/backend` to start returning `400 + Insufficient budget` on streaming under credit exhaustion (the right server-side fix), and (b) plumbing `provider: Option<String>` through `AgentError::EmptyProviderResponse` so the classifier can be provider-aware (the #3386 in-session correlator approach). --- ## AI Authored PR Metadata (required for Codex/Linear PRs) > Keep this section for AI-authored PRs. For human-only PRs, mark each field `N/A`. ### Linear Issue - Key: N/A — GitHub issue #3335, not Linear-tracked. - URL: N/A ### Commit & Branch - Branch: `fix/3335-managed-empty-response-diagnostic-and-copy` - Commit SHA: `7064cbfee` ### Validation Run - [x] `pnpm --filter openhuman-app format:check` — `N/A: no frontend / TS files changed` - [x] `pnpm typecheck` — `N/A: no frontend / TS files changed` - [x] Focused tests: `cargo test --lib empty_response` (10 pass), `cargo test --lib empty_provider` (5 pass), `cargo test --lib cron::scheduler::tests::agent_error` (11 pass), `cargo test --lib compatible::tests` (153 pass). - [x] Rust fmt/check (if changed): `cargo fmt --manifest-path Cargo.toml` + `cargo check --manifest-path Cargo.toml --lib` — both clean (only pre-existing unrelated warnings). - [x] Tauri fmt/check (if changed): `N/A: no app/src-tauri files changed` ### Validation Blocked - `command:` N/A - `error:` N/A - `impact:` N/A ### Behavior Changes - Intended behavior change: User-facing error copy for `AgentError::EmptyProviderResponse` no longer claims a "local provider" exists; new copy names `Settings → Billing` (credits) alongside `Settings → AI → LLM` (provider config) and the model-switch fallback. - User-visible effect: Managed-route users who hit empty-response failures now receive remedy text that matches the most likely actual cause (credit exhaustion, per #3386 evidence). Self-hosted users see the same three-remedy framing, which remains accurate for their configurations. ### Parity Contract - Legacy behavior preserved: `AgentError::EmptyProviderResponse` Display, `classify_inference_error` error_type / source / retryable / provider fields, `agent_error_to_user_message` dispatch matrix, the `Settings → AI → LLM` deep-link reference. Existing test `classify_inference_error_empty_response_is_actionable_and_retryable` continues to pass unchanged. - Guard/fallback/dispatch parity checks: The empty-response arm in `web_errors.rs` keeps the same `error_type: "empty_response"`, `source: "agent_loop"`, `retryable: true`, `retry_after_ms: None`, `provider: None`, `fallback_available: None` — only the message string changes. The cron `agent_error_to_user_message` dispatch matrix is unchanged; only the `EmptyProviderResponse` branch's returned string changes. ### Duplicate / Superseded PR Handling - Duplicate PR(s): None — no other open PR references #3335. - Canonical PR: This one. - Resolution: N/A (no prior PR to supersede). <!-- This is an auto-generated comment: release notes by coderabbit.ai --> ## Summary by CodeRabbit ## Release Notes * **Bug Fixes** * Improved error messaging when model returns an empty response, now guiding users to check inference credits (Settings → Billing), try a different model, or verify LLM provider configuration (Settings → AI → LLM). * **Improvements** * Added enhanced diagnostic logging for empty response scenarios to better support troubleshooting. <!-- end of auto-generated comment: release notes by coderabbit.ai --> Closes #3335 Co-authored-by: M3gA-Mind <megamind@mahadao.com>
This commit is contained in:
@@ -579,10 +579,30 @@ pub(crate) fn classify_inference_error(err: &str) -> ClassifiedError {
|
||||
// neither can be shadowed by the broad provider-429 / 5xx arms below.
|
||||
// No `with_provider_detail` — an empty response carries no JSON body
|
||||
// to quote.
|
||||
//
|
||||
// Issue #3335: the prior copy ("Try a different model or check your
|
||||
// local provider in Settings → AI → LLM") sent Managed users in
|
||||
// exactly the wrong direction — there is no local provider on the
|
||||
// Managed route, and the common underlying cause is credit
|
||||
// exhaustion (see #3386). The provider name is not available here
|
||||
// because `EmptyProviderResponse`'s flattened Display carries no
|
||||
// `" API error"` infix for `extract_provider_name` to anchor on,
|
||||
// and routing the typed provider through every layer is out of
|
||||
// scope for this fix. So the copy is rewritten to be accurate for
|
||||
// ALL providers: it names the three real remedies (credits, model
|
||||
// health, configuration) without claiming any single one of them
|
||||
// applies, and lets the user pick which is relevant to their
|
||||
// setup. The companion empty-2xx-stream diagnostic in
|
||||
// `compatible.rs::stream_native_chat` records elapsed_ms,
|
||||
// chunk_count, and has_usage so the next iteration of this arm
|
||||
// can be provider-aware once we have evidence on which path is
|
||||
// dominant in production.
|
||||
ClassifiedError {
|
||||
error_type: "empty_response",
|
||||
message: "The model returned an empty response. Try a different model or check \
|
||||
your local provider in Settings → AI → LLM."
|
||||
message: "The model returned an empty response. This usually means your inference \
|
||||
credits are exhausted (Settings → Billing), the upstream model is temporarily \
|
||||
unhealthy, or your provider configuration is rejecting the request \
|
||||
(Settings → AI → LLM). Try one of those, or pick a different model."
|
||||
.to_string(),
|
||||
source: "agent_loop",
|
||||
retryable: true,
|
||||
|
||||
@@ -971,6 +971,60 @@ fn classify_inference_error_empty_response_is_actionable_and_retryable() {
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn classify_inference_error_empty_response_copy_names_billing_remedy_and_drops_local_provider_misdirect(
|
||||
) {
|
||||
// Issue #3335: the prior copy ("Try a different model or check your
|
||||
// local provider in Settings → AI → LLM") sent Managed-route users
|
||||
// toward a remedy that does not exist for them. The common underlying
|
||||
// cause is credit exhaustion (issue #3386), so the revised copy must
|
||||
// name the credits / billing path explicitly, must NOT claim a "local
|
||||
// provider" exists, and must still offer the model-switch path for
|
||||
// users on self-hosted providers.
|
||||
let raw = "run_chat_task failed client_id=abc thread_id=t-1 request_id=r-1 \
|
||||
error=The model returned an empty response. Please try again.";
|
||||
let classified = classify_inference_error(raw);
|
||||
assert_eq!(classified.error_type, "empty_response");
|
||||
// New: names the credits / billing remedy (was absent in the old copy,
|
||||
// so Managed users had no way to self-diagnose credit exhaustion).
|
||||
assert!(
|
||||
classified.message.contains("Settings → Billing"),
|
||||
"must point at the billing surface for credit exhaustion: {}",
|
||||
classified.message
|
||||
);
|
||||
// New: drops the misleading "local provider" framing — the previous
|
||||
// copy made a false claim for Managed users where no local provider
|
||||
// exists.
|
||||
assert!(
|
||||
!classified.message.contains("local provider"),
|
||||
"must not claim a local provider exists: {}",
|
||||
classified.message
|
||||
);
|
||||
// Preserved: the model-switch remedy and the provider-config
|
||||
// settings deep link both still apply (some users hit empty response
|
||||
// because their custom OpenAI-compatible endpoint or local model is
|
||||
// misconfigured / unhealthy).
|
||||
assert!(
|
||||
classified.message.contains("different model"),
|
||||
"must keep the model-switch remedy: {}",
|
||||
classified.message
|
||||
);
|
||||
assert!(
|
||||
classified.message.contains("Settings → AI → LLM"),
|
||||
"must keep the provider-config deep link: {}",
|
||||
classified.message
|
||||
);
|
||||
// Preserved: provider is intentionally None until the typed
|
||||
// `AgentError::EmptyProviderResponse` plumbs through a provider
|
||||
// identifier (see comment in `web_errors.rs::classify_inference_error`
|
||||
// empty_response arm).
|
||||
assert!(
|
||||
classified.provider.is_none(),
|
||||
"provider stays None until plumbed through the typed error: {:?}",
|
||||
classified.provider
|
||||
);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn classify_inference_error_vision_capability_is_non_retryable() {
|
||||
// A multimodal turn sent an image to a text-only model. Retrying the
|
||||
|
||||
@@ -57,7 +57,16 @@ fn agent_error_to_user_message(err: &AgentError) -> &'static str {
|
||||
"The agent stopped after too many tool iterations. Raise the iteration cap in Settings \u{2192} AI \u{2192} LLM or simplify the task."
|
||||
}
|
||||
AgentError::EmptyProviderResponse { .. } => {
|
||||
"The model returned an empty response. Try a different model or check your local provider in Settings \u{2192} AI \u{2192} LLM."
|
||||
// Issue #3335: the prior copy named a "local provider"
|
||||
// remedy that doesn't exist on the Managed route. This
|
||||
// shorter form (≤120 chars per the
|
||||
// `agent_error_to_user_message_canned_strings_are_short`
|
||||
// contract, for clean notification-drawer rendering) names
|
||||
// the two highest-signal remedies — credits and model
|
||||
// configuration. The richer three-remedy copy lives on the
|
||||
// chat-surface side (`channels/providers/web_errors.rs`'s
|
||||
// empty_response arm) where there's no drawer-width limit.
|
||||
"Empty model response. Out of credits (Settings \u{2192} Billing) or try a different model in Settings \u{2192} AI \u{2192} LLM."
|
||||
}
|
||||
AgentError::CompactionFailed { .. } => {
|
||||
"Automatic history compaction failed. The next run will start with a fresh context."
|
||||
|
||||
@@ -846,6 +846,33 @@ fn agent_error_to_user_message_classifies_max_iterations() {
|
||||
assert_ne!(msg, AGENT_JOB_USER_FAILURE_MESSAGE);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn agent_error_to_user_message_classifies_empty_provider_response_for_3335() {
|
||||
// Issue #3335: the cron-path copy must stay in lock-step with the
|
||||
// web-channel `empty_response` arm — names the credits / billing
|
||||
// remedy explicitly and drops the misleading "local provider"
|
||||
// misdirect that broke remediation for Managed users.
|
||||
let err = AgentError::EmptyProviderResponse { iteration: 1 };
|
||||
let msg = agent_error_to_user_message(&err);
|
||||
assert!(
|
||||
msg.contains("Settings \u{2192} Billing"),
|
||||
"must point at billing for credit exhaustion: {msg}"
|
||||
);
|
||||
assert!(
|
||||
!msg.contains("local provider"),
|
||||
"must not claim a local provider exists: {msg}"
|
||||
);
|
||||
assert!(
|
||||
msg.contains("different model"),
|
||||
"must keep the model-switch remedy: {msg}"
|
||||
);
|
||||
assert!(
|
||||
msg.contains("Settings \u{2192} AI \u{2192} LLM"),
|
||||
"must keep the provider-config deep link: {msg}"
|
||||
);
|
||||
assert_ne!(msg, AGENT_JOB_USER_FAILURE_MESSAGE);
|
||||
}
|
||||
|
||||
#[test]
|
||||
fn agent_error_to_user_message_classifies_compaction_failed() {
|
||||
let err = AgentError::CompactionFailed {
|
||||
@@ -921,6 +948,13 @@ fn agent_error_to_user_message_canned_strings_are_short() {
|
||||
required_level: "x".into(),
|
||||
channel_max_level: "x".into(),
|
||||
},
|
||||
// Issue #3335: EmptyProviderResponse was historically absent from
|
||||
// this variants list — its old copy happened to fit, but nothing
|
||||
// enforced it. The fix shipped a new copy that explicitly names
|
||||
// the credits / billing remedy, which makes the length tradeoff
|
||||
// active rather than incidental. Lock it in so a future copy
|
||||
// change can't quietly grow past the drawer-render budget.
|
||||
AgentError::EmptyProviderResponse { iteration: 0 },
|
||||
];
|
||||
for v in &variants {
|
||||
let msg = agent_error_to_user_message(v);
|
||||
|
||||
@@ -32,6 +32,11 @@ impl OpenAiCompatibleProvider {
|
||||
native_request.tools.as_ref().map_or(0, |t| t.len()),
|
||||
);
|
||||
|
||||
// Captured at request send so the empty-2xx-stream diagnostic
|
||||
// below can report elapsed_ms — a fast empty stream points at a
|
||||
// backend reject, a slow one at an upstream stall / timeout.
|
||||
let stream_started_at = std::time::Instant::now();
|
||||
|
||||
let response = self
|
||||
.apply_auth_header(
|
||||
self.http_client()
|
||||
@@ -157,9 +162,45 @@ impl OpenAiCompatibleProvider {
|
||||
self.name,
|
||||
);
|
||||
let response_bytes = response.bytes().await?;
|
||||
let body_bytes_received = response_bytes.len();
|
||||
dump_response_if_enabled(&self.name, &native_request.model, dump_seq, &response_bytes);
|
||||
let api_resp: ApiChatResponse = serde_json::from_slice(&response_bytes)
|
||||
.map_err(|err| anyhow::anyhow!("{} response parse error: {err}", self.name))?;
|
||||
|
||||
// Mirror the SSE-branch empty-2xx-stream diagnostic (#3335 /
|
||||
// #3386) on the buffered JSON path. The same upstream
|
||||
// collapse to `AgentError::EmptyProviderResponse` is
|
||||
// reachable here when a managed backend returns 200 with a
|
||||
// content-less JSON payload (credit exhaustion served as
|
||||
// JSON instead of SSE, or an upstream stall flushed as an
|
||||
// empty completion). Without this sibling guard the warn
|
||||
// would only fire on the SSE branch and the buffered case
|
||||
// would silently miss the very signal we're trying to
|
||||
// capture.
|
||||
let buffered_is_empty = api_resp
|
||||
.choices
|
||||
.first()
|
||||
.map(|c| {
|
||||
let m = &c.message;
|
||||
let content_empty = m.content.as_deref().is_none_or(str::is_empty);
|
||||
let reasoning_empty = m.reasoning_content.as_deref().is_none_or(str::is_empty);
|
||||
let tool_calls_empty = m.tool_calls.as_ref().is_none_or(|t| t.is_empty());
|
||||
let function_call_empty = m.function_call.is_none();
|
||||
content_empty && reasoning_empty && tool_calls_empty && function_call_empty
|
||||
})
|
||||
.unwrap_or(true);
|
||||
if buffered_is_empty {
|
||||
let elapsed_ms = stream_started_at.elapsed().as_millis() as u64;
|
||||
log::warn!(
|
||||
"[stream] {} empty 2xx buffered JSON — model={} elapsed_ms={} body_bytes={} has_usage={} has_openhuman_meta={}",
|
||||
self.name,
|
||||
native_request.model,
|
||||
elapsed_ms,
|
||||
body_bytes_received,
|
||||
api_resp.usage.is_some(),
|
||||
api_resp.openhuman.is_some(),
|
||||
);
|
||||
}
|
||||
return Self::parse_native_response(api_resp, &self.name);
|
||||
}
|
||||
|
||||
@@ -174,9 +215,23 @@ impl OpenAiCompatibleProvider {
|
||||
let mut buffer = String::new();
|
||||
let mut repeat_detector = StreamRepeatDetector::new();
|
||||
let mut degenerate_repeat = false;
|
||||
// Forensic counters for the empty-2xx-stream diagnostic below
|
||||
// (issue #3335 / #3386). Both are append-only, never read by
|
||||
// request path logic — strictly observability.
|
||||
//
|
||||
// `body_bytes_received` is the count of body bytes yielded by
|
||||
// `bytes_stream()`. This crate builds reqwest without the
|
||||
// `gzip` / `brotli` / `zstd` / `deflate` features (see
|
||||
// root Cargo.toml), so no Content-Encoding decompression happens
|
||||
// and the count matches what's on the wire. The neutral name
|
||||
// (rather than `raw_bytes` / `decoded_bytes`) sidesteps the
|
||||
// ambiguity an operator would otherwise hit reading the log.
|
||||
let mut sse_chunks_parsed: usize = 0;
|
||||
let mut body_bytes_received: usize = 0;
|
||||
|
||||
'stream: while let Some(item) = bytes_stream.next().await {
|
||||
let bytes = item?;
|
||||
body_bytes_received += bytes.len();
|
||||
buffer.push_str(&String::from_utf8_lossy(&bytes));
|
||||
|
||||
while let Some(sep_idx) = buffer.find("\n\n") {
|
||||
@@ -214,7 +269,10 @@ impl OpenAiCompatibleProvider {
|
||||
}
|
||||
|
||||
let chunk: StreamChunkResponse = match serde_json::from_str(data) {
|
||||
Ok(v) => v,
|
||||
Ok(v) => {
|
||||
sse_chunks_parsed += 1;
|
||||
v
|
||||
}
|
||||
Err(e) => {
|
||||
log::debug!(
|
||||
"[stream] {} skipping unparseable chunk: {} — data={}",
|
||||
@@ -413,6 +471,42 @@ impl OpenAiCompatibleProvider {
|
||||
tool_call_count,
|
||||
);
|
||||
|
||||
// Issue #3335 / #3386 forensic signal. The streaming chat call
|
||||
// completed with HTTP 2xx but delivered zero visible text, zero
|
||||
// thinking, and zero tool calls. This is the upstream shape that
|
||||
// collapses to `AgentError::EmptyProviderResponse` and gets
|
||||
// rendered to the user as "The model returned an empty
|
||||
// response" with the wrong remediation. Most likely causes:
|
||||
// (a) backend closed the SSE cleanly under credit exhaustion
|
||||
// (no `[stream] streaming API error` breadcrumb fires
|
||||
// because status was 200) — common on the OpenHuman
|
||||
// managed route under #3386,
|
||||
// (b) backend's upstream LLM provider stalled / timed out
|
||||
// and the backend forwarded an empty stream instead of
|
||||
// propagating the upstream error,
|
||||
// (c) a genuine degenerate model output (rare on hosted
|
||||
// reasoning models; more common on community quants).
|
||||
// Logged at warn so it lands in Sentry breadcrumbs even after
|
||||
// `AgentError::skips_sentry()` (PR #2790) silences the parent
|
||||
// event. Correlate by elapsed_ms (fast == reject, slow ==
|
||||
// stall), sse_chunks (0 == no SSE at all, >0 == backend
|
||||
// streamed metadata-only chunks), has_usage (the upstream
|
||||
// counted tokens but delivered no content), and
|
||||
// has_openhuman_meta (managed backend reported routing info).
|
||||
if text_accum.is_empty() && thinking_accum.is_empty() && tool_call_count == 0 {
|
||||
let elapsed_ms = stream_started_at.elapsed().as_millis() as u64;
|
||||
log::warn!(
|
||||
"[stream] {} empty 2xx stream — model={} elapsed_ms={} sse_chunks={} body_bytes={} has_usage={} has_openhuman_meta={}",
|
||||
self.name,
|
||||
native_request.model,
|
||||
elapsed_ms,
|
||||
sse_chunks_parsed,
|
||||
body_bytes_received,
|
||||
last_usage.is_some(),
|
||||
last_openhuman.is_some(),
|
||||
);
|
||||
}
|
||||
|
||||
let tool_calls_for_api: Vec<ToolCall> = tool_accum
|
||||
.into_values()
|
||||
.map(|c| ToolCall {
|
||||
|
||||
Reference in New Issue
Block a user