mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-28 14:59:47 +00:00
* fix(ai): drop empty-string env values before merge so they can't clobber config keys (#1249) Claude Code injects ANTHROPIC_API_KEY='' to neuter subprocess LLM calls; an unconditional process.env spread let that empty string override a valid config.json key, breaking every gateway op with NO_ANTHROPIC_API_KEY. Filter '' / undefined before the merge; '0' and 'false' are preserved. * fix(ai): normalize native provider base URLs + replace embedding guard with a dims-presence check (#1250, #1292) #1250: createAnthropic/createOpenAI were called with no baseURL, so an env-injected bare host (e.g. ANTHROPIC_BASE_URL without /v1) 404'd. Add a shared resolveNativeBaseUrl and pass a normalized baseURL at all anthropic + openai native sites (google deferred until its suffix is verified). #1292/D6: the user_provided_model_unset guard was structurally unreachable as a no-model check (parseModelId throws on a bare provider) and only ever false-positived for litellm:<model>, silently disabling vector search. Replace it with a real dims-presence check for user-provided/zero-default recipes and delete the dead branch in both consumers. Also stop configureGateway from fabricating a default embedding_dimensions, so 'no dims set' stays honest. * fix(ai): trust user-declared embedding dims for local recipes + litellm /v1 hint (#2271, #2209) #2271: a new trust_custom_dims flag adds a passthrough tier so ollama / llama-server / litellm accept a user-supplied --embedding-dimensions instead of being hard-rejected. Fail-closed for fixed-dim providers (openai/voyage/ zeroentropy) and excludes openrouter (declares dims_options). Register modern ollama embed model names. #2209: litellm setup_hint now states the /v1 path convention and the docs pointer is corrected to docs/integrations/embedding-providers.md. * docs+test(ai): KEY_FILES current-state for provider-agnostic gateway + embed-preflight dims-unset test (#1249, #1250, #1292) * fix(ai): point user_provided_dims_unset remediation at 'gbrain init' (config set rejects it) + coverage Pre-landing adversarial review (P1): the new dims-unset guard told users to run 'gbrain config set embedding_dimensions <N>', which config.ts hard-rejects (it's a schema-sizing field). Both consumer messages now point at 'gbrain init --embedding-dimensions'. Adds: pgvector-cap-still-fires regression for the trust_custom_dims passthrough, and a configureGateway backfill-invariant test. * chore: bump version and changelog (v0.42.57.0) Provider-agnostic plumbing wave: #1249 empty-env clobber, #1250 native baseURL normalization, #1292 embedding dims-presence guard, #2271 trust_custom_dims passthrough, #2209 litellm /v1 hint. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: sync embedding-providers guide for provider-agnostic gateway wave (v0.42.57.0) Post-ship doc drift fix for the v0.42.57.0 AI-gateway wave: - LiteLLM section now names the /v1 base-URL convention (#2209). - Ollama section lists the newly-registered modern embedders qwen3-embed-8b + snowflake-arctic-embed-l-v2, and notes dims-trust for local recipes (#2271). - llama-server section notes gbrain trusts the user-declared dimension (#2271). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * docs: post-ship doc sweep for v0.42.57.0 provider-agnostic gateway wave - KEY_FILES.md types.ts entry: document EmbeddingTouchpoint.trust_custom_dims (#2271 passthrough tier, runs after dims_options + Matryoshka allowlists) - ENGINES.md: embedding design-choice note now names the provider-agnostic gateway delegation instead of the stale OpenAI-only parenthetical - embedding-providers.md: drop an exact-duplicate doctor-8c paragraph - llms-full.txt regenerated (ENGINES.md is inlined in the bundle) Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * docs: apply codex doc-review findings for v0.42.57.0 (base-URL env note, litellm multimodal) - embedding-providers.md OpenAI section: document OPENAI_BASE_URL / ANTHROPIC_BASE_URL bare-host /v1 normalization (#1250 user-facing surface) - TL;DR table: litellm multimodal is backend-permitting (recipe declares supports_multimodal: true, routed via the openai-compat multimodal path), not "no" Co-Authored-By: Claude Fable 5 <noreply@anthropic.com> * test: pin engine-find-trajectory schema to 1536 + stop gateway-state leaks across shard files CI shard 5 failed 7 findTrajectory tests with 'expected 1280 dimensions, not 1536': engine-find-trajectory hardcodes 1536-d vectors but sizes its schema from AMBIENT gateway state in beforeAll — which runs before the legacy-embedding-preload's per-test 1536 restore. A preceding file that ends with a dimensionless configureGateway (facts-extract-silent-no-op) or a bare resetGateway poisons the next fresh initSchema down to 1280-d columns. The new test files in this PR reshuffled shard bin-packing and exposed the trap. - engine-find-trajectory: pin OpenAI/1536 explicitly before initSchema (the pattern bunfig's preload documents) — deterministic regardless of neighbors - facts-extract-silent-no-op, diagnose-embedding-dims, embed-preflight: restore the legacy 1536 pin in afterAll instead of ending reset/dimensionless Reproduced: synthetic dimensionless-gateway file + old victim = the exact 7 CI failures; with the pin = 0. Verified in-process pair runs both orders. --------- Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
126 lines
4.8 KiB
TypeScript
126 lines
4.8 KiB
TypeScript
/**
|
|
* Embedding credential preflight.
|
|
*
|
|
* v0.41.6.0 D1 — fail fast at sync/embed/import entry when the configured
|
|
* embedding provider can't be reached. Without this, gbrain proceeds into
|
|
* the import phase, hits 565 per-file embed errors, writes 565 identical
|
|
* "OpenAI embedding requires OPENAI_API_KEY." rows to `~/.gbrain/sync-failures.jsonl`,
|
|
* and blocks the sync bookmark from advancing.
|
|
*
|
|
* Routes through `gateway.diagnoseEmbedding()` so the structured reason
|
|
* (missing_env / no_touchpoint / unknown_provider / etc.) drives a precise
|
|
* user-facing error message.
|
|
*
|
|
* Skip protocol: callers SHOULD NOT call validateEmbeddingCreds when the
|
|
* user explicitly passed `--no-embed` (the canonical opt-out). The
|
|
* deferred-setup sentinel is owned by `assertEmbeddingEnabled` in
|
|
* `embedding-dim-check.ts`; this preflight runs AFTER that check fires.
|
|
*/
|
|
|
|
import { diagnoseEmbedding, type EmbeddingDiagnosis } from './ai/gateway.ts';
|
|
|
|
/**
|
|
* Tagged error thrown by validateEmbeddingCreds. CLI catch sites format
|
|
* `.userMessage` to stderr and exit non-zero. The structured fields
|
|
* (`provider`, `model`, `missingEnvVars`, `reason`) enable programmatic
|
|
* consumers (`gbrain doctor --json`, future autopilot health checks) to
|
|
* read state without parsing the human message.
|
|
*/
|
|
export class EmbeddingCredentialError extends Error {
|
|
readonly diagnosis: EmbeddingDiagnosis;
|
|
readonly userMessage: string;
|
|
|
|
constructor(diagnosis: EmbeddingDiagnosis, userMessage: string) {
|
|
super(userMessage);
|
|
this.name = 'EmbeddingCredentialError';
|
|
this.diagnosis = diagnosis;
|
|
this.userMessage = userMessage;
|
|
}
|
|
}
|
|
|
|
/**
|
|
* Run the preflight. Throws EmbeddingCredentialError when the gateway
|
|
* can't serve embeddings. Returns silently when ok.
|
|
*
|
|
* Pure function: reads nothing except what `gateway.diagnoseEmbedding()`
|
|
* already had at gateway configure-time.
|
|
*/
|
|
export function validateEmbeddingCreds(): void {
|
|
const d = diagnoseEmbedding();
|
|
if (d.ok) return;
|
|
throw new EmbeddingCredentialError(d, formatEmbeddingCredsError(d));
|
|
}
|
|
|
|
/**
|
|
* Format a paste-ready, multi-line error message from a non-ok diagnosis.
|
|
* Exported for tests and for the doctor JSON output.
|
|
*/
|
|
export function formatEmbeddingCredsError(d: EmbeddingDiagnosis): string {
|
|
if (d.ok) return '';
|
|
|
|
switch (d.reason) {
|
|
case 'no_gateway_config':
|
|
return [
|
|
'Embedding gateway is not configured.',
|
|
'This is usually a startup-order bug. Re-run with --no-embed to import',
|
|
'without embedding, then file an issue at https://github.com/garrytan/gbrain/issues',
|
|
].join('\n');
|
|
|
|
case 'no_model_configured':
|
|
return [
|
|
'No embedding model is configured for this brain.',
|
|
'',
|
|
' Set one: gbrain config set embedding_model openai:text-embedding-3-small',
|
|
' Or skip embedding now: re-run with --no-embed',
|
|
].join('\n');
|
|
|
|
case 'unknown_provider':
|
|
return [
|
|
`Embedding model "${d.model}" uses an unknown provider "${d.provider}".`,
|
|
'',
|
|
` ${d.message}`,
|
|
'',
|
|
' Pick a known provider: gbrain config set embedding_model openai:text-embedding-3-small',
|
|
].join('\n');
|
|
|
|
case 'no_touchpoint':
|
|
return [
|
|
`Provider "${d.provider}" does not offer an embedding touchpoint.`,
|
|
'',
|
|
' Switch providers: gbrain config set embedding_model openai:text-embedding-3-small',
|
|
' Or run with --no-embed to import-only and embed later.',
|
|
].join('\n');
|
|
|
|
case 'user_provided_dims_unset':
|
|
return [
|
|
`Provider "${d.provider}" ships no default embedding dimension; set one explicitly.`,
|
|
'',
|
|
` Re-init with the dimension: gbrain init --embedding-dimensions <N>`,
|
|
' (embedding_dimensions is a schema-sizing field — `config set` rejects it on purpose)',
|
|
' Or run with --no-embed to import-only and embed later.',
|
|
].join('\n');
|
|
|
|
case 'missing_env': {
|
|
const envs = d.missingEnvVars.join(', ');
|
|
const primaryEnv = d.missingEnvVars[0];
|
|
const lines = [
|
|
`Embedding model "${d.model}" requires ${envs}.`,
|
|
'',
|
|
`Set it in your shell, or:`,
|
|
` • Re-run with --no-embed to import-only and embed later once the key is set.`,
|
|
];
|
|
// Only offer a provider-switch hint when the current provider isn't openai
|
|
// (otherwise we'd be suggesting they switch to the thing they already have).
|
|
if (d.provider !== 'openai') {
|
|
lines.push(` • Switch providers: gbrain config set embedding_model openai:text-embedding-3-small`);
|
|
} else {
|
|
lines.push(` • Switch providers: gbrain config set embedding_model voyage:voyage-3-large`);
|
|
}
|
|
lines.push('');
|
|
lines.push(`Example shell setup:`);
|
|
lines.push(` export ${primaryEnv}=...`);
|
|
return lines.join('\n');
|
|
}
|
|
}
|
|
}
|