mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
v0.42.19.0 fix(skillopt): close the last gap in the AI SDK v6 tool-loop fix (write-capture mapper + regression test) (#1809)
* fix(ai/gateway): AI SDK v6 tool loop on non-Anthropic providers (#1782, #1764) v6's asSchema() rejected the bare { jsonSchema } object and called it as a function ("schema is not a function"), killing every tool-using agent run on non-Anthropic providers and skillopt on all providers. Wrap tool inputSchema with the SDK jsonSchema() helper, and add a pure toModelMessages() boundary adapter in chat() that converts tool results to the v6 ModelMessage shape (role:'tool' + typed output:{type:'json'|'error-text',value}, isError dropped, non-JSON-safe output normalized). toolLoop stays provider-neutral and unchanged. Both skillopt tool builders (rollout.ts, write-capture.ts) switch to the shared paramDefToSchema mapper so enum/default/items survive. Tests run the produced shapes through real generateText + MockLanguageModelV3. Co-Authored-By: michaeladair44 <michaeladair44@users.noreply.github.com> Co-Authored-By: justemu <justemu@users.noreply.github.com> Co-Authored-By: JE4NVRG <JE4NVRG@users.noreply.github.com> Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> * chore: bump version and changelog (v0.42.19.0) Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> --------- Co-authored-by: michaeladair44 <michaeladair44@users.noreply.github.com> Co-authored-by: justemu <justemu@users.noreply.github.com> Co-authored-by: JE4NVRG <JE4NVRG@users.noreply.github.com> Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
This commit is contained in:
co-authored by
michaeladair44
justemu
JE4NVRG
Claude Opus 4.8
parent
bde11bb18f
commit
3d2add15d9
@@ -2,6 +2,17 @@
|
||||
|
||||
All notable changes to GBrain will be documented in this file.
|
||||
|
||||
## [0.42.19.0] - 2026-06-02
|
||||
|
||||
**`gbrain skillopt --write-capture` rollouts now get full tool schemas, closing the last gap in the AI SDK v6 tool-loop fix.** The v6 fix that got agent loops working again on non-Anthropic providers (DeepSeek, Qwen, Groq, local models) shipped in v0.42.11.0 — but it fixed only one of the two places skillopt builds tool definitions. The `--write-capture` path (the virtual put_page/submit_job/file_upload registry the optimizer uses to test write-flavored skills) still handed the model a stripped-down schema with `enum`, `default`, and `items` dropped, so the optimizer couldn't see a tool's allowed values and proposed invalid calls. Both builders now use the same shared mapper.
|
||||
|
||||
### What changed
|
||||
|
||||
- **SkillOpt write-capture schema fidelity** (`src/core/skillopt/write-capture.ts`): the virtual-write tool builder now uses the shared `paramDefToSchema` mapper, matching the `rollout.ts` builder fixed in v0.42.11.0. `enum`/`default`/`items` survive into the schema the model sees.
|
||||
- **Regression hardening for the v6 fix** (`test/ai/gateway-tools-schema.test.ts`): a real-AI-SDK integration test (`generateText` + `MockLanguageModelV3`, no network) pins that the tool schema + tool-result message shapes the gateway produces are accepted by AI SDK v6 — with guards that the pre-fix bare-`{jsonSchema}` object AND the raw-tool-result-in-a-user-message shape both throw. The kind of test the original fix lacked, so the convergent v6 fix can't silently regress.
|
||||
- **SkillOpt schema-metadata test** (`test/skillopt/rollout-schema.test.ts`): asserts `enum` survives in BOTH the rollout and write-capture tool builders.
|
||||
|
||||
Closes the remaining surface of #1782 / #1764. The core fix was a convergent effort — credit michaeladair44, justemu, and JE4NVRG for the original diagnoses and patches.
|
||||
## [0.42.18.0] - 2026-06-03
|
||||
|
||||
**A scheduled `gbrain sync` can no longer spin forever and pile up dead processes, and `gbrain doctor` stops showing "100% of pages need link extraction" right after you ran the thing that's supposed to fix it.**
|
||||
|
||||
@@ -1,5 +1,40 @@
|
||||
# TODOS
|
||||
|
||||
## v0.42.x AI SDK v6 tool-schema fix follow-ups (#1782/#1764)
|
||||
|
||||
Surfaced by the codex outside-voice pass during `/plan-eng-review` and
|
||||
deliberately scoped OUT of the tool-schema fix (it's pre-existing + a separate
|
||||
structural change). Plan + GSTACK REVIEW REPORT at
|
||||
`~/.claude/plans/system-instruction-you-are-working-abstract-willow.md`.
|
||||
|
||||
- [ ] **P1 — Gateway toolLoop crash-replay sends a malformed ModelMessage
|
||||
history.** The gateway path never persists the tool-result feedback message:
|
||||
`toolLoop` pushes `{role:'user', content: toolResultBlocks}` with `void
|
||||
messageIdx` and NO persistence callback, so only assistant turns reach
|
||||
`subagent_messages` (via `onAssistantTurn`). On any multi-turn resume,
|
||||
`loadPriorMessages` (`subagent.ts:769`) returns
|
||||
`[user, assistant(tool-call), assistant(...), ...]` with the tool-result
|
||||
messages MISSING — a history the real AI SDK v6 rejects ("tool result missing
|
||||
for tool call"). The direct-Anthropic path reconciles this at
|
||||
`subagent.ts:334-418` (synthesize + persist the tool-result turn before the
|
||||
first chat call); the gateway branch does not. **Fresh runs — the actual
|
||||
#1782/#1764 reports — are unaffected**, which is why the tool-schema fix
|
||||
shipped without it. Two fix options: (a) add an `onToolResults` persistence
|
||||
callback to `toolLoop` so the feedback message lands in `subagent_messages`,
|
||||
or (b) mirror the direct-path reconciliation in the gateway branch of
|
||||
`subagent.ts` before the first `gatewayToolLoop` chat. Either is a structural
|
||||
change to the replay contract — own PR, own review. Caught because every
|
||||
toolLoop/replay test stubs the transport and never inspects the input
|
||||
messages; pair the fix with a `MockLanguageModelV3 + generateText` replay test
|
||||
(the seam landed in `test/ai/gateway-tools-schema.test.ts`).
|
||||
|
||||
- [ ] **P2 — SkillOpt `best.md` not written in `--no-mutate` runs.** From PR
|
||||
#1708 (scoped out of the tool-schema wave as tangential): in `--no-mutate`
|
||||
SkillOpt runs the accepted proposal isn't persisted because `acceptCandidate`
|
||||
is gated by the mutate decision. Write it explicitly via `atomicWrite`
|
||||
(`apply-edits.ts:311`) + `mkdirSync(recursive)` in
|
||||
`runOptimizationLoop` (`src/core/skillopt/orchestrator.ts`). Small, own PR.
|
||||
|
||||
## v0.42.12.0 #1685 brain-health-as-solved follow-ups (v0.42+)
|
||||
|
||||
Deferred from the v0.42.12.0 wave (issue #1685, the posture umbrella over #1678/#1735).
|
||||
|
||||
+1
-1
@@ -143,5 +143,5 @@
|
||||
"bun": ">=1.3.10"
|
||||
},
|
||||
"license": "MIT",
|
||||
"version": "0.42.18.0"
|
||||
"version": "0.42.19.0"
|
||||
}
|
||||
|
||||
@@ -21,7 +21,8 @@
|
||||
*/
|
||||
|
||||
import type { ChatToolDef, ToolHandler } from '../ai/gateway.ts';
|
||||
import { operations, type OperationContext } from '../operations.ts';
|
||||
import { operations, type OperationContext, type ParamDef } from '../operations.ts';
|
||||
import { paramDefToSchema } from '../../mcp/tool-defs.ts';
|
||||
import { loadConfig } from '../config.ts';
|
||||
import { BRAIN_TOOL_ALLOWLIST } from '../minions/tools/brain-allowlist.ts';
|
||||
import type { BrainEngine } from '../engine.ts';
|
||||
@@ -202,11 +203,13 @@ function buildOpContext(engine: BrainEngine): OperationContext {
|
||||
};
|
||||
}
|
||||
|
||||
function paramsToSchema(params: Record<string, { type: string; description?: string; required?: boolean }>): Record<string, unknown> {
|
||||
function paramsToSchema(params: Record<string, ParamDef>): Record<string, unknown> {
|
||||
return {
|
||||
type: 'object' as const,
|
||||
properties: Object.fromEntries(
|
||||
Object.entries(params).map(([k, v]) => [k, { type: v.type, description: v.description }]),
|
||||
// Canonical ParamDef→JSON Schema mapper (shared with subagent + MCP + HTTP)
|
||||
// so enum/default/items survive. Prior {type,description}-only map dropped them.
|
||||
Object.entries(params).map(([k, v]) => [k, paramDefToSchema(v)]),
|
||||
),
|
||||
required: Object.entries(params).filter(([, v]) => v.required).map(([k]) => k),
|
||||
};
|
||||
|
||||
@@ -0,0 +1,96 @@
|
||||
import { describe, expect, it } from 'bun:test';
|
||||
import { generateText, jsonSchema } from 'ai';
|
||||
import { MockLanguageModelV3 } from 'ai/test';
|
||||
import { toModelMessages, type ChatMessage } from '../../src/core/ai/gateway.ts';
|
||||
|
||||
// v0.42 AI SDK v6 fix — the regression guard that the original bug evaded.
|
||||
// Every gateway/toolLoop test stubs the chat transport, which short-circuits
|
||||
// BEFORE generateText runs, so neither asSchema() (tool schema normalization)
|
||||
// nor ModelMessage conversion was ever exercised in CI. This test drives the
|
||||
// REAL generateText with a MockLanguageModelV3 (no network/keys) so both
|
||||
// failure points are validated against the actual installed AI SDK v6.
|
||||
|
||||
function mockModel(): MockLanguageModelV3 {
|
||||
return new MockLanguageModelV3({
|
||||
doGenerate: async () => ({
|
||||
content: [{ type: 'text', text: 'ok' }],
|
||||
finishReason: 'stop',
|
||||
usage: { inputTokens: 1, outputTokens: 1, totalTokens: 2 },
|
||||
warnings: [],
|
||||
}),
|
||||
} as any);
|
||||
}
|
||||
|
||||
const internalMessages: ChatMessage[] = [
|
||||
{ role: 'user', content: 'find foo' },
|
||||
{
|
||||
role: 'assistant',
|
||||
content: [{ type: 'tool-call', toolCallId: 'tc1', toolName: 'search', input: { q: 'foo' } }],
|
||||
},
|
||||
{
|
||||
role: 'user', // toolLoop's internal feedback role; adapter must promote to 'tool'
|
||||
content: [{ type: 'tool-result', toolCallId: 'tc1', toolName: 'search', output: { hits: 3 } }],
|
||||
},
|
||||
];
|
||||
|
||||
describe('gateway tool schema + message shape (real AI SDK v6)', () => {
|
||||
it('jsonSchema()-wrapped tools + adapted messages pass generateText without throwing', async () => {
|
||||
const model = mockModel();
|
||||
// Built exactly as gateway.chat() builds it (the primary fix).
|
||||
const tools = {
|
||||
search: {
|
||||
description: 'search the brain',
|
||||
inputSchema: jsonSchema({ type: 'object', properties: { q: { type: 'string' } } } as any),
|
||||
},
|
||||
};
|
||||
|
||||
const result = await generateText({
|
||||
model: model as any,
|
||||
tools: tools as any,
|
||||
messages: toModelMessages(internalMessages) as any,
|
||||
});
|
||||
|
||||
expect(result.text).toBe('ok');
|
||||
// The tool result was promoted to a role:'tool' message in the prompt the
|
||||
// model actually received — proving ModelMessage conversion accepted it.
|
||||
const prompt = model.doGenerateCalls[0]!.prompt as any[];
|
||||
expect(prompt.some((m) => m.role === 'tool')).toBe(true);
|
||||
});
|
||||
|
||||
it('REGRESSION: the bare { jsonSchema } object (pre-fix shape) is rejected by v6', async () => {
|
||||
const model = mockModel();
|
||||
const badTools = {
|
||||
search: {
|
||||
description: 'search the brain',
|
||||
// The exact pre-v0.42 bug shape: a plain object, not a Schema.
|
||||
inputSchema: { jsonSchema: { type: 'object' } } as any,
|
||||
},
|
||||
};
|
||||
await expect(
|
||||
generateText({
|
||||
model: model as any,
|
||||
tools: badTools as any,
|
||||
messages: toModelMessages(internalMessages) as any,
|
||||
}),
|
||||
).rejects.toThrow();
|
||||
});
|
||||
|
||||
it('REGRESSION: raw tool-result in a role:user message (pre-fix shape) is rejected by v6', async () => {
|
||||
const model = mockModel();
|
||||
const tools = {
|
||||
search: {
|
||||
description: 'search',
|
||||
inputSchema: jsonSchema({ type: 'object', properties: { q: { type: 'string' } } } as any),
|
||||
},
|
||||
};
|
||||
// Pre-fix: toolLoop pushed { role:'user', content:[{type:'tool-result', output: <raw> }] }.
|
||||
const preFixMessages = [
|
||||
{ role: 'user', content: 'find foo' },
|
||||
{ role: 'assistant', content: [{ type: 'tool-call', toolCallId: 'tc1', toolName: 'search', input: { q: 'foo' } }] },
|
||||
{ role: 'user', content: [{ type: 'tool-result', toolCallId: 'tc1', toolName: 'search', output: { hits: 3 } }] },
|
||||
];
|
||||
await expect(
|
||||
generateText({ model: model as any, tools: tools as any, messages: preFixMessages as any }),
|
||||
).rejects.toThrow();
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,58 @@
|
||||
import { describe, expect, test } from 'bun:test';
|
||||
import { runRollout } from '../../src/core/skillopt/rollout.ts';
|
||||
import { buildWriteCaptureRegistry } from '../../src/core/skillopt/write-capture.ts';
|
||||
import type { ChatToolDef, ToolLoopOpts, ToolLoopResult } from '../../src/core/ai/gateway.ts';
|
||||
|
||||
// v0.42 AI SDK v6 fix: both skillopt tool-schema mappers (rollout.ts AND
|
||||
// write-capture.ts) previously mapped only {type,description}, silently
|
||||
// dropping enum/default/items. Both now route through the canonical
|
||||
// paramDefToSchema(). This pins that the dropped metadata survives in BOTH
|
||||
// sites (the "fix every path or one silently diverges" class).
|
||||
|
||||
function findTool(defs: ChatToolDef[], name: string): ChatToolDef {
|
||||
const t = defs.find((d) => d.name === name);
|
||||
expect(t).toBeDefined();
|
||||
return t!;
|
||||
}
|
||||
|
||||
describe('skillopt rollout tool schemas preserve ParamDef metadata', () => {
|
||||
test('rollout.ts: enum metadata survives for list_pages.sort + traverse_graph.direction', async () => {
|
||||
let captured: ChatToolDef[] = [];
|
||||
await runRollout({
|
||||
engine: {} as never,
|
||||
skillText: 'Answer the task using read-only brain tools when useful.',
|
||||
task: { task_id: 'schema-capture', task: 'noop', judge: { kind: 'rule', checks: [] } },
|
||||
targetModel: 'anthropic:claude-sonnet-4-6',
|
||||
toolLoopFn: async (opts: ToolLoopOpts): Promise<ToolLoopResult> => {
|
||||
captured = opts.tools;
|
||||
return {
|
||||
finalText: 'done',
|
||||
totalTurns: 0,
|
||||
totalUsage: { input_tokens: 0, output_tokens: 0, cache_read_tokens: 0, cache_creation_tokens: 0 },
|
||||
stopReason: 'end',
|
||||
messages: [{ role: 'assistant', content: [{ type: 'text', text: 'done' }] }],
|
||||
};
|
||||
},
|
||||
});
|
||||
|
||||
const listPages = findTool(captured, 'brain_list_pages');
|
||||
const sort = (listPages.inputSchema.properties as Record<string, any>).sort;
|
||||
expect(sort.enum).toContain('updated_desc');
|
||||
|
||||
const traverse = findTool(captured, 'brain_traverse_graph');
|
||||
const direction = (traverse.inputSchema.properties as Record<string, any>).direction;
|
||||
expect(direction.enum).toEqual(['in', 'out', 'both']);
|
||||
});
|
||||
|
||||
test('write-capture.ts: the second mapper preserves the same enum metadata', () => {
|
||||
const { defs } = buildWriteCaptureRegistry({} as never);
|
||||
|
||||
const traverse = findTool(defs, 'brain_traverse_graph');
|
||||
const direction = (traverse.inputSchema.properties as Record<string, any>).direction;
|
||||
expect(direction.enum).toEqual(['in', 'out', 'both']);
|
||||
|
||||
const listPages = findTool(defs, 'brain_list_pages');
|
||||
const sort = (listPages.inputSchema.properties as Record<string, any>).sort;
|
||||
expect(sort.enum).toContain('updated_desc');
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user