mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
Garry TanandClaude Opus 4.7
c39dbdb1ec
There was no tutorial for skillopt — only a reference guide
(docs/guides/skillopt.md) that opens at --bootstrap-from-routing and
assumes you already understand benchmarks, and an agent-facing SKILL.md.
README had ZERO skillopt mention. The one thing a user must hand-author
(the benchmark JSONL) was taught nowhere with a worked example.
New: docs/tutorials/improving-skills-with-skillopt.md — Diataxis tutorial
(learning-oriented), copy-pasteable end to end:
1. mental model in two sentences (SKILL.md is the trainable param, the
agent is frozen)
2. write your first benchmark from scratch — a complete 15-task rule-judge
starter you paste and run, with the full check-op table
(contains/regex/section_present/max_chars/min_citations/tool_called/
tool_not_called)
3. --dry-run cost preview (and that it exits 2 by convention, not failure)
4. real run + reading accepted(0)/no_improvement(1)/aborted(2) with the
actual stderr output shape
5. where output lands (best.md, versions/, history.json, rejected.json,
audit jsonl)
6. accept/reject — bundled vs user skills, --no-mutate vs
--allow-mutate-bundled
7. iterate by sharpening the benchmark
The load-bearing fix the tutorial makes that the reference guide got wrong:
the DEFAULT --split 4:1:5 needs ~50 tasks before it runs (sel = N/10, floor
5). A first-time author writing 10-15 tasks hits `D_sel has N task(s)
(need >=5)` and bounces. The tutorial ships 15 tasks + `--split 1:1:1`
(clean 5/5/5) so the copy-paste path actually works. Verified against the
real loadBenchmark + splitBench: the exact shipped block parses 15 unique
tasks and splits 5/5/5 with sel>=5; the system's own error message confirms
"need ~50 total for 4:1:5".
Discoverability (Diataxis cross-linking):
- README.md tutorials section: new entry (was zero skillopt mention)
- docs/tutorials/README.md: added under ## Shipped
- docs/guides/skillopt.md: "New to this? Start with the tutorial" callout
Every claim devex-verified against source: exit-code map from
skillopt.ts (accepted:0/no_improvement:1/aborted:2/errored:2), stderr
format from skillopt.ts:286-292, check ops from score.ts, output paths
from SKILL.md, split math from benchmark.ts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>