mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-30 11:22:34 +00:00
Garry TanandClaude Opus 4.7
0eb125c4d6
Optional anti-bias rewrite mode for `gbrain think`. When set, the active
calibration profile gets injected per the D22 placement spec (AFTER
retrieval evidence, BEFORE the user's question). The bias filter applies
to QUESTION FRAMING, not evidence interpretation — matches LLM-as-judge
best practice (bias prompts near end of context perform better).
Default behavior unchanged (R1 regression guard): omitting
--with-calibration produces the v0.28-vintage user-message shape with the
question first, then retrieval. Existing think users see no change.
Two user-message shapes in buildThinkUserMessage:
Default (no calibration):
Question: X
<pages>...</pages>
<takes>...</takes>
<graph>...</graph>
Respond with a single JSON object...
With calibration (D22):
<pages>...</pages>
<takes>...</takes>
<graph>...</graph>
<calibration holder="garry">
Track record: Brier 0.210 (lower is better).
Active patterns:
- You called early-stage tactics well — 8 of 10 held up.
Active bias tags: over-confident-geography
</calibration>
Question: X
Respond...
Calibration block is built by buildCalibrationBlock (exported for the
E3 contradictions probe to render the same shape).
System prompt extension (withCalibration:true):
- Names BOTH the user's PRIOR (default reasoning) AND the COUNTER-PRIOR
from their hedged-domain self.
- References active bias tags by name when relevant ("this fits the
over-confident-geography pattern").
- Does NOT silently substitute the debiased answer. ALWAYS surfaces
both priors transparently.
- Adds a "Calibration" section between Conflicts and Gaps in the
answer body.
RunThinkOpts extension:
- withCalibration?: boolean — opt-in
- calibrationHolder?: string — defaults to 'garry'
When withCalibration=true and no profile exists, runThink falls back to
baseline behavior + pushes NO_CALIBRATION_PROFILE to warnings (visible
to the operator). When the calibration fetch fails, CALIBRATION_FETCH_FAILED
warning surfaces with the underlying error. Either path keeps think working;
the calibration loop is enhancement, not requirement.
CLI: `gbrain think "<q>" --with-calibration [--calibration-holder <id>]`
Tests: 11 cases.
buildThinkSystemPrompt (4 cases): R1 regression — default/false/omitted
→ no anti-bias rules; with calibration → adds PRIOR + COUNTER-PRIOR +
bias-tag reference; preserves existing hard rules.
buildCalibrationBlock (3 cases): happy path, null brier omitted (not
"Brier null"), empty patterns + tags still well-formed.
buildThinkUserMessage (4 cases): R1 regression — without calibration:
question first; D22 placement — retrieval → calibration → question →
instruction; graph + calibration ordering; empty retrieval blocks render
placeholders without breaking shape.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>