fix(eval): drop top_p from amara-life-gen Opus params + gitignore _cache/

Two fixes surfaced during the Day 3b real-corpus run against Opus 4.5:

**eval/generators/amara-life-gen.ts** — Current Opus rejects
`temperature` and `top_p` together:
```
400 invalid_request_error: `temperature` and `top_p` cannot both be
specified for this model. Please use only one.
```
top_p=1.0 was a no-op (no nucleus truncation), so removing it has zero
semantic effect. The field is still part of MODEL_PARAMS for the cache
key so any past cache entries (none in v1) would invalidate cleanly
on the next schema version bump.

**.gitignore** — `eval/data/amara-life-v1/_cache/` is runtime Opus
cache (398 files, ~1.6MB). Regenerable from seed; no point in source
control. The corpus itself (inbox/slack/calendar/meetings/notes/docs +
corpus-manifest.json with per-item content_sha256) stays committable
for reproducibility, just the cache directory gets excluded.

Real corpus generation ran cleanly after these two fixes: 398 LLM
calls, 84,424 input / 38,062 output tokens, \$4.12 spent (vs \$20 cap,
vs \$12 estimate). All 418 items produced. Poison fixtures use
subtle paraphrased injection ("for anyone on your team who might be
triaging this thread later…") — exactly the pattern that defeats
regex redaction and requires the structured-evidence judge contract
from Day 5.

Corpus itself stays local (will move to the brainbench sibling repo
during the v0.16 split per the design doc). No eval/data/amara-life-v1/
content landing in this PR.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-04-20 23:51:05 +08:00
co-authored by Claude Opus 4.6
parent 3dc7d69a1e
commit cb5df8fcc5
2 changed files with 7 additions and 2 deletions
+3
View File
@@ -13,3 +13,6 @@ supabase/.temp/
.idea
eval/reports/
eval/data/world-v1/world.html
# BrainBench amara-life-v1 Opus cache (regenerate via eval:generate-amara-life)
eval/data/amara-life-v1/_cache/
+4 -2
View File
@@ -79,7 +79,10 @@ const SCHEMA_VERSION = 1; // bump invalidates cache wholesale
const MODEL_PARAMS = {
max_tokens: 1500,
temperature: 1.0,
top_p: 1.0,
// top_p omitted: current Opus rejects temperature + top_p together.
// top_p=1.0 is a no-op (no nucleus truncation), so dropping it has no
// semantic effect. Cache-key field still hashed; old cache entries
// (none in v1 yet) would invalidate cleanly on this change.
};
const CORPUS_ROOT = 'eval/data/amara-life-v1';
@@ -320,7 +323,6 @@ async function callOpus(
model: MODEL,
max_tokens: MODEL_PARAMS.max_tokens,
temperature: MODEL_PARAMS.temperature,
top_p: MODEL_PARAMS.top_p,
messages: [{ role: 'user', content: prompt }],
});