fix: pre-landing review fixes — specialist round (4 reviewers, 22 findings)

The catch that mattered (testing + maintainability, independently): --llm
extraction metrics were computed per fixture but never reached any scoreboard
cell — dead matched_any_gold/stored_rows aggregation fields proved the
unfinished wiring. Now aggregated Σ-style into the write-back cell when llm
is on, pinned by a harness-level stubbed-transport test.

Also: RUN-scoped --llm BudgetTracker (a per-invocation cap multiplied by
fixture count — ~$550 worst case — now one tracker, exhaustion aborts the
run loudly); continuity loop collapsed to one prep per pair + read-only
reader per harness (the writer replay was provably observable-effect-free
and orderings rebuilt byte-identical brains — 90 preps → 15, runtime ~12s →
~7s, identical scores; baseline counts updated 24 → 12 honestly);
factKeywordProbe escapes ILIKE metacharacters; validator rejects unpassable
slug-less retrieve-turns; ci-gate fails HARD on a broken ref instead of
silently running ungated; privacy year-scan covers the gold dir; e2e tests
self-sufficient (shared artifacts in beforeAll) with a minimal-env --llm
gate; round4/cellKey single-sourced; explicit return in eval.ts dispatch;
stale comment + type anchor fixes.

scripts/check-synthetic-corpus-privacy.sh

@@ -115,7 +115,7 @@ if [ -d "$BB_DIR/fixtures" ]; then
       echo "  VIOLATION: out-of-range year in $match (fixtures use 2024-2026)"
       VIOLATIONS=$((VIOLATIONS + 1))
     fi
-  done < <(grep -rEn '\b(201[0-9]|202[0-3]|202[7-9]|20[3-9][0-9])\b' "$BB_DIR/fixtures" --include='*.json' 2>/dev/null || true)
+  done < <(grep -rEn '\b(201[0-9]|202[0-3]|202[7-9]|20[3-9][0-9])\b' "$BB_DIR/fixtures" "$BB_DIR/gold" --include='*.json' 2>/dev/null || true)
 fi
 
 if [ "$VIOLATIONS" -gt 0 ]; then

scripts/ci-brainbench-gate.sh

@@ -17,16 +17,26 @@ set -euo pipefail
 
 BASELINE_PATH="evals/brainbench/baselines/main.json"
 MAIN_REF="${BRAINBENCH_MAIN_REF:-origin/master}"
-OUT="${BRAINBENCH_OUT:-/tmp/brainbench-result.json}"
+# mktemp default (review finding): a fixed world-writable /tmp path is a
+# symlink-planting target on shared hosts. CI overrides via BRAINBENCH_OUT.
+OUT="${BRAINBENCH_OUT:-$(mktemp /tmp/brainbench-result-XXXXXX.json)}"
 MAIN_BASELINE="$(mktemp /tmp/brainbench-main-baseline-XXXXXX.json)"
 trap 'rm -f "$MAIN_BASELINE"' EXIT
 
+# Fail HARD when the ref itself is broken — only a genuinely-absent baseline
+# may take the ungated first-landing path (review finding: an unfetched ref
+# or typo'd BRAINBENCH_MAIN_REF must not silently disable the gate).
+if ! git rev-parse --verify --quiet "${MAIN_REF}^{commit}" > /dev/null; then
+  echo "[brainbench-gate] ERROR: ref ${MAIN_REF} does not resolve — fetch it or fix BRAINBENCH_MAIN_REF" >&2
+  exit 2
+fi
+
 if git show "${MAIN_REF}:${BASELINE_PATH}" > "$MAIN_BASELINE" 2>/dev/null; then
   echo "[brainbench-gate] comparing against ${MAIN_REF}:${BASELINE_PATH}"
   bun src/cli.ts eval brainbench --compare "$MAIN_BASELINE" --out "$OUT"
 else
-  # First landing: main has no baseline yet. Run without a gate so the PR
-  # that introduces BrainBench can commit the initial baseline.
+  # First landing: the ref exists but carries no baseline yet. Run without a
+  # gate so the PR that introduces BrainBench can commit the initial baseline.
   echo "[brainbench-gate] no baseline on ${MAIN_REF} yet — running ungated (initial-landing path)"
   bun src/cli.ts eval brainbench --out "$OUT"
 fi