Garry Tan and Claude Opus 4.7
fd9a4ae1ce
cycle: grade_takes ensemble tiebreaker for borderline verdicts (T5 / E2)
...
Multi-judge ensemble tiebreaker, additive on top of T4's single-judge
foundation. Reuses gateway.chat as the per-model judge interface; runs
three judges in parallel via Promise.allSettled. Pure aggregation logic
in aggregateEnsemble() — no SQL, no LLM, hermetically testable.
When ensemble fires (T5 trigger band):
Only when ALL of:
- opts.useEnsemble === true (default false)
- opts.ensembleJudges array is non-empty
- single-model confidence in [0.6, 0.95) (configurable via
opts.ensembleTriggerBand)
- single-model verdict !== 'unresolvable'
Above 0.95 the single judge is already sufficient (T4 path). Below 0.6
the verdict is clearly review-only — ensemble wouldn't change the
posture. 'unresolvable' from single-judge means no evidence yet; calling
three more judges on the same evidence won't manufacture some.
Conservative auto-apply (D12):
Ensemble verdict auto-applies via engine.resolveTake only when ALL of:
- autoResolve === true (operator opt-in per D17)
- ensemble.agreement === 3 (3/3 unanimous)
- ensemble.minConfidence >= ensembleThreshold (default 0.85)
- winning verdict !== 'unresolvable'
Schema-level monotonic-tightening guard for ensembleThreshold lives in
the takes resolution layer.
Cache identity:
When ensemble fires, the cache row's judge_model_id becomes
'ensemble:<modelA>+<modelB>+<modelC>' — a future re-run with different
ensemble membership doesn't collide with prior verdicts. evidence_signature
is recomputed because it includes the judge_model_id.
aggregateEnsemble (pure):
- 3/3 unanimous → agreement=3, minConfidence=min across the three
- 2/3 majority → agreement=2, minConfidence across the agreeing two
- 1/1/1 disagreement → tie-break: prefer non-'unresolvable', then
alphabetical for determinism
- 'unresolvable' from one model NEVER tips a 2-vote majority toward
'unresolvable' — by-label tally only counts a model toward its own
label
- All three judges failing (allSettled rejected) → verdict='unresolvable'
with agreement=0; auto-apply path blocked
- Single judge survives + two fail → agreement=1; the lone verdict wins
but auto-apply gated by the 3/3 requirement
Tests: 16 cases.
aggregateEnsemble (6): 3/3, 2/3, 1/1/1, unresolvable-tipping-resistance,
all-failed, partial-failed-but-survives.
Phase trigger conditions (5): useEnsemble=false default, useEnsemble=true
in borderline band, single >= 0.95 skip, single < 0.6 skip, single =
'unresolvable' skip.
Phase auto-apply rules (5): 3/3+threshold+autoResolve, 2/3 majority no
apply, 3/3 below threshold no apply, one ensemble judge throws still
aggregates from allSettled, empty ensembleJudges falls through to
single.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com >
2026-05-17 16:12:34 -07:00
..
2026-05-15 06:48:36 -07:00
2026-05-11 22:54:35 -07:00
2026-05-14 19:13:14 -07:00
2026-05-17 15:57:03 -07:00
2026-05-17 08:32:03 -07:00
2026-05-12 14:33:29 -07:00
2026-05-17 14:02:45 -07:00
2026-05-17 08:32:03 -07:00
2026-05-15 06:48:36 -07:00
2026-05-10 20:06:31 -07:00
2026-04-29 23:46:36 -07:00
2026-05-07 21:52:58 -07:00
2026-05-10 20:06:31 -07:00
2026-05-11 19:25:48 -07:00
2026-05-02 08:41:11 -07:00
2026-05-11 22:54:35 -07:00
2026-05-06 21:14:34 -07:00
2026-04-18 16:57:38 +08:00
2026-04-21 18:35:45 -07:00
2026-05-14 21:06:42 -07:00
2026-05-08 13:25:48 -07:00
2026-05-08 13:25:48 -07:00
2026-04-11 21:25:44 -10:00
2026-04-14 19:41:34 -10:00
2026-04-22 01:34:22 -07:00
2026-05-14 19:13:14 -07:00
2026-05-14 19:13:14 -07:00
2026-05-07 21:52:58 -07:00
2026-05-14 20:15:29 -07:00
2026-04-30 20:47:32 -07:00
2026-05-07 21:52:58 -07:00
2026-05-10 20:50:40 -07:00
2026-05-02 22:01:05 -07:00
2026-04-20 23:14:38 +08:00
2026-04-29 10:12:10 -07:00
2026-05-10 20:06:31 -07:00
2026-04-21 11:51:32 -07:00
2026-04-25 12:25:34 -07:00
2026-05-10 13:20:07 -07:00
2026-05-02 08:41:11 -07:00
2026-05-11 19:25:48 -07:00
2026-04-09 12:25:04 -10:00
2026-05-14 21:06:42 -07:00
2026-04-25 12:25:34 -07:00
2026-05-14 19:13:14 -07:00
2026-04-25 12:25:34 -07:00
2026-05-11 22:54:35 -07:00
2026-04-29 23:46:36 -07:00
2026-05-09 19:47:56 -07:00
2026-05-07 13:41:46 -07:00
2026-05-09 19:47:56 -07:00
2026-05-02 08:41:11 -07:00
2026-05-07 10:03:38 -07:00
2026-05-09 20:46:34 -07:00
2026-04-25 12:25:34 -07:00
2026-04-25 12:25:34 -07:00
2026-04-25 12:25:34 -07:00
2026-05-14 19:50:50 -04:00
2026-05-13 13:14:58 -04:00
2026-05-09 22:58:19 -07:00
2026-05-13 13:14:58 -04:00
2026-04-06 16:50:15 -07:00
2026-05-08 13:25:48 -07:00
2026-04-26 15:49:48 -07:00
2026-05-11 21:49:57 -07:00
2026-05-06 20:39:56 -07:00
2026-05-06 20:39:56 -07:00
2026-05-06 20:39:56 -07:00
2026-04-29 22:13:05 -07:00
2026-05-09 16:57:47 -07:00
2026-04-30 01:23:29 -07:00
2026-05-08 17:07:51 -07:00
2026-05-09 20:46:34 -07:00
2026-05-11 23:02:03 -07:00
2026-05-09 20:46:34 -07:00
2026-04-14 19:41:34 -10:00
2026-05-08 13:25:48 -07:00
2026-04-22 16:24:23 -07:00
2026-05-03 20:41:39 -07:00
2026-04-29 22:21:07 -07:00
2026-04-20 21:54:36 +08:00
2026-04-26 15:49:48 -07:00
2026-05-10 18:59:11 -07:00
2026-05-07 22:56:22 -07:00
2026-05-13 13:14:58 -04:00
2026-05-17 14:27:31 -07:00
2026-04-30 01:23:29 -07:00
2026-05-02 22:01:05 -07:00
2026-05-13 13:14:58 -04:00
2026-04-20 21:54:36 +08:00
2026-05-14 19:13:14 -07:00
2026-05-07 21:52:58 -07:00
2026-05-14 21:22:10 -07:00
2026-05-06 20:58:19 -07:00
2026-05-15 18:54:36 -07:00
2026-05-07 21:52:58 -07:00
2026-04-11 00:01:09 -10:00
2026-05-07 10:03:38 -07:00
2026-05-10 06:34:40 -07:00
2026-04-14 19:41:34 -10:00
2026-05-11 23:02:03 -07:00
2026-05-17 08:51:33 -07:00
2026-05-17 08:51:33 -07:00
2026-04-25 12:25:34 -07:00
2026-05-01 09:02:43 -07:00
2026-05-01 09:02:43 -07:00
2026-05-01 09:02:43 -07:00
2026-05-13 13:14:58 -04:00
2026-05-17 08:32:03 -07:00
2026-05-17 08:32:03 -07:00
2026-05-11 22:42:54 -07:00
2026-05-17 08:32:03 -07:00
2026-05-11 22:42:54 -07:00
2026-05-17 08:32:03 -07:00
2026-05-17 08:32:03 -07:00
2026-05-17 08:32:03 -07:00
2026-05-11 22:42:54 -07:00
2026-05-17 08:32:03 -07:00
2026-05-11 22:42:54 -07:00
2026-05-17 08:32:03 -07:00
2026-05-17 08:32:03 -07:00
2026-05-17 08:32:03 -07:00
2026-05-17 08:32:03 -07:00
2026-05-01 09:02:43 -07:00
2026-05-16 13:19:04 -07:00
2026-05-01 09:02:43 -07:00
2026-05-01 09:02:43 -07:00
2026-05-13 13:14:58 -04:00
2026-05-10 06:34:40 -07:00
2026-05-10 06:34:40 -07:00
2026-05-10 06:34:40 -07:00
2026-05-10 06:34:40 -07:00
2026-05-10 06:34:40 -07:00
2026-05-10 06:34:40 -07:00
2026-05-10 06:34:40 -07:00
2026-05-10 06:34:40 -07:00
2026-05-10 06:34:40 -07:00
2026-05-10 06:34:40 -07:00
2026-05-10 06:34:40 -07:00
2026-05-10 06:34:40 -07:00
2026-05-12 14:33:29 -07:00
2026-04-13 21:03:40 -10:00
2026-05-17 14:27:31 -07:00
2026-05-02 22:01:05 -07:00
2026-05-11 19:25:48 -07:00
2026-05-11 19:25:48 -07:00
2026-05-09 20:46:34 -07:00
2026-04-29 10:12:10 -07:00
2026-05-10 06:34:40 -07:00
2026-05-06 21:14:34 -07:00
2026-05-09 20:46:34 -07:00
2026-05-10 12:49:14 -07:00
2026-05-09 16:57:47 -07:00
2026-05-09 16:57:47 -07:00
2026-05-10 12:49:14 -07:00
2026-05-17 08:51:33 -07:00
2026-05-09 16:57:47 -07:00
2026-05-10 12:49:14 -07:00
2026-05-09 16:57:47 -07:00
2026-05-10 12:49:14 -07:00
2026-05-09 16:57:47 -07:00
2026-05-10 12:49:14 -07:00
2026-05-10 12:49:14 -07:00
2026-05-10 20:06:31 -07:00
2026-05-10 12:49:14 -07:00
2026-05-10 12:49:14 -07:00
2026-05-11 19:25:48 -07:00
2026-05-11 19:25:48 -07:00
2026-05-09 16:57:47 -07:00
2026-05-09 16:57:47 -07:00
2026-05-09 16:57:47 -07:00
2026-05-09 16:57:47 -07:00
2026-05-09 16:57:47 -07:00
2026-05-09 16:57:47 -07:00
2026-05-09 16:57:47 -07:00
2026-04-20 07:30:00 +08:00
2026-04-14 21:40:48 -10:00
2026-04-25 12:25:34 -07:00
2026-05-17 08:51:33 -07:00
2026-04-09 10:17:13 -07:00
2026-04-13 07:49:13 -10:00
2026-04-16 23:03:15 -07:00
2026-04-13 07:49:13 -10:00
2026-04-23 19:34:27 -07:00
2026-05-12 14:33:29 -07:00
2026-04-29 23:46:36 -07:00
2026-04-29 23:46:36 -07:00
2026-04-26 20:45:05 -07:00
2026-04-29 23:26:25 -07:00
2026-04-26 20:45:05 -07:00
2026-04-29 23:46:36 -07:00
2026-05-14 19:13:14 -07:00
2026-05-17 08:00:29 -07:00
2026-05-17 16:12:34 -07:00
2026-05-17 16:08:20 -07:00
2026-04-18 18:16:18 +08:00
2026-04-29 22:13:05 -07:00
2026-05-09 22:58:19 -07:00
2026-05-01 09:02:43 -07:00
2026-05-13 13:14:58 -04:00
2026-05-14 20:51:11 -07:00
2026-05-11 22:54:35 -07:00
2026-05-07 10:03:38 -07:00
2026-05-14 20:51:11 -07:00
2026-04-12 07:48:47 -10:00
2026-04-25 12:25:34 -07:00
2026-05-07 22:56:22 -07:00
2026-04-18 16:57:38 +08:00
2026-05-13 13:14:58 -04:00
2026-05-11 19:25:48 -07:00
2026-04-16 23:03:15 -07:00
2026-04-22 01:34:22 -07:00
2026-05-13 13:14:58 -04:00
2026-04-25 12:25:34 -07:00
2026-04-25 12:25:34 -07:00
2026-05-11 21:49:57 -07:00
2026-04-26 20:45:05 -07:00
2026-04-11 21:25:44 -10:00
2026-05-07 13:41:46 -07:00
2026-05-07 19:49:46 -07:00
2026-04-26 20:45:05 -07:00
2026-04-24 00:08:54 -07:00
2026-05-09 19:47:56 -07:00
2026-05-07 22:56:22 -07:00
2026-05-04 21:11:15 -07:00
2026-05-01 09:02:43 -07:00
2026-05-17 08:00:29 -07:00
2026-05-13 13:14:58 -04:00
2026-05-08 13:25:48 -07:00
2026-05-14 21:22:10 -07:00
2026-04-26 20:45:05 -07:00
2026-05-10 12:49:14 -07:00
2026-05-09 20:46:34 -07:00
2026-05-09 20:46:34 -07:00
2026-05-11 22:54:35 -07:00
2026-04-18 16:57:38 +08:00
2026-04-18 16:57:38 +08:00
2026-04-18 18:16:18 +08:00
2026-04-19 07:14:24 +08:00
2026-04-22 01:34:22 -07:00
2026-04-20 07:30:00 +08:00
2026-04-29 23:46:36 -07:00
2026-04-22 01:34:22 -07:00
2026-04-25 12:25:34 -07:00
2026-04-25 12:25:34 -07:00
2026-04-26 20:45:05 -07:00
2026-05-07 10:03:38 -07:00
2026-05-11 19:25:48 -07:00
2026-05-10 06:34:40 -07:00
2026-04-22 01:34:22 -07:00
2026-04-24 01:09:28 -07:00
2026-04-29 23:06:16 -07:00
2026-05-10 20:06:31 -07:00
2026-05-02 22:01:05 -07:00
2026-05-02 22:01:05 -07:00
2026-05-10 13:33:05 -07:00
2026-05-03 20:41:39 -07:00
2026-05-10 12:49:14 -07:00
2026-05-09 19:47:56 -07:00
2026-05-14 20:15:29 -07:00
2026-05-14 20:15:29 -07:00
2026-05-14 19:13:14 -07:00
2026-04-30 01:23:29 -07:00
2026-05-07 21:52:58 -07:00
2026-05-17 14:02:45 -07:00
2026-05-06 21:14:34 -07:00
2026-05-07 10:03:38 -07:00
2026-05-03 20:41:39 -07:00
2026-04-25 12:25:34 -07:00
2026-04-08 23:26:11 -10:00
2026-05-09 20:46:34 -07:00
2026-05-14 21:22:10 -07:00
2026-04-12 17:38:48 -07:00
2026-04-23 19:34:27 -07:00
2026-05-02 08:41:11 -07:00
2026-04-20 07:30:00 +08:00
2026-04-21 18:35:45 -07:00
2026-05-09 20:46:34 -07:00
2026-05-03 20:16:15 -07:00
2026-05-11 19:25:48 -07:00
2026-04-29 23:46:36 -07:00
2026-04-30 20:47:32 -07:00
2026-05-17 16:03:42 -07:00
2026-05-15 18:54:36 -07:00
2026-04-12 13:23:55 -10:00
2026-05-14 19:13:14 -07:00
2026-04-25 12:25:34 -07:00
2026-05-13 13:14:58 -04:00
2026-05-13 13:14:58 -04:00
2026-05-07 10:03:38 -07:00
2026-05-07 21:52:58 -07:00
2026-05-07 21:52:58 -07:00
2026-04-16 23:03:15 -07:00
2026-04-22 01:34:22 -07:00
2026-04-22 01:34:22 -07:00
2026-05-11 23:20:17 -07:00
2026-05-11 23:20:17 -07:00
2026-05-07 21:52:58 -07:00
2026-05-07 21:52:58 -07:00
2026-05-03 20:16:15 -07:00
2026-04-23 19:34:27 -07:00
2026-04-25 12:25:34 -07:00
2026-05-11 22:54:35 -07:00
2026-04-19 07:14:24 +08:00
2026-05-10 13:20:07 -07:00
2026-04-11 21:25:44 -10:00
2026-04-25 12:25:34 -07:00
2026-05-15 06:48:36 -07:00
2026-04-21 18:35:45 -07:00
2026-05-10 13:20:07 -07:00
2026-04-24 01:39:58 -07:00
2026-05-17 08:00:29 -07:00
2026-05-06 16:32:11 -07:00
2026-05-08 13:25:48 -07:00
2026-04-30 23:17:54 -07:00
2026-04-23 19:34:27 -07:00
2026-05-07 21:52:58 -07:00
2026-04-29 23:46:36 -07:00
2026-05-17 14:02:45 -07:00
2026-04-27 22:59:08 -07:00
2026-05-06 21:14:34 -07:00
2026-05-07 10:03:38 -07:00
2026-04-25 12:25:34 -07:00
2026-04-16 23:03:15 -07:00
2026-05-15 06:48:36 -07:00
2026-05-13 13:14:58 -04:00
2026-05-12 14:33:29 -07:00
2026-04-18 18:16:18 +08:00
2026-04-29 23:46:36 -07:00
2026-04-30 20:47:32 -07:00
2026-05-07 13:17:27 -07:00
2026-05-14 20:15:29 -07:00
2026-04-09 10:17:13 -07:00
2026-04-23 19:34:27 -07:00
2026-04-22 02:07:00 -07:00
2026-05-06 20:39:56 -07:00
2026-04-18 16:57:38 +08:00
2026-04-30 23:17:54 -07:00
2026-04-23 19:34:27 -07:00
2026-05-02 08:41:11 -07:00
2026-04-14 19:41:34 -10:00
2026-05-11 22:54:35 -07:00
2026-05-14 20:51:11 -07:00
2026-05-14 19:13:14 -07:00
2026-04-29 22:21:07 -07:00
2026-05-14 19:13:14 -07:00
2026-05-06 21:14:34 -07:00
2026-05-06 21:14:34 -07:00
2026-04-22 16:24:23 -07:00
2026-05-06 11:54:06 -07:00
2026-05-09 22:58:19 -07:00
2026-05-03 20:41:39 -07:00
2026-04-22 16:24:23 -07:00
2026-04-29 22:21:07 -07:00
2026-04-29 22:21:07 -07:00
2026-04-29 22:21:07 -07:00
2026-04-29 22:21:07 -07:00
2026-05-17 14:02:45 -07:00
2026-04-13 07:49:13 -10:00
2026-05-17 08:51:33 -07:00
2026-04-21 21:14:17 -07:00
2026-04-21 21:14:17 -07:00
2026-04-22 01:34:22 -07:00
2026-05-08 17:07:51 -07:00
2026-04-22 01:34:22 -07:00
2026-04-09 10:17:13 -07:00
2026-05-17 14:27:31 -07:00
2026-05-06 11:54:06 -07:00
2026-05-14 21:06:42 -07:00
2026-04-25 12:25:34 -07:00
2026-04-29 22:53:41 -07:00
2026-04-25 12:25:34 -07:00
2026-05-09 20:46:34 -07:00
2026-04-29 22:53:41 -07:00
2026-04-25 12:25:34 -07:00
2026-05-09 21:18:53 -07:00
2026-05-17 14:02:45 -07:00
2026-05-10 06:34:40 -07:00
2026-05-07 23:07:00 -07:00
2026-05-09 20:46:34 -07:00
2026-05-07 23:07:00 -07:00
2026-05-10 06:34:40 -07:00
2026-05-10 06:34:40 -07:00
2026-05-09 20:46:34 -07:00
2026-05-07 23:07:00 -07:00
2026-05-10 06:34:40 -07:00
2026-05-11 23:20:17 -07:00
2026-05-10 18:59:11 -07:00
2026-05-17 14:02:45 -07:00
2026-05-06 21:14:34 -07:00
2026-05-13 13:14:58 -04:00
2026-04-29 23:46:36 -07:00
2026-04-14 19:41:34 -10:00
2026-05-07 21:52:58 -07:00
2026-04-20 23:14:38 +08:00
2026-05-04 21:11:15 -07:00
2026-04-25 12:25:34 -07:00
2026-05-08 13:25:48 -07:00
2026-05-11 22:54:35 -07:00
2026-05-09 20:46:34 -07:00
2026-05-08 13:25:48 -07:00
2026-04-19 18:23:02 +08:00
2026-05-07 21:52:58 -07:00
2026-05-08 13:25:48 -07:00
2026-05-07 13:41:46 -07:00
2026-05-13 12:21:21 -04:00
2026-04-22 01:34:22 -07:00
2026-05-06 21:14:34 -07:00
2026-05-12 14:33:29 -07:00
2026-05-12 14:33:29 -07:00
2026-05-14 21:06:42 -07:00
2026-05-06 11:54:06 -07:00
2026-04-20 07:30:00 +08:00
2026-04-09 10:17:13 -07:00
2026-05-06 11:54:06 -07:00