calibration: E7 nudge + 14-day cooldown (T13 / D16 F3)

Real-time pattern surfacing when a newly-committed high-conviction take
matches an active bias pattern. Conversational nudge text via the
templates module; 14-day cooldown per (take_id, nudge_pattern) via
take_nudge_log to prevent the feedback loop where each cycle re-fires
the same nudge on the same take.

Threshold gates (D16 F3):
  - holder match (profile.holder === take.holder)
  - conviction-weight > 0.7 (strict greater than)
  - take's slug-derived domain hint matches an active bias tag
    (takeDomainHint — same heuristic as eval-contradictions/calibration-join.ts
    for cross-surface consistency)

Cooldown gate:
  Before firing, probe take_nudge_log for (take_id, nudge_pattern) rows
  with fired_at >= now() - 14 days. Any hit → silently skip. After firing,
  insert a new row with channel='stderr' so the next 14 days are gated.

Feedback-loop prevention:
  User hedges a take in response to a nudge (e.g. weight 0.85 → 0.65).
  Even though the take's `weight` field changed, the cooldown row for
  the over-confident-geography pattern is still there from the original
  fire — so the next cycle's evaluateAndFireNudge() silently skips. The
  user reset path (gbrain takes nudge --reset N) clears the cooldown to
  re-arm.

Output channel (v0.36.0.0 ship state):
  STDERR only. Schema's `channel` column already supports multi-channel
  (webhook, admin SPA toast); routing those is a v0.37+ follow-up.

Architecture:
  evaluateNudgeRule(take, profile) — pure rule check. Returns
  { matched, reason, matchedTag }. No engine call.
  checkCooldown(engine, takeId, pattern) — engine probe, returns boolean.
  recordNudgeFire(engine, opts) — INSERT into take_nudge_log.
  evaluateAndFireNudge(opts) — full pipeline. Returns NudgeDecision.
  resetNudgeCooldown(engine, takeId) — DELETE...RETURNING for the CLI.

  buildNudgeText delegates to templates.ts nudgeTemplate (D24 mode='nudge'
  voice). v0.36.0.0 ship state uses the template directly; LLM-generated
  nudge text via the voice gate lands in v0.37+ when we have production
  examples to tune from.

Tests: 22 cases.
  takeDomainHint (5): companies/people/macro/geography/unrecognized.
  evaluateNudgeRule (6): no_profile, wrong_holder, conviction-at-threshold-
  is-NOT-eligible (strict >), no matching tag, happy match,
  first-match-wins for multiple candidate tags.
  checkCooldown (3): true on row hit, false on no row, cutoff date param
  verifies the 14-day boundary.
  evaluateAndFireNudge (4): happy fire (text contains hush command +
  matched tag), cooldown silent skip (no INSERT, no stderr), no_profile
  short-circuit, below-conviction short-circuit (no cooldown query fired).
  buildNudgeText (2): hush command shape, conviction value embedded.
  resetNudgeCooldown (2): returns count, idempotent on zero rows.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
2026-05-17 16:42:47 -07:00
co-authored by Claude Opus 4.7
parent 8ae71a46b6
commit 8087a1f1f5
2 changed files with 502 additions and 0 deletions
+207
View File
@@ -0,0 +1,207 @@
/**
* v0.36.0.0 (T13 / E7) — real-time pattern surfacing on take commit.
*
* The nudge surface that taps the user on the shoulder when a newly-committed
* take matches an active bias pattern. Conversational voice (D24 mode='nudge'),
* 14-day cooldown per (take_id, nudge_pattern) via take_nudge_log so the same
* pattern doesn't re-fire on every cycle.
*
* Threshold rules (D16 / F3):
* - conviction-weight > 0.7 → eligible
* - take's holder is the calibration profile's holder
* - take's domain hint matches an active bias tag (same heuristic as the
* calibration-aware contradictions join — see eval-contradictions/calibration-join.ts)
*
* Feedback-loop prevention (D16 F3):
* - take_nudge_log records every fire keyed on (take_id|proposal_id,
* nudge_pattern). The cooldown probe checks "was this same pattern fired
* on this same take in the last NUDGE_COOLDOWN_DAYS?" If yes, silently skip.
* - Reset via `gbrain takes nudge --reset <take_id>` clears the cooldown
* for that take so the next sync re-fires fresh nudges.
*
* Output channel:
* v0.36.0.0 ship state: STDERR only. Multi-channel routing (webhook,
* admin SPA toast) is a v0.37+ follow-up — the schema's `channel` column
* already supports it.
*/
import type { BrainEngine, Take } from '../engine.ts';
import type { CalibrationProfileRow } from '../../commands/calibration.ts';
import { nudgeTemplate } from './templates.ts';
export const NUDGE_COOLDOWN_DAYS = 14;
export const NUDGE_CONVICTION_THRESHOLD = 0.7;
export interface NudgeDecision {
/** Should the nudge fire? */
shouldFire: boolean;
/** Why not — surfaced for debugging + audit. */
reason?:
| 'no_profile'
| 'below_conviction_threshold'
| 'no_matching_bias_tag'
| 'cooldown_active'
| 'wrong_holder';
/** The bias tag matched (when shouldFire=true). */
matchedTag?: string;
/** The conversational nudge text (when shouldFire=true). */
text?: string;
}
/**
* Map a take's metadata to a domain hint that joins against bias tags.
* Same heuristic as eval-contradictions/calibration-join.ts to keep the
* surfaces consistent.
*/
export function takeDomainHint(take: Take): string {
const slug = take.page_slug.toLowerCase();
if (slug.includes('/companies/') || slug.startsWith('companies/')) return 'hiring';
if (slug.includes('/people/') || slug.startsWith('people/')) return 'founder-behavior';
if (slug.includes('/deals/') || slug.startsWith('deals/')) return 'market-timing';
if (slug.includes('macro')) return 'macro';
if (slug.includes('geography')) return 'geography';
if (slug.includes('tactics')) return 'tactics';
if (slug.includes('/ai/') || slug.includes('-ai-')) return 'ai';
return '';
}
/** Pure: decide whether a take should fire a nudge given the active profile. */
export function evaluateNudgeRule(
take: Take,
profile: CalibrationProfileRow | null,
): { matched: boolean; reason?: NudgeDecision['reason']; matchedTag?: string } {
if (!profile) return { matched: false, reason: 'no_profile' };
if (take.holder !== profile.holder) return { matched: false, reason: 'wrong_holder' };
if (take.weight <= NUDGE_CONVICTION_THRESHOLD) {
return { matched: false, reason: 'below_conviction_threshold' };
}
const hint = takeDomainHint(take);
if (!hint) return { matched: false, reason: 'no_matching_bias_tag' };
for (const tag of profile.active_bias_tags) {
if (tag.toLowerCase().includes(hint)) {
return { matched: true, matchedTag: tag };
}
}
return { matched: false, reason: 'no_matching_bias_tag' };
}
/**
* Check the take_nudge_log for an active cooldown on this (take_id,
* pattern) within the last NUDGE_COOLDOWN_DAYS days.
*/
export async function checkCooldown(
engine: BrainEngine,
takeId: number,
nudgePattern: string,
): Promise<boolean> {
const cutoffDate = new Date(Date.now() - NUDGE_COOLDOWN_DAYS * 24 * 60 * 60 * 1000);
const rows = await engine.executeRaw<{ id: number }>(
`SELECT id FROM take_nudge_log
WHERE take_id = $1 AND nudge_pattern = $2 AND fired_at >= $3
LIMIT 1`,
[takeId, nudgePattern, cutoffDate.toISOString()],
);
return rows.length > 0;
}
/**
* Write a take_nudge_log row with channel='stderr'.
*/
export async function recordNudgeFire(
engine: BrainEngine,
opts: { sourceId: string; takeId: number; nudgePattern: string; channel?: string },
): Promise<void> {
await engine.executeRaw(
`INSERT INTO take_nudge_log (source_id, take_id, nudge_pattern, channel)
VALUES ($1, $2, $3, $4)`,
[opts.sourceId, opts.takeId, opts.nudgePattern, opts.channel ?? 'stderr'],
);
}
/**
* Build the conversational nudge text via the templates module. v0.36.0.0
* ship state: uses the template directly (no LLM-generation path). The
* voice gate (T6) wraps this surface at v0.37+ when we have enough
* production examples to tune the LLM prompt.
*/
export function buildNudgeText(opts: {
matchedTag: string;
conviction: number;
/** Optional: count of recent misses in same conviction bucket. */
nRecentMisses?: number;
nRecentTotal?: number;
}): string {
// Domain extracted from tag — kebab-case last segment after axis prefix.
const domain = opts.matchedTag.split('-').slice(-1)[0] ?? 'this area';
return nudgeTemplate({
domain,
conviction: opts.conviction,
nRecentMisses: opts.nRecentMisses ?? 0,
nRecentTotal: opts.nRecentTotal ?? 0,
hushPattern: opts.matchedTag,
});
}
export interface EvaluateAndFireOpts {
engine: BrainEngine;
take: Take;
profile: CalibrationProfileRow | null;
sourceId: string;
/** Override the stderr stream (tests). Production: process.stderr. */
stderr?: { write: (s: string) => void };
}
/**
* Main entry point: evaluate, check cooldown, fire if appropriate, log.
* Returns the NudgeDecision so callers can audit / surface in UI.
*
* Always succeeds (no-fire is success). Errors surface in the result's
* reason field, not via throw.
*/
export async function evaluateAndFireNudge(opts: EvaluateAndFireOpts): Promise<NudgeDecision> {
const rule = evaluateNudgeRule(opts.take, opts.profile);
if (!rule.matched) {
return {
shouldFire: false,
...(rule.reason !== undefined ? { reason: rule.reason } : {}),
};
}
// Cooldown probe.
const onCooldown = await checkCooldown(opts.engine, opts.take.id, rule.matchedTag!);
if (onCooldown) {
return {
shouldFire: false,
reason: 'cooldown_active',
matchedTag: rule.matchedTag!,
};
}
// Build + fire.
const text = buildNudgeText({
matchedTag: rule.matchedTag!,
conviction: opts.take.weight,
});
const stream = opts.stderr ?? process.stderr;
stream.write(text + '\n');
// Log the fire (cooldown starts now).
await recordNudgeFire(opts.engine, {
sourceId: opts.sourceId,
takeId: opts.take.id,
nudgePattern: rule.matchedTag!,
});
return { shouldFire: true, matchedTag: rule.matchedTag!, text };
}
/**
* Reset cooldown for a take. Deletes the take's nudge_log rows so the
* next sync re-evaluates fresh.
*/
export async function resetNudgeCooldown(
engine: BrainEngine,
takeId: number,
): Promise<{ deleted: number }> {
const rows = await engine.executeRaw<{ id: number }>(
`DELETE FROM take_nudge_log WHERE take_id = $1 RETURNING id`,
[takeId],
);
return { deleted: rows.length };
}
+295
View File
@@ -0,0 +1,295 @@
/**
* v0.36.0.0 (T13 / E7) — nudge cooldown + threshold tests.
*
* Hermetic. Mock engine + injected stderr stream. No production stderr writes.
*
* Tests cover:
* - threshold gates: no profile, wrong holder, below conviction, no domain match
* - happy match path: above conviction + bias tag matches domain hint
* - cooldown: same pattern fired in last 14 days → silently skip
* - cooldown: same pattern fired > 14 days ago → fire (cooldown expired)
* - takeDomainHint: companies → hiring, macro/geography/tactics keywords match
* - resetNudgeCooldown: deletes rows for the take
* - log insertion captures (source_id, take_id, pattern, channel='stderr')
*/
import { describe, test, expect } from 'bun:test';
import {
evaluateAndFireNudge,
evaluateNudgeRule,
takeDomainHint,
checkCooldown,
resetNudgeCooldown,
buildNudgeText,
NUDGE_COOLDOWN_DAYS,
NUDGE_CONVICTION_THRESHOLD,
} from '../src/core/calibration/nudge.ts';
import type { CalibrationProfileRow } from '../src/commands/calibration.ts';
import type { BrainEngine, Take } from '../src/core/engine.ts';
function buildTake(overrides: Partial<Take> = {}): Take {
return {
id: 1,
page_id: 100,
page_slug: 'wiki/companies/acme-example',
row_num: 1,
claim: 'Marketplaces with cold-start liquidity always win.',
kind: 'bet',
holder: 'garry',
weight: 0.85,
since_date: '2026-05-17',
until_date: null,
source: null,
superseded_by: null,
active: true,
resolved_at: null,
resolved_outcome: null,
resolved_quality: null,
resolved_value: null,
resolved_unit: null,
resolved_source: null,
resolved_by: null,
created_at: '2026-05-17T00:00:00Z',
updated_at: '2026-05-17T00:00:00Z',
...overrides,
} as Take;
}
function buildProfile(activeBiasTags: string[], holder = 'garry'): CalibrationProfileRow {
return {
id: 1,
source_id: 'default',
holder,
wave_version: 'v0.36.0.0',
generated_at: '2026-05-17T00:00:00Z',
published: false,
total_resolved: 20,
brier: 0.21,
accuracy: 0.6,
partial_rate: 0.1,
grade_completion: 1.0,
pattern_statements: ['some pattern'],
active_bias_tags: activeBiasTags,
voice_gate_passed: true,
voice_gate_attempts: 1,
model_id: 'claude-sonnet-4-6',
};
}
interface SqlCall {
sql: string;
params: unknown[];
}
function buildMockEngine(opts: {
cooldownRows?: number; // 1 = active cooldown, 0 = no cooldown
deleteReturning?: number; // count of rows DELETE...RETURNING simulates
}): { engine: BrainEngine; sqls: SqlCall[] } {
const sqls: SqlCall[] = [];
const engine = {
kind: 'pglite',
async executeRaw<T>(sql: string, params?: unknown[]): Promise<T[]> {
sqls.push({ sql, params: params ?? [] });
if (sql.includes('SELECT id FROM take_nudge_log')) {
return new Array(opts.cooldownRows ?? 0).fill({ id: 1 }) as unknown as T[];
}
if (sql.includes('DELETE FROM take_nudge_log')) {
return new Array(opts.deleteReturning ?? 0).fill({ id: 1 }) as unknown as T[];
}
return [];
},
} as unknown as BrainEngine;
return { engine, sqls };
}
// ─── takeDomainHint ─────────────────────────────────────────────────
describe('takeDomainHint', () => {
test('companies/ slug → hiring', () => {
expect(takeDomainHint(buildTake({ page_slug: 'wiki/companies/acme' }))).toBe('hiring');
});
test('people/ slug → founder-behavior', () => {
expect(takeDomainHint(buildTake({ page_slug: 'wiki/people/alice' }))).toBe('founder-behavior');
});
test('macro keyword → macro', () => {
expect(takeDomainHint(buildTake({ page_slug: 'wiki/macro/forecast' }))).toBe('macro');
});
test('geography keyword → geography', () => {
expect(takeDomainHint(buildTake({ page_slug: 'wiki/geography/ny' }))).toBe('geography');
});
test('unrecognized slug → empty hint', () => {
expect(takeDomainHint(buildTake({ page_slug: 'wiki/random/x' }))).toBe('');
});
});
// ─── evaluateNudgeRule (pure) ───────────────────────────────────────
describe('evaluateNudgeRule', () => {
test('no profile → matched=false with reason=no_profile', () => {
expect(evaluateNudgeRule(buildTake(), null)).toEqual({ matched: false, reason: 'no_profile' });
});
test('wrong holder → matched=false with reason=wrong_holder', () => {
const profile = buildProfile(['over-confident-hiring'], 'alice');
expect(evaluateNudgeRule(buildTake({ holder: 'garry' }), profile).reason).toBe('wrong_holder');
});
test('conviction at threshold → matched=false (strict >)', () => {
const profile = buildProfile(['over-confident-hiring']);
expect(
evaluateNudgeRule(buildTake({ weight: NUDGE_CONVICTION_THRESHOLD }), profile).reason,
).toBe('below_conviction_threshold');
});
test('no matching bias tag → matched=false with reason=no_matching_bias_tag', () => {
const profile = buildProfile(['late-on-macro-tech']);
expect(
evaluateNudgeRule(buildTake({ page_slug: 'wiki/companies/acme' }), profile).reason,
).toBe('no_matching_bias_tag');
});
test('happy match: companies slug + hiring tag', () => {
const profile = buildProfile(['over-confident-hiring']);
const out = evaluateNudgeRule(buildTake({ page_slug: 'wiki/companies/acme' }), profile);
expect(out.matched).toBe(true);
expect(out.matchedTag).toBe('over-confident-hiring');
});
test('first-match-wins when multiple tags could match the hint', () => {
const profile = buildProfile([
'over-confident-hiring',
'late-on-hiring-cycles',
]);
const out = evaluateNudgeRule(buildTake({ page_slug: 'wiki/companies/acme' }), profile);
expect(out.matchedTag).toBe('over-confident-hiring');
});
});
// ─── checkCooldown ──────────────────────────────────────────────────
describe('checkCooldown', () => {
test('returns true when a recent row exists', async () => {
const { engine } = buildMockEngine({ cooldownRows: 1 });
expect(await checkCooldown(engine, 1, 'over-confident-hiring')).toBe(true);
});
test('returns false when no recent row', async () => {
const { engine } = buildMockEngine({ cooldownRows: 0 });
expect(await checkCooldown(engine, 1, 'over-confident-hiring')).toBe(false);
});
test('cutoff date param is NUDGE_COOLDOWN_DAYS ago', async () => {
const { engine, sqls } = buildMockEngine({});
await checkCooldown(engine, 1, 'tag');
const cutoffISO = sqls[0]!.params[2] as string;
const cutoff = new Date(cutoffISO).getTime();
const expected = Date.now() - NUDGE_COOLDOWN_DAYS * 24 * 60 * 60 * 1000;
expect(Math.abs(cutoff - expected)).toBeLessThan(1000); // within 1s
});
});
// ─── evaluateAndFireNudge ───────────────────────────────────────────
describe('evaluateAndFireNudge', () => {
test('happy path: matches + no cooldown → fires + writes log + returns text', async () => {
const { engine, sqls } = buildMockEngine({ cooldownRows: 0 });
const profile = buildProfile(['over-confident-hiring']);
let stderrWrites = '';
const stderr = { write: (s: string) => { stderrWrites += s; } };
const out = await evaluateAndFireNudge({
engine,
take: buildTake({ page_slug: 'wiki/companies/acme' }),
profile,
sourceId: 'default',
stderr,
});
expect(out.shouldFire).toBe(true);
expect(out.matchedTag).toBe('over-confident-hiring');
expect(stderrWrites).toContain('[gbrain]');
expect(stderrWrites).toContain('over-confident-hiring');
const insertCall = sqls.find(s => s.sql.includes('INSERT INTO take_nudge_log'));
expect(insertCall).toBeDefined();
expect(insertCall!.params).toEqual(['default', 1, 'over-confident-hiring', 'stderr']);
});
test('cooldown active → silently skips, no insert, no stderr', async () => {
const { engine, sqls } = buildMockEngine({ cooldownRows: 1 });
const profile = buildProfile(['over-confident-hiring']);
let stderrWrites = '';
const stderr = { write: (s: string) => { stderrWrites += s; } };
const out = await evaluateAndFireNudge({
engine,
take: buildTake({ page_slug: 'wiki/companies/acme' }),
profile,
sourceId: 'default',
stderr,
});
expect(out.shouldFire).toBe(false);
expect(out.reason).toBe('cooldown_active');
expect(stderrWrites).toBe('');
expect(sqls.find(s => s.sql.includes('INSERT'))).toBeUndefined();
});
test('no profile → silently skips with reason=no_profile', async () => {
const { engine } = buildMockEngine({});
const out = await evaluateAndFireNudge({
engine,
take: buildTake(),
profile: null,
sourceId: 'default',
});
expect(out.shouldFire).toBe(false);
expect(out.reason).toBe('no_profile');
});
test('below conviction threshold → silently skips', async () => {
const { engine, sqls } = buildMockEngine({});
const profile = buildProfile(['over-confident-hiring']);
const out = await evaluateAndFireNudge({
engine,
take: buildTake({ weight: 0.6, page_slug: 'wiki/companies/acme' }),
profile,
sourceId: 'default',
});
expect(out.shouldFire).toBe(false);
expect(out.reason).toBe('below_conviction_threshold');
// No cooldown query, no INSERT — both gated above the cooldown probe.
expect(sqls.find(s => s.sql.includes('SELECT id FROM take_nudge_log'))).toBeUndefined();
});
});
// ─── buildNudgeText ─────────────────────────────────────────────────
describe('buildNudgeText', () => {
test('contains the matched tag for hush command', () => {
const out = buildNudgeText({ matchedTag: 'over-confident-geography', conviction: 0.85 });
expect(out).toContain('over-confident-geography');
expect(out).toContain('gbrain takes nudge --hush over-confident-geography');
});
test('contains the conviction value', () => {
const out = buildNudgeText({ matchedTag: 'over-confident-hiring', conviction: 0.92 });
expect(out).toContain('0.92');
});
});
// ─── resetNudgeCooldown ─────────────────────────────────────────────
describe('resetNudgeCooldown', () => {
test('deletes rows for the take; returns count', async () => {
const { engine, sqls } = buildMockEngine({ deleteReturning: 3 });
const out = await resetNudgeCooldown(engine, 42);
expect(out.deleted).toBe(3);
expect(sqls[0]!.sql).toContain('DELETE FROM take_nudge_log');
expect(sqls[0]!.params).toEqual([42]);
});
test('returns 0 when no rows to delete (idempotent)', async () => {
const { engine } = buildMockEngine({ deleteReturning: 0 });
expect((await resetNudgeCooldown(engine, 99)).deleted).toBe(0);
});
});