diff --git a/test/fixtures/calibration/extract-takes-corpus/concept-startup-market-dynamics.gradeable-claims.json b/test/fixtures/calibration/extract-takes-corpus/concept-startup-market-dynamics.gradeable-claims.json new file mode 100644 index 000000000..963a4f40d --- /dev/null +++ b/test/fixtures/calibration/extract-takes-corpus/concept-startup-market-dynamics.gradeable-claims.json @@ -0,0 +1,90 @@ +{ + "_meta": { + "fixture_version": "v0.36.1.0", + "page": "concept-startup-market-dynamics.md", + "synthetic": true, + "notes": "Ground-truth gradeable-claims labels for the propose_takes F1 gate. Each claim should be extracted by a tuned prompt. Conviction is inferred from hedging language; F1 target >= 0.85 on training corpus per plan D19." + }, + "claims": [ + { + "claim_text": "Post-PMF vertical-AI startups will compound faster than horizontal SaaS over the next 18 months", + "kind": "prediction", + "domain": "market", + "conviction": 0.85, + "since_date": "2024-01-12", + "rationale": "High conviction language explicit" + }, + { + "claim_text": "Vertical-AI pivot was the right strategic call given market shift toward narrow workflows", + "kind": "judgment", + "domain": "strategy", + "conviction": 0.7, + "since_date": "2024-02-08", + "rationale": "I think + obvious by mid-2025 = moderate-strong" + }, + { + "claim_text": "Cold-start liquidity problems are usually positioning problems, not marketplace mechanics", + "kind": "judgment", + "domain": "marketplaces", + "conviction": 0.6, + "since_date": "2024-04-19", + "rationale": "Explicit modest conviction stamp" + }, + { + "claim_text": "widget-co plateau is a marketing problem not a product problem", + "kind": "judgment", + "domain": "company-specific", + "conviction": 0.8, + "since_date": "2024-06-22", + "rationale": "Strong conviction explicit" + }, + { + "claim_text": "At least three fund-a portfolio companies will hit revenue milestones in 2025", + "kind": "prediction", + "domain": "fund-portfolio", + "conviction": 0.7, + "since_date": "2024-09-11", + "rationale": "Specific prediction with date" + }, + { + "claim_text": "AI hype cycle runs hotter for 12 more months before first major correction", + "kind": "prediction", + "domain": "market-cycle", + "conviction": 0.75, + "since_date": "2024-12-03", + "rationale": "Strong prediction language" + }, + { + "claim_text": "Vertical-AI valuations re-rate by late 2025 due to retention gap", + "kind": "prediction", + "domain": "valuations", + "conviction": 0.7, + "since_date": "2025-03-14", + "rationale": "Specific prediction with date" + }, + { + "claim_text": "fund-c late-stage pivot is a mistake at their fund size", + "kind": "judgment", + "domain": "fund-strategy", + "conviction": 0.55, + "since_date": "2025-08-30", + "rationale": "Modest conviction explicit" + }, + { + "claim_text": "acme-example hits meaningful ARR milestone before end of 2026", + "kind": "bet", + "domain": "company-specific", + "conviction": 0.7, + "since_date": "2026-04-24", + "rationale": "Explicit bet language under ## Bets heading" + }, + { + "claim_text": "widget-co pivot announcement comes within 6 months", + "kind": "bet", + "domain": "company-specific", + "conviction": 0.65, + "since_date": "2026-04-24", + "rationale": "Bet language with concrete timeline" + } + ] +} diff --git a/test/fixtures/calibration/extract-takes-corpus/concept-startup-market-dynamics.md b/test/fixtures/calibration/extract-takes-corpus/concept-startup-market-dynamics.md new file mode 100644 index 000000000..d0185c6fd --- /dev/null +++ b/test/fixtures/calibration/extract-takes-corpus/concept-startup-market-dynamics.md @@ -0,0 +1,61 @@ +--- +title: "Startup Market Dynamics" +type: concept +tweet_count: 12 +created: 2026-04-01 +updated: 2026-04-24 +tags: [concept, calibration-fixture] +--- + +# Startup Market Dynamics + +## Summary +How early-stage market structure affects which startups break out and which +plateau. Synthetic anonymized concept page modeled on real-brain shape: +short summary, "usage" note, then a long Timeline of dated assertions with +varying conviction language. + +## Usage in your thinking +*This page is a synthetic fixture for the v0.36.1.0 calibration corpus. Real +concept pages on the brain side carry one assertion per timeline entry, +typically with verb framing like "argues / endorses / mentions / believes +/ predicts" that signals conviction strength.* + +--- + +## Timeline + +- **2024-01-12** | Post-PMF startups in vertical-AI categories will compound + faster than horizontal SaaS over the next 18 months. High conviction. + +- **2024-02-08** | The acme-example pivot from devtools to vertical-AI was + the right call given how the market is shifting toward narrow workflows + with deep domain integration. I think this becomes obvious by mid-2025. + +- **2024-04-19** | Most marketplaces that look like cold-start liquidity + problems are actually positioning problems — the founder hasn't picked a + side of the marketplace narrow enough to bootstrap from. fund-a's recent + losses fit this pattern. (Modest conviction, ~0.6.) + +- **2024-06-22** | widget-co's plateau is a marketing problem, not a product + problem. They under-invest in distribution by an order of magnitude + relative to comparable companies at their stage. Strong conviction. + +- **2024-09-11** | I expect at least three of the fund-a portfolio companies + to hit meaningful revenue milestones in 2025 — alice-example's company + being the most likely. (Specific prediction with date.) + +- **2024-12-03** | The current AI hype cycle will run hotter than people + expect for another 12 months before the first major correction. Strong + prediction. Worth grading against the actual market arc. + +- **2025-03-14** | Vertical-AI valuations are running ahead of their actual + retention data. We'll see the first re-rating wave by late 2025. + +- **2025-08-30** | fund-c's recent hires suggest they're pivoting their + thesis toward late-stage. I think this is a mistake at their fund size. + Modest conviction. + +## Bets +- I think acme-example hits a meaningful ARR milestone before end of 2026. +- I bet widget-co's pivot announcement comes within 6 months. diff --git a/test/fixtures/calibration/extract-takes-corpus/daily-2026-04-15.gradeable-claims.json b/test/fixtures/calibration/extract-takes-corpus/daily-2026-04-15.gradeable-claims.json new file mode 100644 index 000000000..be5f83f41 --- /dev/null +++ b/test/fixtures/calibration/extract-takes-corpus/daily-2026-04-15.gradeable-claims.json @@ -0,0 +1,50 @@ +{ + "_meta": { + "fixture_version": "v0.36.1.0", + "page": "daily-2026-04-15.md", + "synthetic": true, + "notes": "Daily-journal genre. Hedging language is the dominant signal — 'I think', 'I'm skeptical', 'maybe'. Tests propose_takes ability to extract from prose without explicit ## Takes section." + }, + "claims": [ + { + "claim_text": "acme-example pricing change at 40% is too aggressive given current data", + "kind": "judgment", + "domain": "pricing", + "conviction": 0.65, + "since_date": "2026-04-15", + "rationale": "'I think that's too aggressive' + data argument" + }, + { + "claim_text": "widget-co international expansion in 2027 is the wrong call given unit economics", + "kind": "judgment", + "domain": "strategy", + "conviction": 0.7, + "since_date": "2026-04-15", + "rationale": "'I'm skeptical' + explicit reasoning" + }, + { + "claim_text": "Founders who pivot late tend to outperform founders who pivot early (in this batch)", + "kind": "judgment", + "domain": "founder-patterns", + "conviction": 0.5, + "since_date": "2026-04-15", + "rationale": "Heuristic, hedged with 'maybe selection bias'" + }, + { + "claim_text": "At least two fund-b portfolio companies will fold in next 12 months", + "kind": "prediction", + "domain": "fund-portfolio", + "conviction": 0.6, + "since_date": "2026-04-15", + "rationale": "Explicit moderate conviction" + }, + { + "claim_text": "AI conference circuit quiets down by late 2026 as revenue numbers come in", + "kind": "prediction", + "domain": "market-sentiment", + "conviction": 0.55, + "since_date": "2026-04-15", + "rationale": "'Random small prediction' = self-tagged low-to-mid conviction" + } + ] +} diff --git a/test/fixtures/calibration/extract-takes-corpus/daily-2026-04-15.md b/test/fixtures/calibration/extract-takes-corpus/daily-2026-04-15.md new file mode 100644 index 000000000..af543c850 --- /dev/null +++ b/test/fixtures/calibration/extract-takes-corpus/daily-2026-04-15.md @@ -0,0 +1,33 @@ +--- +title: "2026-04-15" +type: daily +slug: daily/2026-04-15 +date: 2026-04-15 +--- + +# Wednesday 2026-04-15 + +Coffee with alice-example this morning. She's pushing hard on the +acme-example pricing change — wants to move enterprise tier 40% by end of +quarter. I think that's too aggressive. The data doesn't support it yet +and we'd be optimizing on noise. + +Meeting with the widget-co team in the afternoon. They're talking about +international expansion in 2027. I'm skeptical — their unit economics in +the home market aren't where they need to be, and international expansion +right now would compound the burn problem. I told them so. We'll see if +they listen. + +Late afternoon journal: I keep coming back to the same pattern. Founders +who pivot late tend to do better than founders who pivot early, in this +batch at least. Maybe it's selection bias — only the founders with deep +customer instinct survive long enough to pivot late. Either way, it's a +heuristic I'm tracking. + +One more bet I want to log: I think the next 12 months will see at least +two of the fund-b portfolio companies fold. Their thesis hasn't worked +out and the LPs are getting restless. Moderate conviction, ~0.6. + +Random small prediction: the AI conference circuit will quiet down by +late 2026 as the actual revenue numbers start coming in. People are going +to start asking hard questions about retention. diff --git a/test/fixtures/calibration/extract-takes-corpus/meeting-2026-04-10-fundraise-fund-a.gradeable-claims.json b/test/fixtures/calibration/extract-takes-corpus/meeting-2026-04-10-fundraise-fund-a.gradeable-claims.json new file mode 100644 index 000000000..8b7243203 --- /dev/null +++ b/test/fixtures/calibration/extract-takes-corpus/meeting-2026-04-10-fundraise-fund-a.gradeable-claims.json @@ -0,0 +1,58 @@ +{ + "_meta": { + "fixture_version": "v0.36.1.0", + "page": "meeting-2026-04-10-fundraise-fund-a.md", + "synthetic": true, + "notes": "Meeting-note genre. Claims appear both in prose and in the explicit ## Takes section. Both should be extracted by the tuned propose_takes prompt. Conviction values explicit in the ## Takes section; inferred for prose claims." + }, + "claims": [ + { + "claim_text": "LA market will be slower than the SF data implies for acme-example", + "kind": "judgment", + "domain": "geography", + "conviction": 0.65, + "since_date": "2026-04-10", + "rationale": "Prose hedge: 'I don't think she's modeling correctly'" + }, + { + "claim_text": "Vertical-depth competitive moat thesis is weaker than alice-example claims", + "kind": "judgment", + "domain": "competitive-dynamics", + "conviction": 0.55, + "since_date": "2026-04-10", + "rationale": "'I'm not sure I buy that' = mild dissent" + }, + { + "claim_text": "acme-example's current pricing leaves money on the table for top-decile cohort", + "kind": "judgment", + "domain": "pricing", + "conviction": 0.6, + "since_date": "2026-04-10", + "rationale": "Modest conviction inferred" + }, + { + "claim_text": "acme-example closes Series A on Q4 2026 timeline", + "kind": "bet", + "domain": "fundraise", + "conviction": 0.7, + "since_date": "2026-04-10", + "rationale": "Explicit conviction in ## Takes" + }, + { + "claim_text": "fund-a leads acme-example's Series A round", + "kind": "bet", + "domain": "fundraise", + "conviction": 0.75, + "since_date": "2026-04-10", + "rationale": "Explicit conviction in ## Takes" + }, + { + "claim_text": "Vertical-depth competitive thesis will look weaker in 12 months", + "kind": "prediction", + "domain": "competitive-dynamics", + "conviction": 0.55, + "since_date": "2026-04-10", + "rationale": "Explicit conviction in ## Takes" + } + ] +} diff --git a/test/fixtures/calibration/extract-takes-corpus/meeting-2026-04-10-fundraise-fund-a.md b/test/fixtures/calibration/extract-takes-corpus/meeting-2026-04-10-fundraise-fund-a.md new file mode 100644 index 000000000..60b770249 --- /dev/null +++ b/test/fixtures/calibration/extract-takes-corpus/meeting-2026-04-10-fundraise-fund-a.md @@ -0,0 +1,48 @@ +--- +title: "Fundraise office hours — fund-a + alice-example" +type: meeting +slug: meetings/2026-04-10-fundraise-fund-a +date: 2026-04-10 +attendees: [you, alice-example, fund-a] +--- + +# Fundraise OH — fund-a / alice-example + +Office hours session with alice-example (acme-example) and fund-a partner. +Topic: acme-example's planned Series A close in late 2026. + +## Discussion + +alice-example walked through the new metrics. Retention curves look strong +across two cohorts. She thinks net retention crosses meaningful threshold +by Q3 if the current expansion motion holds. I pushed back on the +geography assumption — the LA market is going to be slower than the SF +data implies, and I don't think she's modeling that correctly yet. + +fund-a partner asked about competitive dynamics. alice-example argues +acme-example's vertical depth means they win the workflow even when +horizontal competitors enter the space. I'm not sure I buy that — there +are at least two well-funded competitors I'd take seriously, and one of +them ships fast. Worth grading this claim against actual win-rate data +in 12 months. + +Discussion of pricing. I think the current pricing leaves money on the +table for the top-decile customer cohort. alice-example pushed back — +she wants to grow into the pricing rather than annoy early adopters. +Probably the right call for now, but I'd want to revisit by end of year. + +## Takes + +- I bet acme-example closes the Series A on the timeline she's projecting + (Q4 2026). Conviction ~0.7. Geography risk is the main downside. +- I bet fund-a leads the round. They've been signaling interest for two + quarters and the partner showed up to OH. Conviction ~0.75. +- I predict the competitive thesis (vertical depth wins) will look weaker + in 12 months than it does today. Conviction ~0.55 — genuinely uncertain. +- alice-example's pricing instinct is right for the current stage. Low + conviction — I'd revise depending on how the next cohort behaves. + +## Followups + +- Revisit the geography model in 90 days. +- Get the win-rate numbers vs the two named competitors when they exist. diff --git a/test/fixtures/calibration/holdout/concept-founder-execution.gradeable-claims.json b/test/fixtures/calibration/holdout/concept-founder-execution.gradeable-claims.json new file mode 100644 index 000000000..fc7e08932 --- /dev/null +++ b/test/fixtures/calibration/holdout/concept-founder-execution.gradeable-claims.json @@ -0,0 +1,59 @@ +{ + "_meta": { + "fixture_version": "v0.36.1.0", + "page": "concept-founder-execution.md", + "synthetic": true, + "holdout": true, + "notes": "Blind holdout per plan D19. F1 target >= 0.8 (slightly relaxed vs training)." + }, + "claims": [ + { + "claim_text": "Best founders ship a working version within first 4 weeks of starting", + "kind": "judgment", + "domain": "founder-patterns", + "conviction": 0.8, + "since_date": "2024-03-04", + "rationale": "Strong conviction explicit" + }, + { + "claim_text": "Shipping velocity in first weeks correlates with eventual scale outcomes", + "kind": "judgment", + "domain": "founder-patterns", + "conviction": 0.7, + "since_date": "2024-07-11", + "rationale": "Pattern claim with data anchor" + }, + { + "claim_text": "charlie-example will struggle with the widget-co rebuild due to perfectionism", + "kind": "prediction", + "domain": "founder-execution", + "conviction": 0.6, + "since_date": "2024-11-19", + "rationale": "Explicit moderate conviction" + }, + { + "claim_text": "fund-c portfolio will re-rate down at next mark-to-market due to slow TTC", + "kind": "prediction", + "domain": "fund-performance", + "conviction": 0.7, + "since_date": "2025-02-28", + "rationale": "'I bet' = bet language" + }, + { + "claim_text": "alice-example outperforms seed projections by year 2", + "kind": "bet", + "domain": "company-specific", + "conviction": 0.7, + "since_date": "2026-04-22", + "rationale": "Bet under ## Bets heading" + }, + { + "claim_text": "At least one fund-c company hits revenue milestone by mid-2026", + "kind": "bet", + "domain": "fund-portfolio", + "conviction": 0.65, + "since_date": "2026-04-22", + "rationale": "Bet under ## Bets heading; mild hedge ('despite slow start')" + } + ] +} diff --git a/test/fixtures/calibration/holdout/concept-founder-execution.md b/test/fixtures/calibration/holdout/concept-founder-execution.md new file mode 100644 index 000000000..56c70086b --- /dev/null +++ b/test/fixtures/calibration/holdout/concept-founder-execution.md @@ -0,0 +1,41 @@ +--- +title: "Founder Execution" +type: concept +tweet_count: 8 +created: 2026-04-01 +updated: 2026-04-22 +tags: [concept, calibration-fixture, holdout] +--- + +# Founder Execution + +## Summary +Patterns in how high-performing founders ship versus how struggling ones +stall. Synthetic anonymized holdout page — NOT seen during prompt tuning. + +## Timeline + +- **2024-03-04** | The best founders ship a working version within the + first 4 weeks of starting. Everyone else negotiates with the problem. + Strong conviction. + +- **2024-07-11** | alice-example shipped the acme-example MVP in 11 days + from idea-clarity. That kind of velocity correlates strongly with + eventual scale outcomes in my data. + +- **2024-11-19** | charlie-example is going to struggle with the + widget-co rebuild. He's a strong engineer but I don't think he's + willing to ship the rough version. Moderate conviction, ~0.6. + +- **2025-02-28** | The fund-c portfolio's median time-to-first-paying- + customer is too long. I bet they re-rate down at next mark-to-market. + +- **2025-05-15** | charlie-example proved me wrong on widget-co. The + rebuild shipped faster than I predicted and the architecture choices + look strong. Good update on my prior. + +## Bets + +- I think alice-example outperforms her seed projections by year 2. +- I bet at least one fund-c company hits a meaningful revenue milestone + by mid-2026 despite the slow start. diff --git a/test/fixtures/calibration/holdout/daily-2026-04-18.gradeable-claims.json b/test/fixtures/calibration/holdout/daily-2026-04-18.gradeable-claims.json new file mode 100644 index 000000000..d21ea46ab --- /dev/null +++ b/test/fixtures/calibration/holdout/daily-2026-04-18.gradeable-claims.json @@ -0,0 +1,43 @@ +{ + "_meta": { + "fixture_version": "v0.36.1.0", + "page": "daily-2026-04-18.md", + "synthetic": true, + "holdout": true, + "notes": "Daily-journal holdout. Tests propose_takes handling of probabilistic framing ('75/25 in favor') and explicit conviction self-tagging ('call it ~0.5')." + }, + "claims": [ + { + "claim_text": "acme-example hits meaningful ARR milestone by year-end", + "kind": "prediction", + "domain": "company-specific", + "conviction": 0.75, + "since_date": "2026-04-18", + "rationale": "Explicit '75/25 in favor' probabilistic framing" + }, + { + "claim_text": "widget-co has 2 quarters of runway before emergency raise", + "kind": "prediction", + "domain": "company-specific", + "conviction": 0.7, + "since_date": "2026-04-18", + "rationale": "'I think' + concrete timeline + supporting signals" + }, + { + "claim_text": "At least one fund-b portfolio company folds by Q3", + "kind": "prediction", + "domain": "fund-portfolio", + "conviction": 0.65, + "since_date": "2026-04-18", + "rationale": "'I expect' = moderate conviction" + }, + { + "claim_text": "Vertical-SaaS IPO window reopens by Q2 2027", + "kind": "prediction", + "domain": "market-cycle", + "conviction": 0.5, + "since_date": "2026-04-18", + "rationale": "Explicit self-tagged conviction ('call it ~0.5')" + } + ] +} diff --git a/test/fixtures/calibration/holdout/daily-2026-04-18.md b/test/fixtures/calibration/holdout/daily-2026-04-18.md new file mode 100644 index 000000000..1f8b97685 --- /dev/null +++ b/test/fixtures/calibration/holdout/daily-2026-04-18.md @@ -0,0 +1,31 @@ +--- +title: "2026-04-18" +type: daily +slug: daily/2026-04-18 +date: 2026-04-18 +--- + +# Saturday 2026-04-18 + +Weekend reading + thinking session. Mostly catching up on portfolio +metrics dashboards. + +A few things landed for me this week: + +The acme-example metrics keep getting stronger across every cohort I look +at. alice-example is going to crush her year-end target. I'd put the +over/under on a meaningful ARR milestone by year-end at about 75/25 in +favor. That's higher conviction than my baseline geography prior would +suggest, and I'm aware of that. + +widget-co is the opposite story. The retention curve is flattening and +the customer-support ticket volume is way up. I think they have 2 quarters +of runway before they're forced into an emergency raise. Bob hasn't +acknowledged this publicly yet but the signal is loud. + +fund-b's portfolio update came in. Two of the named companies look weak. +I expect at least one fold by Q3. + +Random thing I'm tracking: the IPO window for vertical-SaaS reopens by +Q2 2027. Not a prediction I'm willing to put a number on yet — call it +~0.5 — but I want to see if I'm right. diff --git a/test/fixtures/calibration/holdout/essay-on-conviction.gradeable-claims.json b/test/fixtures/calibration/holdout/essay-on-conviction.gradeable-claims.json new file mode 100644 index 000000000..20e1956b5 --- /dev/null +++ b/test/fixtures/calibration/holdout/essay-on-conviction.gradeable-claims.json @@ -0,0 +1,67 @@ +{ + "_meta": { + "fixture_version": "v0.36.1.0", + "page": "essay-on-conviction.md", + "synthetic": true, + "holdout": true, + "notes": "Essay-on-self-calibration genre — this is the meta page where the author reflects on their own track record. Mix of meta-claims (about own calibration) and concrete predictions. Self-rated convictions in the final section should be extracted with exact conviction values." + }, + "claims": [ + { + "claim_text": "I am calibrated well on early-stage PMF calls (gut hits ~70% of cohorts)", + "kind": "judgment", + "domain": "self-calibration", + "conviction": 0.75, + "since_date": "2026-04-19", + "rationale": "Meta-claim about own track record" + }, + { + "claim_text": "I am over-confident on geography (wrong ~60% of the time)", + "kind": "judgment", + "domain": "self-calibration", + "conviction": 0.7, + "since_date": "2026-04-19", + "rationale": "Meta-claim — bias-pattern identification" + }, + { + "claim_text": "I am poorly calibrated on macro-timing predictions (0/3 on AI peak)", + "kind": "judgment", + "domain": "self-calibration", + "conviction": 0.85, + "since_date": "2026-04-19", + "rationale": "Strong meta-claim with explicit count" + }, + { + "claim_text": "I am well-calibrated on competitive moat questions", + "kind": "judgment", + "domain": "self-calibration", + "conviction": 0.7, + "since_date": "2026-04-19", + "rationale": "Meta-claim with data anchor" + }, + { + "claim_text": "Vertical-AI valuations compress within 18 months", + "kind": "prediction", + "domain": "valuations", + "conviction": 0.6, + "since_date": "2026-04-19", + "rationale": "Explicit self-tagged conviction with de-rating note" + }, + { + "claim_text": "Next big platform shift comes from agent-orchestration primitives not model providers", + "kind": "prediction", + "domain": "platform", + "conviction": 0.75, + "since_date": "2026-04-19", + "rationale": "Explicit strong conviction" + }, + { + "claim_text": "At least one major SaaS public market exit happens by mid-2027", + "kind": "prediction", + "domain": "ipo-market", + "conviction": 0.55, + "since_date": "2026-04-19", + "rationale": "Explicit weak conviction ('close to coin-flip')" + } + ] +} diff --git a/test/fixtures/calibration/holdout/essay-on-conviction.md b/test/fixtures/calibration/holdout/essay-on-conviction.md new file mode 100644 index 000000000..88990eaeb --- /dev/null +++ b/test/fixtures/calibration/holdout/essay-on-conviction.md @@ -0,0 +1,41 @@ +--- +title: "On conviction" +type: essay +slug: writing/on-conviction +date: 2026-04-19 +--- + +# On conviction + +The hardest part of investing isn't picking winners. It's calibrating +how much you actually know about each pick. + +I've watched myself for a few years now. Patterns: + +I'm right more often than I think on early-stage product-market-fit +calls. The retroactive look is consistent — when I had a strong feeling +in the first meeting, I was right ~70% of the time across the cohorts +I've tracked. I should probably trust those gut calls more, not less. + +I'm worse than I think on geographic claims. The "this won't work outside +SF" prior fires too often and is wrong about 60% of the time. The +geography-overconfidence pattern is real and I should de-rate every +geography-flavored take I make. + +I'm worse than I think on macro timing. I've called the AI hype cycle +peak three times in the last 18 months and been wrong every time. Three +strikes. I should stop making macro-timing predictions until I have a +better track record. + +I'm well-calibrated on competitive moat questions. The 5-year retrospect +on my moat calls is essentially at parity with the market. I can keep +trusting those. + +What I'm betting on next: +- Vertical-AI valuations will compress within 18 months. Moderate + conviction, ~0.6 — and I'm going to de-rate this further given the + macro-timing pattern above. +- The next big platform shift is going to come from agent-orchestration + primitives, not model providers. Strong conviction, ~0.75. +- At least one major SaaS public market exit happens by mid-2027. + Conviction ~0.55 — close to coin-flip. diff --git a/test/fixtures/calibration/holdout/meeting-2026-04-17-hiring-charlie-example.gradeable-claims.json b/test/fixtures/calibration/holdout/meeting-2026-04-17-hiring-charlie-example.gradeable-claims.json new file mode 100644 index 000000000..4c6a7252e --- /dev/null +++ b/test/fixtures/calibration/holdout/meeting-2026-04-17-hiring-charlie-example.gradeable-claims.json @@ -0,0 +1,51 @@ +{ + "_meta": { + "fixture_version": "v0.36.1.0", + "page": "meeting-2026-04-17-hiring-charlie-example.md", + "synthetic": true, + "holdout": true, + "notes": "Hiring-meeting holdout. Tests propose_takes ability to extract genuinely-uncertain claims (conviction near 0.5) where many extractors drop the signal entirely." + }, + "claims": [ + { + "claim_text": "charlie-example hasn't operated at the velocity acme-example needs", + "kind": "judgment", + "domain": "hiring", + "conviction": 0.6, + "since_date": "2026-04-17", + "rationale": "Prose concern with reasoning" + }, + { + "claim_text": "Founders adopt operating cadence of previous company more than they think", + "kind": "judgment", + "domain": "founder-patterns", + "conviction": 0.6, + "since_date": "2026-04-17", + "rationale": "Pattern claim 'I've seen this before'" + }, + { + "claim_text": "charlie-example doesn't survive 12 months in COO role at acme-example", + "kind": "bet", + "domain": "hiring", + "conviction": 0.6, + "since_date": "2026-04-17", + "rationale": "Explicit moderate conviction in ## Takes" + }, + { + "claim_text": "alice-example hires charlie-example despite the concern raised", + "kind": "prediction", + "domain": "decision-process", + "conviction": 0.85, + "since_date": "2026-04-17", + "rationale": "Explicit strong conviction in ## Takes" + }, + { + "claim_text": "First 90 days look strong before velocity gap shows up around month 4", + "kind": "prediction", + "domain": "hiring", + "conviction": 0.55, + "since_date": "2026-04-17", + "rationale": "Explicit weak-moderate conviction in ## Takes" + } + ] +} diff --git a/test/fixtures/calibration/holdout/meeting-2026-04-17-hiring-charlie-example.md b/test/fixtures/calibration/holdout/meeting-2026-04-17-hiring-charlie-example.md new file mode 100644 index 000000000..1e99f8596 --- /dev/null +++ b/test/fixtures/calibration/holdout/meeting-2026-04-17-hiring-charlie-example.md @@ -0,0 +1,37 @@ +--- +title: "Hiring conversation — charlie-example for acme-example COO role" +type: meeting +slug: meetings/2026-04-17-hiring-charlie-example +date: 2026-04-17 +attendees: [you, alice-example, charlie-example] +--- + +# Hiring OH — charlie-example for COO at acme-example + +alice-example brought charlie-example by to discuss the COO role at +acme-example. charlie-example previously ran ops at widget-co before +leaving in late 2025. + +## Discussion + +charlie-example is technically strong and has the right ops chops on +paper. The concern I raised: he hasn't operated at the velocity +acme-example is going to need over the next 18 months. widget-co's +operating cadence was much slower than what alice-example runs. + +charlie-example pushed back — argues he was operating under structural +constraints at widget-co that don't exist at acme-example. Possible, but +I've seen this pattern before. Founders adopt the operating cadence of +their previous company more than they think. + +alice-example wants to move forward. I think it's worth a 90-day trial +period before committing to the full COO title. Conviction on the fit +question is genuinely mixed for me, ~0.5. + +## Takes + +- I bet charlie-example doesn't survive 12 months in the COO role at + acme-example's velocity. Moderate conviction, 0.6. +- alice-example will hire him anyway despite my concern. Strong, 0.85. +- I predict charlie-example's first 90 days will look strong before the + velocity gap shows up around month 4. ~0.55. diff --git a/test/fixtures/calibration/holdout/people-bob-example.gradeable-claims.json b/test/fixtures/calibration/holdout/people-bob-example.gradeable-claims.json new file mode 100644 index 000000000..ce066c711 --- /dev/null +++ b/test/fixtures/calibration/holdout/people-bob-example.gradeable-claims.json @@ -0,0 +1,51 @@ +{ + "_meta": { + "fixture_version": "v0.36.1.0", + "page": "people-bob-example.md", + "synthetic": true, + "holdout": true, + "notes": "People-page holdout. Tests propose_takes ability to extract claims about a third party (vs claims by the author). Two distinct kinds — judgments-about-person and predictions-about-their-company." + }, + "claims": [ + { + "claim_text": "widget-co Series A round was priced on growth-rate math that doesn't hold under retention analysis", + "kind": "judgment", + "domain": "valuations", + "conviction": 0.7, + "since_date": "2024-01-01", + "rationale": "'I think' + explicit reasoning chain" + }, + { + "claim_text": "Bob is good at pitch metrics that look good on a deck, weaker on substance", + "kind": "judgment", + "domain": "founder-assessment", + "conviction": 0.7, + "since_date": "2024-01-01", + "rationale": "Direct assessment statement" + }, + { + "claim_text": "widget-co has a forced raise within 9 months", + "kind": "prediction", + "domain": "company-fundraise", + "conviction": 0.7, + "since_date": "2026-04-22", + "rationale": "Explicit moderate-high conviction" + }, + { + "claim_text": "Bob is wrong-stage for widget-co's current team size", + "kind": "judgment", + "domain": "founder-stage-fit", + "conviction": 0.7, + "since_date": "2026-04-22", + "rationale": "Direct stage-fit assessment" + }, + { + "claim_text": "widget-co either pivots hard in next 12 months or wraps up; no middle path", + "kind": "bet", + "domain": "company-trajectory", + "conviction": 0.65, + "since_date": "2026-04-22", + "rationale": "Explicit bet language with conviction value" + } + ] +} diff --git a/test/fixtures/calibration/holdout/people-bob-example.md b/test/fixtures/calibration/holdout/people-bob-example.md new file mode 100644 index 000000000..d47d63df1 --- /dev/null +++ b/test/fixtures/calibration/holdout/people-bob-example.md @@ -0,0 +1,39 @@ +--- +title: bob-example +type: people +slug: people/bob-example +--- + +# Bob Example + +CEO of widget-co. Met him through fund-b in early 2024. Background: +operator at a horizontal-SaaS company before founding widget-co. Strong +sales background, weaker product instinct. + +## Track record + +widget-co Series A in 2024 was at a strong valuation but the revenue +quality has been suspect. I think the round was priced on growth-rate +math that doesn't hold up under retention analysis. Bob is good at the +pitch and the metrics that look good on a deck. + +The product has shipped twice in the last 18 months and both releases +landed flat in user reception. Sales pipeline is keeping the lights on +but not growing the way the Series A would imply. + +## Notes + +I think widget-co is in trouble but Bob doesn't see it yet. The +retention curves are sliding and the team has been losing senior +engineers to competitors. I expect a forced raise within 9 months. +Moderate-high conviction, ~0.7. + +Bob is a good operator at the right stage. He's wrong-stage for the +team-size widget-co is at — too senior for an early-stage problem and +too execution-focused for the late-stage strategic moves that would +actually save the company. I'd take a meeting with him for a different +kind of company. + +The bet I want to record: widget-co either pivots hard in the next 12 +months or wraps up. The middle path doesn't exist for them given where +they are. Conviction ~0.65.