mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
f25b674fbf89dcfee2f188fedf8e5517c96b59ef
6
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Garry TanandClaude Opus 4.7
|
f25b674fbf |
Merge origin/master into garrytan/managua-v3
Resolves v0.34.0.0 (W1-W8 code intelligence) with master's v0.33.2.1 + search-lite work (query cache + intent weighting + token budget + drift watch + metric glossary + search modes). Conflict resolutions: - VERSION / package.json: kept 0.34.0.0 (mine; higher than master's 0.33.2.1) - CHANGELOG.md: both entries preserved; reordered so v0.33.2.1 sits above v0.33.2.0 (semver order) - src/cli.ts CLI_ONLY: union of both — `edges-backfill` (mine) + `cache` (master) - src/core/migrate.ts: renumbered my migrations to avoid collision with master's query_cache_search_lite (v55), query_cache_knobs_hash (v56), search_telemetry_rollup (v57). My `edges_backfilled_at_v0_33_2` moves v55 → v58; my `code_traversal_cache_v0_34` moves v56 → v59. Code refs in `src/core/code-intel/traversal-cache.ts` and the paired test updated to match. - src/core/operations.ts query op: kept master's `hybridSearchCached` routing (search-lite cache integration) AND my `sourceId` resolution block (D4 source-routing fix from v0.34 STEP 0). Both apply. Verification: - `bun run typecheck` clean - `bun run verify` clean (includes check-cli-executable, check-jsonb, check-system-of-record, check-eval-glossary-fresh, etc.) - Migration v50→v59 apply cleanly on PGLite in isolated test runs - Individual test files pass (e.g. test/search-lang-symbol-kind.test.ts: 9 pass / 0 fail in 913ms) Known follow-up: the parallel test shard runner times out some beforeAll hooks at the default 7s budget. Tests pass when run sequentially (`--max-concurrency=1`); 27/0 confirmed across 3 sample files in 2.4s sequential vs timeouts under parallel-shard contention. Master added 4 new migrations (v55-v57 + search-lite related) increasing per-test-file PGLite init cost; on 8 shards racing for OS resources, some shards hit the 7s ceiling. This is a test-infrastructure issue (shard isolation under heavier migrations), not a code-correctness issue. Fix is a follow-up: either raise shard test timeout, reduce shard count, or migrate to fixture-based engine setup for hot tests. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Garry TanandClaude Opus 4.7
|
73eaed7691 |
feat(v0.34 W7): per-op graph-traversal metrics module
src/core/eval-capture-graph.ts — pure-function metrics module for comparing code_blast / code_flow / code_cluster_get result shapes across two runs (eval-replay's regression check). Per Codex finding #3 from the plan-review: page-slug Jaccard is the wrong metric for graph traversal. v0.34 W7 ships proper per-op metrics: - nodeSetJaccard(a, b): set Jaccard over (file, line, symbol) tuples. Right metric for code_blast/code_flow node sets. - depthGroupStability(a, b): 1 - (displaced / |union|). Catches the case where node membership is identical but nodes moved between depth buckets between runs. - truncationMatch(a, b): boolean match on the truncation enum. Discrete signal that pairs with Jaccard. - adjustedRandIndex(a, b): cluster-membership stability via ARI for code_cluster_get. v0.34.1 consumer; lands in W7 alongside the rest so the cluster-replay path is ready when clusters ship. - compareCodeWalk(a, b): convenience wrapper returning {jaccard, depth_stability, truncation_match} in one call. Hermetic — no engine, no DB, fully unit-testable. 20 test cases covering identical / disjoint / partial-overlap / empty / dedup / file+line-distinguished, depth-bucket reshuffles, truncation-enum matching, ARI identical-clustering recognition through label-rename, ARI singleton-vs-all-one expected-zero, equal-length contract, and combined compareCodeWalk envelope. Scope reduction from the original plan: extending src/core/eval-capture.ts capture wrapper with `tool` field + `result_shape` payload, and extending src/commands/eval-replay.ts to dispatch on tool — both deferred to v0.34.1. The metric MODULE is the load-bearing piece (Codex finding #3's primary fix); wiring it through the existing capture/replay surface is a follow-up that doesn't change production behavior until clusters ship. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Garry TanandClaude Opus 4.7
|
bfe204d871 |
feat(v0.34 W3): code_blast + code_flow recursive ops + sinks
Recursive caller (code_blast) + recursive callee (code_flow) walks land
as first-class MCP ops. The user-facing payoff for v0.34: v0.33.3
shipped flat callers/callees; v0.34 ships depth-grouped recursive walks
with cycle detection, truncation flags, freshness reporting, sink
tagging on terminal nodes, and bare-name disambiguation with
did_you_mean suggestions.
- src/core/code-intel/recursive-walk.ts: BFS over existing engine
single-hop methods (getCallersOf, getCalleesOf). Depth-grouped output;
confidence = clamp(1 / (1 + 0.3 * depth), 0.05, 1.0). Cycle detection
via visited-set; truncation enum captures both depth_cap and max_nodes
exhaustion. Source-scoped per D4 sourceId REQUIRED.
- src/core/code-intel/sinks/{ts,py,index}.ts: per-language sink patterns
as TypeScript constants (D9 — auditable literal-string + glob; NOT
regex). Pattern cache hits warm after first match per process.
TS_SINKS covers fetch, axios.*, fs.*, Bun.*, execSync, spawnSync;
PY_SINKS covers requests.*, urllib.*, subprocess.*, open, pathlib.*.
- src/core/operations.ts: code_blast + code_flow registered with
scope: 'read'. Both wrap their walks through
getCachedOrCompute (W3b) so repeat blasts in a plan-mode session hit
cache. depth + max_nodes hard-capped at handler entry per design doc
Constraints. exact: true skips bare-name disambiguation.
Response envelope (shared):
{ result: 'ok' | 'not_found' | 'ambiguous' | 'unsupported_language',
depth_groups?, cycles_detected?, truncation?, freshness?,
did_you_mean?, candidates?, supported? }
code_flow adds: terminal_nodes: [{symbol, sink_kind}] where sink_kind ∈
'db_call' | 'http_call' | 'file_io' | 'process_exec' | 'unknown'
Per D18 from eng review — only JS/TS/TSX + Python get walks. Other
languages return {result: 'unsupported_language', supported: ['ts',
'tsx','js','py']} cleanly rather than aliasing same-named callees.
test/code-intel/recursive-walk.test.ts: 11 hermetic PGLite tests:
- 7 sinks classifier cases (http_call, file_io, db_call, process_exec
for TS + Python, unknown for made-up symbol, unknown for ruby lang)
- not_found returns did_you_mean
- happy-path: caller chain emerges in depth_groups; confidence ~0.77
at depth 1
- truncation: depth_cap fires when walk exceeds depth
- sink-tagging: fetch lands in terminal_nodes with http_call kind
v0.34.0.0 scope reductions: stdio rate limiter at dispatch.ts and CLI
wrappers (gbrain blast / gbrain flow) deferred — the ops are MCP-
reachable today and the W8 release packaging step adds CLI thin-shims.
The eng-review's stdio limiter at dispatch.ts (D10) is queued behind
the eval gate run; concurrent code-intel load needed to justify it
hasn't materialized at v0.34.0.0 ship time.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Garry TanandClaude Opus 4.7
|
0c2b1da72c |
feat(v0.34 W3b): code_traversal_cache table, module, and clear admin op
Schema migration v56 (code_traversal_cache_v0_34):
- new table: code_traversal_cache (id, symbol_qualified, depth,
source_id, response_json JSONB, max_chunk_updated_at, xmin_max,
cluster_generation, computed_at)
- unique index on (symbol_qualified, depth, source_id)
- secondary index on source_id for cheap source-scoped clears
D3 — generation-counter cache invalidation. cluster_generation is a
BIGINT column on every cache row; bumped once per recompute_code_clusters
phase via bumpClusterGeneration(). Cache rows referencing stale
generations naturally miss on read. Eliminates the bug class where
cluster recompute leaves stale cache entries that reference dropped or
renamed clusters.
D8 — destructive-guard parity. clearTraversalCache requires either
source_id OR all_sources=true. Without either it throws. Mirrors v0.26.5
destructive-guard pattern; the MCP op (code_traversal_cache_clear,
scope: admin, localOnly: true) inherits the gate.
- src/core/code-intel/traversal-cache.ts: cache module with public API
- getClusterGeneration / bumpClusterGeneration (config-backed counter)
- getCachedTraversal / putCachedTraversal (low-level read/write)
- getCachedOrCompute (try-cache-then-compute wrapper for W3 ops)
- clearTraversalCache (admin clear with source-scope gate)
- src/core/operations.ts: code_traversal_cache_clear op registered with
scope: 'admin' + localOnly: true. Dry-run aware; resolves source_id
from params or ctx.
v0.34.0.0 scope: cache writes use xmin_max=0 sentinel (no snapshot
isolation). REPEATABLE READ + xmin_max snapshot isolation + PGLite
serialization_failure retry is wired in the module but disabled by
default; v0.34.1 enables it once W3 ops produce enough load to justify
the correctness gain. Under low-write workloads (the common case for an
agent's plan-mode session, 5-15 blast calls without concurrent sync),
the cache stays correctness-safe via the cluster_generation invalidation
+ the natural UPSERT on conflict.
test/code-intel/traversal-cache.test.ts: 13 hermetic PGLite tests
covering cache hit/miss, D3 generation-counter invalidation, UPSERT
replacement, source-scoped + all-sources clear paths, and getCachedOrCompute
try-cache-then-compute happy path.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Garry TanandClaude Opus 4.7
|
1318c63752 |
feat(v0.34 W2): edge densification — imports + references edge types
Edge extractor now emits three edge kinds:
- calls (v0.20 baseline; v0.34 W1 added qualified-name receiver
resolution for JS/TS/TSX + Python)
- imports (NEW in v0.34 W2; JS/TS/TSX + Python at depth)
- references (NEW in v0.34 W2; TS-only)
Why this matters: Leiden clusters on a calls-only graph produce overfit
garbage (GitNexus showed 0.052 cluster/node on calls-only — useless).
Adding imports + references densifies the graph so W4-5's clusters can
land meaningful communities. Per design doc Constraint #1.
- src/core/chunkers/edge-extractor.ts: new extractImportEdges and
extractReferenceEdges functions + combined extractAllEdges wrapper.
ExtractedEdge.edgeType widened to 'calls' | 'imports' | 'references'.
- src/core/chunkers/code.ts: switched the chunker's edge-extraction call
site from extractCallEdges to extractAllEdges so imports + references
flow into code_edges_symbol alongside calls.
- src/core/chunkers/symbol-resolver.ts: EDGE_EXTRACTOR_VERSION_TS bumped
to 2026-05-14T01:00:00Z so the next dream cycle re-walks every chunk.
Language scope per D18 from eng review:
- JS/TS/TSX: imports + references emitted
- Python: imports emitted, references skipped (Python type hints too
sparse for v0.34; v0.35 may revisit)
- Ruby/Go/Rust/Java: calls only — no imports, no references. Honest
coverage matrix; code_blast/code_flow return 'unsupported_language'
response for these langs (W2 commit 4 wires this).
Edge schema reused: code_edges_symbol.edge_type is the existing TEXT
column populated by the unique constraint
(from_chunk_id, to_symbol_qualified, edge_type). Adding new types
doesn't conflict with existing calls edges.
test/code-intel/edge-densification.test.ts: 13 hermetic tests covering
named/default/namespace/aliased/side-effect imports for JS/TS, from-x-
import-y + import-pkg for Python, function parameter + return type
references for TS, and unsupported-language returns-empty contract.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Garry TanandClaude Opus 4.7
|
4947a2ac8a |
feat(v0.34 W1): receiver-type resolution at edge-extraction time
The edge-extractor emits qualified callee names (Class::method,
module::method) for the 3 MUST-resolve patterns from the design doc
when running against JS/TS/TSX + Python source:
1. `import { x } from 'y'; x.method()` → emit `y::method`
2. `class C { m() { this.m() } }` → emit `C::m`
3. `const c = new C(); c.m()` → emit `C::m`
When the receiver can't be resolved within WALK_DEPTH_CAP (32) ancestor
hops of the call site, falls back to bare-token emit (pre-W1 behavior).
Ambiguous-but-named-correctly beats wrong-but-confident; the symbol
resolver's second pass still gets a chance to disambiguate via same-page
symbol_name_qualified lookups.
Per D18 from eng review — only JS/TS/TSX + Python get receiver
resolution. Ruby/Go/Rust/Java keep pre-W1 bare-token emit semantics.
RECEIVER_RESOLUTION_LANGS pins the eligible set.
Per D12 from eng review — WALK_DEPTH_CAP=32 covers any realistic code
shape; JSX-in-JSX or closure chains rarely exceed depth-20. The cap
prevents one pathological file from multiplying cycle cost across the
whole brain on every dream run.
- src/core/chunkers/edge-extractor.ts: new `resolveReceiverType` helper
+ WALK_DEPTH_CAP export + RECEIVER_RESOLUTION_LANGS set. extractCallEdges
attempts resolution on every member-call emit; falls back on miss.
- src/core/chunkers/symbol-resolver.ts: EDGE_EXTRACTOR_VERSION_TS bumped
to 2026-05-14 so the next dream cycle re-walks every chunk and lets
the resolver pick up qualified-name matches.
test/code-intel/scope-walker-resolution.test.ts: 10 hermetic snapshot
tests covering all 3 MUST patterns + bare-call fallback + unresolvable
member call. Tests load tree-sitter WASMs on demand and short-circuit
when grammars are unavailable in the test runtime.
Scope reduction from the original plan: the .scm pattern-file
architecture envisioned by the design doc is deferred to v0.34.1. The
codebase doesn't use tree-sitter's Query API anywhere today; introducing
it across chunkers/scope/patterns/* is a multi-day investment that
duplicates the manual-AST-walker idiom edge-extractor.ts already uses.
This commit ships the same functional outcome (qualified names for the
3 MUST patterns + depth cap + honest language scope) via the existing
idiom; v0.34.1 can refactor to .scm files if/when query-API benefits
materialize.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|