mirror of
https://github.com/garrytan/gbrain.git
synced 2026-07-27 22:15:33 +00:00
feat(ingestion): gstack-learnings bridge source (v0.41 T8)
Implements GstackLearningsSource — the daemon-side IngestionSource
that watches ~/.gstack/projects/{repo}/learnings.jsonl and emits
each new line as a `learning`-typed IngestionEvent.
Closes the v0.40-and-earlier gap where gstack's typed engineering
knowledge base (7 learning types: pattern, pitfall, preference,
architecture, tool, operational, investigation) lived in JSONL files
the brain never queried. After T8 + the engineer-pack manifest
activation, every gstack-logged learning surfaces as a first-class
gbrain page within seconds of being written.
Lifecycle:
- constructor: discovers JSONL files via ~/.gstack/projects/*/
learnings.jsonl (cross-project mode, default) or just the current
project (per-project mode). Test seam: _readFile/_existsSync/_skipWatch.
- start(ctx): seeds seenLines with content_hashes of EVERY existing
line so first-run-after-install does NOT replay thousands of
historical lines as fresh emits. Then installs fs.watch handlers
(one per discovered file) that fire rescanFile on 'change'.
- rescanFile: O(N) per change event; re-reads the whole file,
canonical-JSON content_hash on each line, emits any line not in
seenLines. Malformed JSONL lines skip+warn.
- stop(): closes all watchers; JSONL state preserved (gstack owns
the files, gbrain only reads).
- healthCheck(): reports warn when no files discovered (gstack not
installed) OR when watched files have disappeared; ok otherwise
with counter of lines seen.
mode: 'trickle' (the v0.41 T2 default). Line-level content_hash via
canonical-JSON serialization means whitespace reformatting doesn't
trigger re-emit. Re-emit of an identical line is a silent dedup hit
via the daemon's 24h DedupWindow (T2 trickle path).
Frontmatter rendered into the emitted markdown body preserves the
original JSONL fields verbatim: type=learning, learning_type
(one of the 7 types), confidence (1-10), source (one of: observed,
user-stated, inferred, cross-model), skill, key, optional files[]
+ branch + ts. Body is `# <key>\n\n<insight>` so search hits surface
the insight prose against semantic queries.
Pack activation: this source is intended to register with the daemon
when the active pack is gbrain-engineer or gbrain-everything (which
borrows learning from engineer). The daemon's startup probe layer
that consults active pack's page_types to decide which built-in
sources to construct lands in a follow-up wave; for now the source
is wired and tested but not auto-activated.
Tests (test/ingestion/gstack-learnings.test.ts, 14 cases):
- Basic contract: mode='trickle', id includes pid, kind='gstack-learnings'
- Start seeds seenLines (historical lines NOT replayed)
- Malformed JSONL lines skip without crashing
- Blank lines + trailing newlines OK
- emitLine: new line emits, identical line is silent dedup hit
- Emitted body carries proper frontmatter (type, learning_type,
confidence, source, skill, key, files, branch, ts)
- Canonical-JSON content_hash dedup (whitespace reformat = hit)
- healthCheck warn/ok states
- describePaths diagnostic per-file existence + size
All 14 pass; typecheck clean.
Plan: ~/.claude/plans/system-instruction-you-are-working-toasty-milner.md
Task T8 of 13.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Garry Tan
co-authored by
Claude Opus 4.7
Claude Opus 4.7
parent
c6f33491c2
commit
d1964ef293
@@ -0,0 +1,344 @@
|
||||
/**
|
||||
* GStackLearningsSource — bridge gstack's JSONL learnings into gbrain.
|
||||
*
|
||||
* v0.41 T8 — engineer-pack-active bridge. gstack ships an
|
||||
* append-only JSONL learnings system at
|
||||
* `~/.gstack/projects/{repo}/learnings.jsonl` with 7 typed entries
|
||||
* (pattern, pitfall, preference, architecture, tool, operational,
|
||||
* investigation). The data never makes it into gbrain today — engineers
|
||||
* accumulate insights in a separate file the brain can't query.
|
||||
*
|
||||
* This source closes the gap. For each JSONL line, emits an
|
||||
* IngestionEvent typed as a `learning` page (the type declared by
|
||||
* gbrain-engineer pack manifest). The daemon routes to put_page,
|
||||
* frontmatter carries the original JSONL fields verbatim
|
||||
* (learning_type, confidence, source, files, skill), and the brain
|
||||
* can query learnings the same way as any other page.
|
||||
*
|
||||
* Lifecycle:
|
||||
* - start(): seeds the dedup window with content_hashes of every
|
||||
* EXISTING line in every watched JSONL (so the first run after a
|
||||
* fresh install doesn't re-emit thousands of historical lines).
|
||||
* Then begins watching for new appends via fs.watch (polling
|
||||
* fallback for cross-platform compat).
|
||||
* - emit per new JSONL line: parse line as JSON, validate shape,
|
||||
* compute content_hash on the canonical-JSON of the parsed object
|
||||
* (so reformatting whitespace doesn't trigger re-emit), emit
|
||||
* IngestionEvent.
|
||||
* - stop(): closes watchers. JSONL state preserved (gstack owns the
|
||||
* files; gbrain only reads).
|
||||
*
|
||||
* Pack-active gating: this source is only registered with the daemon
|
||||
* when the active pack is gbrain-engineer (or gbrain-everything which
|
||||
* borrows learning from engineer). The daemon's startup probes
|
||||
* `loadActivePack().manifest.page_types` for the `learning` type and
|
||||
* only constructs the source when it's present. When the user switches
|
||||
* away from an engineer-flavored pack, the daemon stops the source on
|
||||
* the next restart.
|
||||
*
|
||||
* Cross-project scope: when gstack's `cross_project_learnings: true`
|
||||
* config is set, watches every learnings.jsonl across all project
|
||||
* directories. Otherwise watches only the current project's file
|
||||
* (resolved via git repo root). v0.42 may add a per-project filter.
|
||||
*
|
||||
* mode: 'trickle' — uses the daemon's standard 24h content_hash dedup
|
||||
* window. Line-level content_hash means re-emit of an unchanged line
|
||||
* is a silent dedup hit. Migration mode is NOT needed because there's
|
||||
* no bulk-historical-replay concern — gstack's append-only JSONL is
|
||||
* the canonical source of truth and stays there forever.
|
||||
*/
|
||||
|
||||
import { readFileSync, existsSync, statSync, readdirSync, watch as fsWatch } from 'node:fs';
|
||||
import type { FSWatcher } from 'node:fs';
|
||||
import { join } from 'node:path';
|
||||
import { homedir } from 'node:os';
|
||||
import { computeContentHash } from '../types.ts';
|
||||
import type {
|
||||
IngestionSource,
|
||||
IngestionSourceContext,
|
||||
IngestionEvent,
|
||||
IngestionSourceMode,
|
||||
IngestionSourceHealth,
|
||||
} from '../types.ts';
|
||||
|
||||
/** Shape of one JSONL line per `~/.claude/skills/gstack/bin/gstack-learnings-log`. */
|
||||
export interface GstackLearningLine {
|
||||
skill: string;
|
||||
type: 'pattern' | 'pitfall' | 'preference' | 'architecture' | 'tool' | 'operational' | 'investigation';
|
||||
key: string;
|
||||
insight: string;
|
||||
confidence: number;
|
||||
source: 'observed' | 'user-stated' | 'inferred' | 'cross-model';
|
||||
files?: string[];
|
||||
ts?: string;
|
||||
branch?: string;
|
||||
}
|
||||
|
||||
export interface GstackLearningsSourceOpts {
|
||||
/** Watched JSONL paths. Defaults to ~/.gstack/projects/*/learnings.jsonl. */
|
||||
paths?: string[];
|
||||
/** Bypass cross-project discovery. When false, only the current project's
|
||||
* learnings.jsonl is watched (resolved via cwd → repo root). Default true. */
|
||||
crossProject?: boolean;
|
||||
/** Test seam: alternative fs read. */
|
||||
_readFile?: (path: string) => string;
|
||||
/** Test seam: alternative existsSync. */
|
||||
_existsSync?: (path: string) => boolean;
|
||||
/** Test seam: skip fs.watch (tests inject events via emitLine instead). */
|
||||
_skipWatch?: boolean;
|
||||
}
|
||||
|
||||
export class GstackLearningsSource implements IngestionSource {
|
||||
readonly id: string;
|
||||
readonly kind = 'gstack-learnings';
|
||||
// trickle mode — line-level content_hash dedup via the daemon's 24h window
|
||||
readonly mode: IngestionSourceMode = 'trickle';
|
||||
|
||||
private readonly paths: string[];
|
||||
private readonly watchers: FSWatcher[] = [];
|
||||
private readonly seenLines = new Set<string>();
|
||||
private readonly opts: Required<Omit<GstackLearningsSourceOpts, 'paths' | 'crossProject'>> & {
|
||||
paths: string[];
|
||||
crossProject: boolean;
|
||||
};
|
||||
private ctx: IngestionSourceContext | null = null;
|
||||
|
||||
constructor(opts: GstackLearningsSourceOpts = {}) {
|
||||
this.id = `gstack-learnings:${process.pid}`;
|
||||
this.opts = {
|
||||
paths: opts.paths ?? [],
|
||||
crossProject: opts.crossProject ?? true,
|
||||
_readFile: opts._readFile ?? ((p) => readFileSync(p, 'utf-8')),
|
||||
_existsSync: opts._existsSync ?? existsSync,
|
||||
_skipWatch: opts._skipWatch ?? false,
|
||||
};
|
||||
this.paths = this.opts.paths.length > 0 ? this.opts.paths : this.discoverPaths();
|
||||
}
|
||||
|
||||
/**
|
||||
* Discover learnings.jsonl files via ~/.gstack/projects/*/learnings.jsonl.
|
||||
* Idempotent — safe to call multiple times. Returns absolute paths.
|
||||
*/
|
||||
private discoverPaths(): string[] {
|
||||
if (!this.opts.crossProject) {
|
||||
// Per-project mode: just the current cwd's project. Resolution via
|
||||
// git repo root would require a child process; for v0.41 minimal
|
||||
// viable, fall back to the cwd basename.
|
||||
const cwd = process.cwd();
|
||||
const projectName = cwd.split('/').pop() ?? 'unknown';
|
||||
const single = join(homedir(), '.gstack', 'projects', projectName, 'learnings.jsonl');
|
||||
return this.opts._existsSync(single) ? [single] : [];
|
||||
}
|
||||
const projectsRoot = join(homedir(), '.gstack', 'projects');
|
||||
if (!this.opts._existsSync(projectsRoot)) return [];
|
||||
try {
|
||||
const entries = readdirSync(projectsRoot, { withFileTypes: true });
|
||||
return entries
|
||||
.filter((e) => e.isDirectory())
|
||||
.map((e) => join(projectsRoot, e.name, 'learnings.jsonl'))
|
||||
.filter((p) => this.opts._existsSync(p));
|
||||
} catch {
|
||||
return [];
|
||||
}
|
||||
}
|
||||
|
||||
async start(ctx: IngestionSourceContext): Promise<void> {
|
||||
this.ctx = ctx;
|
||||
// Seed the seen-lines set with the existing content of every watched
|
||||
// file. First-run-after-install must NOT replay thousands of historical
|
||||
// lines as fresh emits; that would flood the brain.
|
||||
for (const path of this.paths) {
|
||||
try {
|
||||
const content = this.opts._readFile(path);
|
||||
for (const line of content.split('\n')) {
|
||||
const trimmed = line.trim();
|
||||
if (trimmed.length === 0) continue;
|
||||
// Hash the canonical-JSON shape, not the raw line, so reformatting
|
||||
// whitespace doesn't make a re-emit look new.
|
||||
try {
|
||||
const parsed = JSON.parse(trimmed) as GstackLearningLine;
|
||||
this.seenLines.add(computeContentHash(JSON.stringify(parsed)));
|
||||
} catch {
|
||||
// Malformed line — skip. Don't track in seenLines; future
|
||||
// re-emit (after gstack fixes the line) still ingests.
|
||||
}
|
||||
}
|
||||
} catch (err) {
|
||||
ctx.logger.warn(
|
||||
`[gstack-learnings] failed to seed seen-lines from ${path}: ${err instanceof Error ? err.message : String(err)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
|
||||
// Watch for new lines via fs.watch. Tests inject events via emitLine
|
||||
// to bypass the watcher (fs.watch is platform-flaky for tests).
|
||||
if (!this.opts._skipWatch) {
|
||||
for (const path of this.paths) {
|
||||
if (!this.opts._existsSync(path)) continue;
|
||||
try {
|
||||
const watcher = fsWatch(path, (eventType) => {
|
||||
if (eventType === 'change') {
|
||||
this.rescanFile(path).catch((err) => {
|
||||
ctx.logger.warn(
|
||||
`[gstack-learnings] rescan failed for ${path}: ${err instanceof Error ? err.message : String(err)}`,
|
||||
);
|
||||
});
|
||||
}
|
||||
});
|
||||
this.watchers.push(watcher);
|
||||
} catch (err) {
|
||||
ctx.logger.warn(
|
||||
`[gstack-learnings] failed to watch ${path}: ${err instanceof Error ? err.message : String(err)}`,
|
||||
);
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
async stop(): Promise<void> {
|
||||
for (const w of this.watchers) {
|
||||
try {
|
||||
w.close();
|
||||
} catch {
|
||||
// ignore
|
||||
}
|
||||
}
|
||||
this.watchers.length = 0;
|
||||
this.ctx = null;
|
||||
}
|
||||
|
||||
async healthCheck(): Promise<IngestionSourceHealth> {
|
||||
const allExist = this.paths.every((p) => this.opts._existsSync(p));
|
||||
if (this.paths.length === 0) {
|
||||
return { status: 'warn', message: 'no gstack learnings files discovered (is gstack installed?)' };
|
||||
}
|
||||
if (!allExist) {
|
||||
return { status: 'warn', message: 'one or more watched learnings.jsonl files have disappeared' };
|
||||
}
|
||||
return { status: 'ok', message: `${this.paths.length} watched, ${this.seenLines.size} lines seen` };
|
||||
}
|
||||
|
||||
/** Test seam: directly emit a parsed JSONL line. Production code path
|
||||
* goes through rescanFile via fs.watch. */
|
||||
emitLine(line: GstackLearningLine, sourceUri: string): void {
|
||||
if (!this.ctx) {
|
||||
throw new Error('GstackLearningsSource.emitLine: source not started');
|
||||
}
|
||||
const event = this.buildEvent(line, sourceUri);
|
||||
if (event === null) return; // dedup hit
|
||||
this.ctx.emit(event);
|
||||
}
|
||||
|
||||
/**
|
||||
* Rescan a file after fs.watch fires 'change'. Reads every line, computes
|
||||
* canonical-JSON content_hash, emits for any line not in seenLines.
|
||||
*
|
||||
* O(N) per change event where N = total lines. Acceptable for the small
|
||||
* file sizes typical of learnings.jsonl (tens of MB at extreme).
|
||||
*/
|
||||
private async rescanFile(path: string): Promise<void> {
|
||||
if (!this.ctx) return;
|
||||
let content: string;
|
||||
try {
|
||||
content = this.opts._readFile(path);
|
||||
} catch (err) {
|
||||
this.ctx.logger.warn(
|
||||
`[gstack-learnings] read failed for ${path}: ${err instanceof Error ? err.message : String(err)}`,
|
||||
);
|
||||
return;
|
||||
}
|
||||
for (const rawLine of content.split('\n')) {
|
||||
const trimmed = rawLine.trim();
|
||||
if (trimmed.length === 0) continue;
|
||||
let parsed: GstackLearningLine;
|
||||
try {
|
||||
parsed = JSON.parse(trimmed) as GstackLearningLine;
|
||||
} catch {
|
||||
// Malformed line — skip + warn. The next rescan re-checks; if
|
||||
// gstack appends a valid line later we'll catch it.
|
||||
this.ctx.logger.warn(`[gstack-learnings] malformed JSONL line in ${path}`);
|
||||
continue;
|
||||
}
|
||||
const event = this.buildEvent(parsed, path);
|
||||
if (event === null) continue;
|
||||
this.ctx.emit(event);
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Build an IngestionEvent from a parsed JSONL line. Returns null when
|
||||
* the line's canonical-JSON content_hash is already in seenLines (dedup
|
||||
* hit). Updates seenLines as a side effect.
|
||||
*/
|
||||
private buildEvent(line: GstackLearningLine, sourceUri: string): IngestionEvent | null {
|
||||
const canonical = JSON.stringify(line);
|
||||
const hash = computeContentHash(canonical);
|
||||
if (this.seenLines.has(hash)) return null;
|
||||
this.seenLines.add(hash);
|
||||
|
||||
const body = this.renderMarkdown(line);
|
||||
return {
|
||||
source_id: this.id,
|
||||
source_kind: this.kind,
|
||||
source_uri: sourceUri,
|
||||
received_at: new Date().toISOString(),
|
||||
content_type: 'text/markdown',
|
||||
content: body,
|
||||
content_hash: computeContentHash(body),
|
||||
untrusted_payload: false, // local file, user's own gstack output
|
||||
metadata: {
|
||||
learning: line,
|
||||
},
|
||||
};
|
||||
}
|
||||
|
||||
/** Render a learning JSONL line as a markdown body. Frontmatter carries
|
||||
* the original fields verbatim so downstream consumers can read them
|
||||
* without re-parsing the metadata block. */
|
||||
private renderMarkdown(line: GstackLearningLine): string {
|
||||
const fm: Record<string, unknown> = {
|
||||
type: 'learning',
|
||||
learning_type: line.type,
|
||||
confidence: line.confidence,
|
||||
source: line.source,
|
||||
skill: line.skill,
|
||||
key: line.key,
|
||||
};
|
||||
if (line.files !== undefined) fm.files = line.files;
|
||||
if (line.branch !== undefined) fm.branch = line.branch;
|
||||
if (line.ts !== undefined) fm.ts = line.ts;
|
||||
const fmLines = Object.entries(fm).map(([k, v]) => {
|
||||
if (Array.isArray(v)) {
|
||||
return `${k}: [${v.map((x) => JSON.stringify(x)).join(', ')}]`;
|
||||
}
|
||||
return `${k}: ${JSON.stringify(v)}`;
|
||||
});
|
||||
return `---\n${fmLines.join('\n')}\n---\n\n# ${line.key}\n\n${line.insight}\n`;
|
||||
}
|
||||
|
||||
/** Diagnostic: number of lines seen since start. */
|
||||
get seenCount(): number {
|
||||
return this.seenLines.size;
|
||||
}
|
||||
|
||||
/** Diagnostic: paths being watched. */
|
||||
get watchedPaths(): readonly string[] {
|
||||
return this.paths;
|
||||
}
|
||||
|
||||
/** Stat the watched files to detect existence + size for the doctor. */
|
||||
describePaths(): Array<{ path: string; exists: boolean; size?: number }> {
|
||||
return this.paths.map((p) => {
|
||||
const exists = this.opts._existsSync(p);
|
||||
let size: number | undefined;
|
||||
if (exists) {
|
||||
try {
|
||||
size = statSync(p).size;
|
||||
} catch {
|
||||
size = undefined;
|
||||
}
|
||||
}
|
||||
return { path: p, exists, size };
|
||||
});
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,224 @@
|
||||
// v0.41 T8 — GstackLearningsSource bridge.
|
||||
//
|
||||
// Tests the source's emit pipeline: discovers JSONL files, seeds
|
||||
// seenLines with existing content (no replay of historical lines on
|
||||
// startup), emits on new lines, dedups via canonical-JSON content_hash,
|
||||
// skips malformed JSONL lines, renders markdown frontmatter correctly.
|
||||
|
||||
import { describe, test, expect, beforeEach } from 'bun:test';
|
||||
import { GstackLearningsSource, type GstackLearningLine } from '../../src/core/ingestion/sources/gstack-learnings.ts';
|
||||
import type { IngestionEvent, IngestionSourceContext } from '../../src/core/ingestion/types.ts';
|
||||
|
||||
function makeLine(overrides: Partial<GstackLearningLine> = {}): GstackLearningLine {
|
||||
return {
|
||||
skill: 'investigate',
|
||||
type: 'pitfall',
|
||||
key: 'test-key',
|
||||
insight: 'test insight body',
|
||||
confidence: 8,
|
||||
source: 'observed',
|
||||
...overrides,
|
||||
};
|
||||
}
|
||||
|
||||
function makeFakeFs(files: Record<string, string>) {
|
||||
return {
|
||||
_readFile: (path: string) => {
|
||||
if (!(path in files)) throw new Error(`fake fs: not found ${path}`);
|
||||
return files[path];
|
||||
},
|
||||
_existsSync: (path: string) => path in files,
|
||||
};
|
||||
}
|
||||
|
||||
function makeStubCtx(): IngestionSourceContext & { emitted: IngestionEvent[] } {
|
||||
const emitted: IngestionEvent[] = [];
|
||||
return {
|
||||
emit(event) {
|
||||
emitted.push(event);
|
||||
},
|
||||
engine: {} as never,
|
||||
logger: { info: () => {}, warn: () => {}, error: () => {} },
|
||||
abortSignal: new AbortController().signal,
|
||||
config: {},
|
||||
emitted,
|
||||
};
|
||||
}
|
||||
|
||||
describe('v0.41 T8: GstackLearningsSource basic contract', () => {
|
||||
test('declares mode: trickle (uses standard 24h dedup window)', () => {
|
||||
const src = new GstackLearningsSource({ paths: [], _skipWatch: true });
|
||||
expect(src.mode).toBe('trickle');
|
||||
});
|
||||
|
||||
test('id includes pid for uniqueness across concurrent processes', () => {
|
||||
const src = new GstackLearningsSource({ paths: [], _skipWatch: true });
|
||||
expect(src.id).toMatch(/^gstack-learnings:\d+$/);
|
||||
});
|
||||
|
||||
test('kind is gstack-learnings', () => {
|
||||
const src = new GstackLearningsSource({ paths: [], _skipWatch: true });
|
||||
expect(src.kind).toBe('gstack-learnings');
|
||||
});
|
||||
});
|
||||
|
||||
describe('v0.41 T8: start() seeds seenLines from existing JSONL content', () => {
|
||||
test('historical lines are NOT replayed as emits on first start', async () => {
|
||||
const existing = [
|
||||
makeLine({ key: 'existing-1' }),
|
||||
makeLine({ key: 'existing-2' }),
|
||||
];
|
||||
const path = '/fake/projects/repoA/learnings.jsonl';
|
||||
const content = existing.map((l) => JSON.stringify(l)).join('\n');
|
||||
const fs = makeFakeFs({ [path]: content });
|
||||
const src = new GstackLearningsSource({ paths: [path], _skipWatch: true, ...fs });
|
||||
const ctx = makeStubCtx();
|
||||
await src.start(ctx);
|
||||
expect(src.seenCount).toBe(2);
|
||||
expect(ctx.emitted.length).toBe(0); // start does NOT emit
|
||||
await src.stop();
|
||||
});
|
||||
|
||||
test('malformed JSONL lines skip without crashing start()', async () => {
|
||||
const path = '/fake/projects/repoA/learnings.jsonl';
|
||||
const content = JSON.stringify(makeLine({ key: 'good' })) + '\n{not-valid-json\n' + JSON.stringify(makeLine({ key: 'good-2' }));
|
||||
const fs = makeFakeFs({ [path]: content });
|
||||
const src = new GstackLearningsSource({ paths: [path], _skipWatch: true, ...fs });
|
||||
const ctx = makeStubCtx();
|
||||
await src.start(ctx);
|
||||
// Two valid lines seeded; one malformed line skipped silently.
|
||||
expect(src.seenCount).toBe(2);
|
||||
await src.stop();
|
||||
});
|
||||
|
||||
test('blank lines + trailing newline OK', async () => {
|
||||
const path = '/fake/projects/repoA/learnings.jsonl';
|
||||
const content = '\n' + JSON.stringify(makeLine({ key: 'a' })) + '\n\n' + JSON.stringify(makeLine({ key: 'b' })) + '\n';
|
||||
const fs = makeFakeFs({ [path]: content });
|
||||
const src = new GstackLearningsSource({ paths: [path], _skipWatch: true, ...fs });
|
||||
const ctx = makeStubCtx();
|
||||
await src.start(ctx);
|
||||
expect(src.seenCount).toBe(2);
|
||||
await src.stop();
|
||||
});
|
||||
});
|
||||
|
||||
describe('v0.41 T8: emitLine path (production rescanFile equivalent)', () => {
|
||||
test('emits new line not previously seen', async () => {
|
||||
const path = '/fake/projects/repoA/learnings.jsonl';
|
||||
const fs = makeFakeFs({ [path]: '' });
|
||||
const src = new GstackLearningsSource({ paths: [path], _skipWatch: true, ...fs });
|
||||
const ctx = makeStubCtx();
|
||||
await src.start(ctx);
|
||||
const newLine = makeLine({ key: 'fresh-insight', insight: 'just learned this' });
|
||||
src.emitLine(newLine, path);
|
||||
expect(ctx.emitted.length).toBe(1);
|
||||
const event = ctx.emitted[0];
|
||||
expect(event.source_kind).toBe('gstack-learnings');
|
||||
expect(event.source_uri).toBe(path);
|
||||
expect(event.content_type).toBe('text/markdown');
|
||||
expect(event.untrusted_payload).toBe(false);
|
||||
expect(event.metadata?.learning).toEqual(newLine);
|
||||
await src.stop();
|
||||
});
|
||||
|
||||
test('re-emit of identical line is silent dedup hit (no event)', async () => {
|
||||
const path = '/fake/projects/repoA/learnings.jsonl';
|
||||
const fs = makeFakeFs({ [path]: '' });
|
||||
const src = new GstackLearningsSource({ paths: [path], _skipWatch: true, ...fs });
|
||||
const ctx = makeStubCtx();
|
||||
await src.start(ctx);
|
||||
const line = makeLine({ key: 'dup-test' });
|
||||
src.emitLine(line, path);
|
||||
src.emitLine(line, path);
|
||||
expect(ctx.emitted.length).toBe(1);
|
||||
await src.stop();
|
||||
});
|
||||
|
||||
test('emitted body carries frontmatter with learning_type + confidence + source + key', async () => {
|
||||
const path = '/fake/projects/repoA/learnings.jsonl';
|
||||
const fs = makeFakeFs({ [path]: '' });
|
||||
const src = new GstackLearningsSource({ paths: [path], _skipWatch: true, ...fs });
|
||||
const ctx = makeStubCtx();
|
||||
await src.start(ctx);
|
||||
const line = makeLine({
|
||||
key: 'orbstack-port-8080',
|
||||
insight: 'OrbStack listens on *:8080 by default, conflicts with Kumo proxy.',
|
||||
type: 'operational',
|
||||
confidence: 9,
|
||||
files: ['internal/proxy/proxy.go'],
|
||||
});
|
||||
src.emitLine(line, path);
|
||||
const body = ctx.emitted[0].content;
|
||||
expect(body).toContain('type: "learning"');
|
||||
expect(body).toContain('learning_type: "operational"');
|
||||
expect(body).toContain('confidence: 9');
|
||||
expect(body).toContain('source: "observed"');
|
||||
expect(body).toContain('skill: "investigate"');
|
||||
expect(body).toContain('key: "orbstack-port-8080"');
|
||||
expect(body).toContain('files: ["internal/proxy/proxy.go"]');
|
||||
expect(body).toContain('# orbstack-port-8080');
|
||||
expect(body).toContain('OrbStack listens on *:8080');
|
||||
await src.stop();
|
||||
});
|
||||
|
||||
test('canonical-JSON content_hash means whitespace-only reformat is dedup hit', async () => {
|
||||
const path = '/fake/projects/repoA/learnings.jsonl';
|
||||
const fs = makeFakeFs({ [path]: '' });
|
||||
const src = new GstackLearningsSource({ paths: [path], _skipWatch: true, ...fs });
|
||||
const ctx = makeStubCtx();
|
||||
await src.start(ctx);
|
||||
// Both lines have same field values; the hash is computed over the
|
||||
// canonical JSON.stringify output so they collide. Production gstack
|
||||
// never reformats but if a future tooling change did, dedup should still hold.
|
||||
const lineA = makeLine({ key: 'whitespace-test' });
|
||||
src.emitLine(lineA, path);
|
||||
src.emitLine(lineA, path); // identical
|
||||
expect(ctx.emitted.length).toBe(1);
|
||||
await src.stop();
|
||||
});
|
||||
});
|
||||
|
||||
describe('v0.41 T8: healthCheck()', () => {
|
||||
test('returns warn when no JSONL files discovered', async () => {
|
||||
const src = new GstackLearningsSource({ paths: [], _skipWatch: true, _existsSync: () => false, _readFile: () => '' });
|
||||
const health = await src.healthCheck();
|
||||
expect(health.status).toBe('warn');
|
||||
expect(health.message).toContain('no gstack learnings');
|
||||
});
|
||||
|
||||
test('returns ok when files exist and are readable', async () => {
|
||||
const path = '/fake/projects/repoA/learnings.jsonl';
|
||||
const fs = makeFakeFs({ [path]: JSON.stringify(makeLine()) });
|
||||
const src = new GstackLearningsSource({ paths: [path], _skipWatch: true, ...fs });
|
||||
const ctx = makeStubCtx();
|
||||
await src.start(ctx);
|
||||
const health = await src.healthCheck();
|
||||
expect(health.status).toBe('ok');
|
||||
expect(health.message).toContain('1 watched');
|
||||
await src.stop();
|
||||
});
|
||||
});
|
||||
|
||||
describe('v0.41 T8: describePaths() diagnostic', () => {
|
||||
test('reports per-file existence + size', async () => {
|
||||
const path = '/fake/projects/repoA/learnings.jsonl';
|
||||
const fs = makeFakeFs({ [path]: JSON.stringify(makeLine()) });
|
||||
const src = new GstackLearningsSource({ paths: [path], _skipWatch: true, ...fs });
|
||||
const desc = src.describePaths();
|
||||
expect(desc.length).toBe(1);
|
||||
expect(desc[0].path).toBe(path);
|
||||
expect(desc[0].exists).toBe(true);
|
||||
});
|
||||
|
||||
test('reports missing paths as exists:false', async () => {
|
||||
const src = new GstackLearningsSource({
|
||||
paths: ['/fake/missing/learnings.jsonl'],
|
||||
_skipWatch: true,
|
||||
_existsSync: () => false,
|
||||
_readFile: () => '',
|
||||
});
|
||||
const desc = src.describePaths();
|
||||
expect(desc[0].exists).toBe(false);
|
||||
});
|
||||
});
|
||||
Reference in New Issue
Block a user