mirror of
https://github.com/open-jarvis/OpenJarvis.git
synced 2026-07-27 21:05:34 +00:00
bc9fa6b3c8
Memory tools degraded silently and misleadingly when the mandatory
`openjarvis_rust` extension was absent from the *serving* venv:
- `POST /v1/memory/store` returned HTTP 200 `{"status":"stored","note":
"no backend available"}` and stored nothing (silent data loss).
- `POST /v1/memory/index` returned a generic "No memory backend available",
and the desktop frontend discarded the server `detail` and threw a blanket
"Failed to index path", blaming the path instead of the real cause.
- `GET /v1/memory/config` reported `backend_type: sqlite` even though no
backend could be constructed.
Root cause: `SQLiteMemory.__init__` calls `get_rust_module()` (which raises
ImportError by design — the Rust ext is mandatory, no Python fallback), and
`_get_memory_backend` swallowed that ImportError and returned `None`,
conflating "native extension missing" (a hard install error) with "memory
intentionally disabled" (benign). A chunking floor also silently dropped whole
short documents, and the installer never verified the extension imported from
the serving venv before writing its success marker.
Fix (no fake Python fallback — the Rust ext stays mandatory by design):
- Add `MemoryBackendUnavailable` + `RUST_MISSING_HINT` in tools/storage/_stubs.
`SQLiteMemory.__init__` translates the bridge ImportError into this clear,
actionable error ("run `uv run maturin develop ...`").
- `_get_memory_backend` distinguishes the two cases: a missing native ext
raises HTTP 503 with the actionable hint; a benign unconfigured backend
still returns `None` (graceful path preserved for search/stats).
- `/store` now returns 503 instead of a 200 silent no-op.
- `/config` reports `available: false` + `detail` instead of falsely claiming
a healthy `backend_type`.
- `/index` adds a `note` when `chunks_indexed == 0` so "indexed" never
silently means "stored nothing".
- chunk_text no longer drops an entire short document below `min_chunk_size`
(the floor only discards tiny *trailing* fragments now).
- Frontend `storeMemory`/`indexMemoryPath` surface the server `detail` instead
of blanket strings; `MemoryConfig` gains optional `available`/`detail`.
(Left the pre-existing `backend` vs `backend_type` mismatch untouched.)
- build-extension.sh verifies `import openjarvis_rust` succeeds in the serving
venv before writing the `extension-built` marker.
Regression tests: tests/server/test_api_routes.py::TestMemoryRustMissing mocks
`get_rust_module` to raise ImportError and asserts /store (503, not 200 no-op),
/index (actionable detail, not "Failed to index path"), and /config
(available:false) all surface the clear error; tests/memory/test_chunking.py
asserts short-only docs are kept while tiny trailing fragments are still
filtered.
Fixes #502
Co-authored-by: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
61 lines
2.1 KiB
Bash
Executable File
61 lines
2.1 KiB
Bash
Executable File
#!/usr/bin/env bash
|
|
# build-extension.sh — build the Rust maturin extension into the venv.
|
|
#
|
|
# State files under $OPENJARVIS_HOME/.state/:
|
|
# extension-built — atomic marker written on success
|
|
# extension-failed — written on failure with stderr tail
|
|
# extension-build.log — captured stderr/stdout
|
|
|
|
set -euo pipefail
|
|
|
|
OPENJARVIS_HOME="${OPENJARVIS_HOME:-$HOME/.openjarvis}"
|
|
# Self-heal PATH for cargo: install-rust.sh installs to ~/.cargo/bin, but
|
|
# its export doesn't propagate to subsequent subprocess invocations.
|
|
export PATH="$HOME/.cargo/bin:$PATH"
|
|
SRC_DIR="$OPENJARVIS_HOME/src"
|
|
STATE_DIR="$OPENJARVIS_HOME/.state"
|
|
LOG="$STATE_DIR/extension-build.log"
|
|
BUILT="$STATE_DIR/extension-built"
|
|
FAILED="$STATE_DIR/extension-failed"
|
|
MANIFEST="$SRC_DIR/rust/crates/openjarvis-python/Cargo.toml"
|
|
|
|
mkdir -p "$STATE_DIR"
|
|
|
|
if [[ ! -f "$MANIFEST" ]]; then
|
|
echo "build-extension.sh: manifest not found at $MANIFEST" > "$FAILED"
|
|
exit 1
|
|
fi
|
|
|
|
cd "$SRC_DIR"
|
|
if uv run maturin develop -m "$MANIFEST" >>"$LOG" 2>&1; then
|
|
# Verify the extension actually imports from THIS venv before declaring
|
|
# success. `maturin develop` can report success while installing the .so
|
|
# into a different venv than the one that runs the server, which leaves
|
|
# memory silently broken at runtime (#502). Only the import check below
|
|
# proves the serving venv can load it.
|
|
if ! uv run python -c "import openjarvis_rust" >>"$LOG" 2>&1; then
|
|
rc=$?
|
|
{
|
|
echo "build-extension.sh: maturin succeeded but 'import openjarvis_rust'"
|
|
echo "failed in the serving venv ($SRC_DIR/.venv) — the extension was"
|
|
echo "not installed where the server runs. (exit=$rc)"
|
|
tail -n 50 "$LOG" 2>/dev/null || true
|
|
} > "$FAILED"
|
|
rm -f "$BUILT"
|
|
exit "$rc"
|
|
fi
|
|
tmp="$BUILT.tmp"
|
|
date -u +"%Y-%m-%dT%H:%M:%SZ" > "$tmp"
|
|
mv "$tmp" "$BUILT"
|
|
rm -f "$FAILED"
|
|
exit 0
|
|
else
|
|
rc=$?
|
|
{
|
|
echo "build-extension.sh failed (exit=$rc)"
|
|
tail -n 50 "$LOG" 2>/dev/null || true
|
|
} > "$FAILED"
|
|
rm -f "$BUILT"
|
|
exit "$rc"
|
|
fi
|