Subconscious orchestration layer

The orchestration layer is OpenHuman's split-brain coordinator for wrapped Claude Code / Codex sessions that talk to the owner agent over tiny.place Signal-encrypted DMs. It turns each inbound session DM into one autonomous wake cycle driven by a single tinyagents graph, keeps a durable per-session chat model, and runs an offline subconscious that reflects on how the world is trending and steers later cycles.

Domain root: src/openhuman/orchestration/. Design spec: docs/arch-subconscious.md and the staged plan under docs/plans/subconscious-orchestration/.

End-to-end flow

Claude Code / Codex session
  └─ tinyplace harness wrapper — tails the session JSONL → SessionEnvelopeV1
       └─ Signal E2E DM → owner agent's tiny.place inbox   [tagged sessionId, source, role]
            └─ ingest (decrypt-once → classify → persist → ack)     orchestration/ingest.rs
                 └─ OrchestrationSessionMessage event → debounced wake
                      └─ THE WAKE GRAPH (one tinyagents CompiledGraph)  orchestration/graph/
                           normalize → frontend(1) → execute → compress → world_diff
                                          ▲                                   │
                                          └───────────────────────────────────┘
                                          │
                                          └─(channel_response)─► send_dm ─► context_guard ─► done
            └─ subconscious tick (offline, cron/heartbeat) — reviews compressed history +
               cumulative world diff → emits a steering directive that later cycles inject
  UI: Brain → Orchestration tab (orchestration.* RPC + orchestration:message socket)

The wake graph (stages 4 to 5)

One OrchestrationState (graph/state.rs) flows through the whole cycle and is checkpointed at every super-step boundary under thread orchestration:<session_id> by SqlRunLedgerCheckpointer. frontend is the router (command-routing): when channel_response is present it wraps up (send_dm), otherwise it hands macro instructions to execute and loops back. The reasoning core always sets agent_reply, so the second front-end pass compiles a channel_response; a hard max_supersteps backstop guarantees termination (spec §5 loop continuity).

Every behaviour-bearing node is bundled behind one injected OrchestrationRuntime (graph/mod.rs): the two-pass front end (Quick LLM, hint:chat), the reasoning core (hint:reasoning, spawns worker sub-agents), 20:1 compression, the append-only world-state diff, utilization + eviction, and the DM reply. As a result, the graph mechanics are hermetically unit-testable with a single stub while production wires the real agents / store / memory in ops.rs.

Memory mechanics (spec §3 to §4):

compress condenses the cycle's execution trace to a strict input/20 token budget (200-token floor only when still compressive), retry-once-then-truncate, persisted idempotently by cycle_id to the compressed_history table.
world_diff appends one entry to an append-only timeline (monotonic seq from genesis, terminal_state kv), idempotent by cycle_id.
context_guard runs after all mutations, before END: at ≥ context_evict_threshold (clamped 0.8 to 0.9) it evicts the oldest compressed entries to memory RAG under path_scope = orchestration/<session> and resets utilization.

The subconscious steering loop (stage 6)

The existing SubconsciousEngine tick gains an orchestration_review stage that runs fully offline: a single tool-free provider chat on the subconscious route under SubconsciousTainted origin. It reads unreviewed compressed_history plus the cumulative world-diff timeline and emits at most one dense STEERING_DIRECTIVE (with expires_after_cycles) into the append-only steering_directives store (supersede chain + cycle-count expiry). At the start of each wake cycle ops::seed_state bumps a global reasoning-cycle counter and loads the current non-expired directive into state.subconscious_steering, which the execute node weaves into its system prompt via a task-local. The subconscious is a decoupled writer: never an edge in the wake graph, never a channel/effect.

RPC + UI (stage 7)

Renderer-only controllers (internal registry) in orchestration/schemas.rs: openhuman.orchestration_{sessions_list, messages_list, send_master_message, mark_read, status}. Live updates ride an orchestration:message socket event (bus.rs broadcast → core/socketio.rs bridge) fanned out for every persisted chat message. The Brain → Orchestration tab (TinyPlaceOrchestrationTab.tsx + useOrchestrationChats.ts) reads real store classification, live-updates, and lets the owner steer the front-end agent from the Master composer.

Running unattended (stage 8)

No message loss: ingest dedupes by relay message_id before decrypt (the Signal ratchet is never advanced twice); a relay/decrypt error leaves the message un-acked for a clean retry.
No duplicate DM: the idempotence cursor advances only after a completed, DM-sent cycle; dm_sent is checkpointed and the deterministic cycle_id keeps the compressed_history / world_diff store writes idempotent across a checkpoint resume.
Backpressure: each cycle awaits scheduler_gate::wait_for_capacity(), so a Paused/Throttled gate defers the cycle rather than dropping it.
Malformed input: any non-envelope / malformed DM body falls back to the peer's Master window; the parser never panics.
Observability: nodes log entry/exit with session_id / cycle_id / tick_id correlation ids; orchestration.status exposes the current steering directive, last subconscious tick, ingest-cursor lag, and last error. Message bodies / decrypted plaintext / seeds are never logged (guarded by a source-scan test).

Configuration

The [orchestration] config block (src/openhuman/config/schema/orchestration.rs): enabled, debounce_ms, max_supersteps, message_window, context_evict_threshold (clamped 0.8 to 0.9), subagent_concurrency.

6.3 KiB Raw Blame History