Skip to content

0051. Unifying Workstation AI CLIs with LiteLLM and Shared Hindsight Memory

  • Status: accepted
  • Date: 2026-07-25
  • Deciders: lusoris
  • Related: PR #3050, PR #3051

Context

The workstation runs four AI CLIs — codex, claude-code, opencode, and antigravity/agy — each with its own separate config for model routing, tool access, memory, and prompting. Nothing was shared: a fact codex learned during a task was invisible to claude-code or opencode on the next related task, tool wiring was per-CLI, and model calls bypassed the governed proxy on a per-tool basis. The goal was one control plane across every CLI, with memory as the load-bearing spine and routing plus MCP as the plumbing that lets memory flow. Detailed reference lives in tools/workstation-ai/README.md.

Decision

Unify all four CLIs behind one control plane, shipped under tools/workstation-ai/ and installed by apply.sh:

  • Memory (the spine) — one shared Hindsight bank cordana::private::project::lusoris-k8s (tenant default). Every CLI reads and writes the same bank: recall (INJECT) prepends a bounded observation block before each run (recallMaxTokens=768, types=observation first) so context replaces re-derivation; retain (RETAIN) posts the transcript back after the run and nightly consolidation folds facts into compact observations. Provenance is kept with tags (client:<cli>, project:lusoris-k8s, trust-zone:private); recall does not filter by client, so learnings compound across CLIs while the tag still allows slicing by author.
  • Routing — model calls go through the LiteLLM gateway (litellm.ai.cauda.dev, ADR-0030): codex via [model_providers.cordana] on /v1 and opencode via provider.cordana, both authenticating with the agent key. claude-code stays Anthropic-direct by design (its model is claude-*); agy is unsupported for routing since it uses the Antigravity/Google backend + OAuth and exposes no OpenAI base_url.
  • Tools — the same MCP toolset for every CLI, fronted by the LiteLLM MCP gateway: one aggregate cordana-mcp entry (searxng web search + devtools, 13 tools, fleet key), a separately-mounted cordana-knowledge server on its own least-privilege agent key scoped to /knowledge/mcp (Cordana registry + knowledge, 13 tools), and the public context-awesome server. A stdio bridge injects the per-request bearer token that codex/opencode/agy cannot send natively.
  • Per-CLI wiring — codex and claude-code are fully wired with native inject+retain (codex UserPromptSubmit/Stop hooks → recall.py/retain.py; claude-code the hindsight-memory plugin). opencode is staged via the @vectorize-io/opencode-hindsight plugin pointed at the shared bank. agy has MCP plus headless memory through the bin/agy-mem wrapper (agy has no hook surface); interactive agy is recall-only via the knowledge MCP tool.
  • Installation — apply.sh copy-installs each tracked config into its ~ location (not symlinks), backing up every replaced file to <file>.bak-<timestamp>, merging the codex TOML infra block and the Claude .mcp.json servers while preserving user prefs, and linking + enabling the Hindsight tunnel systemd unit. Copy-install rather than symlink is required because the repo is a bare checkout whose worktrees are throwaway (PR #3051); the codex-TOML merge additionally refuses to run against an empty/malformed fragment so it cannot wipe a live config.

Alternatives considered

  • Per-CLI / per-agent memory split, like the paperclip fleet's [agent, gitProject] banks: rejected — the workstation is one operator doing related work across CLIs on the same repo, so a per-CLI split would silo the learnings and defeat the entire point. One project bank plus a client: tag gives both sharing and provenance.
  • Route claude-code and agy through LiteLLM as well: not done — claude-code is intentionally Anthropic-direct, and agy's Antigravity/Google + OAuth backend exposes no OpenAI base_url; the limitation is documented rather than faked.
  • Symlink each tracked config into ~: rejected — the bare/worktree layout means a symlink would point into a throwaway checkout; copy-install is durable (PR #3051).
  • Register the originally-named wikipedia and arxiv MCP servers: not available on the gateway; web/paper/library research is covered by searxng_web_search, devtools-internet_search, and devtools-get_library_documentation.

Consequences

  • A fact established by one CLI is recalled by the others on the next related task; the injected observation block is bounded (≤768 tokens), so cross-CLI sharing reduces re-derivation without bloating context or cost.
  • codex and claude-code run fully wired; opencode is staged; agy gets MCP + headless memory (recall-only when interactive). The shared-bank topology needs no Hindsight server change — the bank id is client-chosen and the tenant is fixed default.
  • Hindsight remains a single replica (Recreate, RWO pg0, cannot scale replicas), and its extraction/consolidation LLM shares the B580 8b lane capped at LLM_MAX_CONCURRENT=3. Adding four interactive CLIs is low extra load (one operator, retainEveryNTurns=10, async retains, one bank), so the existing caps absorb it. The one scenario that would strain the 8b lane — many parallel non-interactive batch jobs (codex exec / agy -p fan-out) — is flagged; the lever is a dedicated memory LLM lane, to be proposed only when measured.
  • apply.sh is idempotent and backs up every replaced file, so rollback is a restore-from-.bak plus disabling the tunnel unit. Runtime prerequisites (the MCP stdio bridge, the LiteLLM token helper and keys, and the per-CLI plugin/hook installs) stay operator-provided and are verified by apply.sh as PASS / MISSING / SETUP rather than committed.

References

  • PR #3050 — feat: unify workstation AI CLIs behind LiteLLM + MCP + shared Hindsight memory
  • PR #3051 — fix(tools): workstation-ai copy-install not symlink (bare repo)