The official billion-context plugin: a context-compression plugin for small context windows (a 100K context is enough), token savings (5x fewer tokens), and month-long single sessions (billions of tokens).
Install
# from npm (prebuilt)
dsh plugin --profile web add billion-context
# from GitHub (first run asks for allowBuilds approval — follow the hint, retry)
dsh plugin --profile web add github:ranxianglei/billion-context
Any plugin you install runs third-party code with your own permissions — it can read your files, use your credentials, and reach the network, and tool approvals don’t sandbox it. GitHub-sourced plugins also run build scripts at install time — pnpm blocks those until you allow them, so an install can stop with ERR_PNPM_GIT_DEP_PREPARE_NOT_ALLOWED or ERR_PNPM_IGNORED_BUILDS; dsh prints the exact key to add under allowBuilds in your profile’s pnpm-workspace.yaml, and the install works on the next run. Allowing a build is a trust decision: only install sources you trust, and pin a commit (github:owner/repo#sha).
README
Cache health at a glance: a healthy session keeps a 96–99% prefix-cache hit rate — compression itself costs ≤2%. Sustained lower? Check attribution with
/acpor/acp-cache(see FAQ); usual causes, in order: upstream cache TTL expiry · model switch · a bili bug (please report) · other/unknown.
Community
QQ Group: 1056132097 (full) 1108730198 (full) 1129868472 (open)
📄 Paper / Preprint
- Model-Driven Incremental Hierarchical Compression: Training-Free Multi-Generational Context Management for Long-Lived Coding Agents (English, v0.2)
📝 The paper itself is open-sourced under the MIT License as part of the codebase (
paper/). It is a living document — anyone may edit it; improvements are welcome via pull request.
A production-scale longitudinal study: 4.5 months, three hosts, 174,327 model calls, 18.76B cumulative input tokens (~24.7B across all hosts), zero window violations on 204,800-token models, marathon sessions of 8,584–12,049 calls.
📊 Independent Third-Party Benchmarks
- 1,500-Item Cross-Benchmark Evaluation (
LongMemEval-S·ContextBench·MemoryAgentBench) (中文版) — head-to-head comparison ofbillion-context(v0.1.202) againstfullcontext and 3 RAG arms (bm25,embedding,embedding-rerank) withDeepSeek V4.1 Flash,Qwen3-Embedding-8B,Qwen3-Reranker-4B, andPi Agent: #1 on ContextBench (0.3097Utilized F1,-58.3%cost vsfull), +16.6% over best RAG on LongMemEval-S (79.20%vs62.60%,-71.1%independent-session cost vsfull), and 100% survival on 1.48M-token MemoryAgentBench TTL (30×steady-state compression wherefullfails199/199).
billion-context sits between any agent and its model API, rewriting Anthropic/OpenAI streams with acp-kernel compression. The model decides when and what to compress into high-fidelity summaries — not a hard truncation limit.
Why
Long coding sessions blow up context. Each provider charges per token, and once you pass the context window the session degrades or dies. billion-context compresses consumed conversation into layered summaries so you can run a single session for days — billions of tokens through one context window.
Unlike a host's built-in summarizer, compression here is incremental, reversible, and prefix-cache friendly: summaries are written in small ranges, can be decompressed on demand, and the cache prefix stays intact.
How it works
Context-management tools
The proxy injects four context-management tools into the conversation; the model calls them itself as context grows, and the proxy executes compress server-side so folded ranges stay summarized in history until restored:
compress— fold a message range into a detailed summary.decompress— restore a compressed range when exact details are needed again.search_context— keyword search over compressed summaries and visible messages.acp_status— context-usage overview plus which ranges are still compressible.
Which do I need?
Pick by your client:
| Client | Use |
|---|---|
| pi | billion-context — bili pi (launcher) or bili plugin install pi (native); standalone billion-context-pi remains usable — details: docs/clients.md |
| opencode (1.x / 2.x) | billion-context — bili opencode (launcher) or bili plugin install opencode (native); standalone opencode-acp remains usable on 1.x — full guide: OpenCode |
| omp | billion-context via bili omp (built-in plugin) or bili plugin install omp (self-spawning native plugin, no launcher) |
| dsh | bili dsh (launcher — full native plugin via --patch) or bili plugin install dsh ≡ dsh plugin --profile <name> add billion-context (one unified lane) — details: docs/clients.md |
| kimi | bili plugin install kimi (self-spawning native, Kimi Code ≥ 2.0.0) or bili kimi (cert-MITM) or /bili/ prefix — details: docs/clients.md |
| hermes | bili plugin install hermes (self-spawning native, Python plugin #958) or bili hermes (cert-MITM) |
| zcode (Z.ai / bigmodel coding plan) | bili plugin install zcode (self-spawning native, #1145) or cert-MITM via the GUI's Settings → Network or /bili/ prefix — details: docs/clients.md |
| claude | bili claude (launcher) or bili plugin install claude (native posture, #964 — managed settings block + session-owned proxy; see the notes below) |
| codex | bili codex (launcher — the full zero-config posture) or bili plugin install codex (MCP-shell tools companion: start bili first — the shell never spawns a proxy and never routes codex's own traffic) — details: docs/clients.md |
| jcode | billion-context via bili jcode (cert-MITM) or /bili/ prefix — no native mode (compiled Rust binary, no plugin seam, #962) |
| gemini (Gemini CLI) | bili gemini (launcher, GOOGLE_GEMINI_BASE_URL /bili/ rewrite) or /bili/ prefix — launcher-only (no in-loop tool seam, #1043) |
| iflow (iFlow CLI) | bili iflow (launcher, IFLOW_BASE_URL /bili/ rewrite) or /bili/ prefix |
| qwen (Qwen Code) | bili qwen (launcher, cert-MITM) or /bili/ prefix |
| antigravity (Antigravity CLI / agy, Google) | bili antigravity (launcher, CLOUD_CODE_URL /bili/ rewrite of daily-cloudcode-pa.googleapis.com — Antigravity's real default upstream, #3129) or /bili/ prefix — no plugin seam (closed Go language_server; its user-plugin surface is additive-only, #2115) — details: docs/clients.md |
| mcode (MiniMax Code) | billion-context via bili mcode (cert-MITM) or /bili/ prefix — no native mode (event hooks only, no model-request seam, #1050) |
| aider | billion-context via bili aider (cert-MITM) or /bili/ prefix — no native mode (shell-command-only hooks, no tool-injection seam, #1048) |
| copilot (GitHub Copilot CLI) | bili copilot (launcher, cert-MITM) — closed Go binary, no plugin seam (#1049) |
| amp (Amp CLI) | bili amp (launcher, cert-MITM) — closed Go binary, no plugin seam (#1049) |
| kiro (Kiro CLI, AWS) | bili kiro (launcher, cert-MITM) — Rust shell + Node agent server, no plugin seam; model traffic is the Responses wire on runtime.<region>.kiro.dev, gateway quirks handled built-in — details: docs/clients.md |
| crush (Charm Crush) | bili crush (launcher, cert-MITM) — open-source Go binary, no plugin seam; built-in provider hosts whitelisted, custom base_urls auto-discovered from crush.json (#2340) |
| zed (Zed editor) | bili zed (launcher, cert-MITM) — open-source Rust editor, no plugin seam; reqwest honors HTTPS_PROXY, CA via SSL_CERT_FILE (Linux env probing); built-in provider hosts whitelisted, custom api_urls auto-discovered from settings.json; loopback providers (ollama/lmstudio) stay direct via NO_PROXY (#2340) |
| goose (Goose CLI) | bili goose (launcher) — rustls trusts no CA file, so no cert-MITM: openai/anthropic legs via OPENAI_HOST/ANTHROPIC_HOST, custom providers via a regenerated GOOSE_PATH_ROOT overlay (#1049) |
| everything else (no context hook) | billion-context — bili <client> (launcher, preferred) or /bili/ prefix |
Native mode vs standalone extensions. The host-native plugins (bili plugin install …) and the standalone in-process extensions (billion-context-pi, opencode-acp) are mutually exclusive — both active means double compression. The installer makes the switch: it replaces the legacy entries (bare name, npm: alias, versioned, path form; array or object shape) and snapshots the original config to .bili-bak once; a project-local install is not touched — remove that one by hand. As a runtime safety net for manual installs, the native entries set BILLION_CONTEXT_NATIVE=<host> synchronously at load so a standalone extension can stand down at action time. On the pi side the marker needs billion-context-pi 0.1.72+, and the pi-native entry scans both pi settings files once its proxy is up and warns loudly when it spots a co-resident legacy entry the installer never saw — that warning is the only visible signal while an old billion-context-pi silently double-compresses.
Install
Linux / macOS — install with a user-level prefix (no sudo, no npm config
changes, and bili's self-update never hits permission errors):
npm install -g billion-context --prefix=~/.local
The bili command lands in ~/.local/bin — already on PATH in most distros;
if not, add export PATH="$HOME/.local/bin:$PATH" to ~/.bashrc or
~/.zshrc. On nvm or Homebrew Node the default prefix is already user-owned —
a plain npm install -g billion-context works as-is. On Windows the default
prefix (%APPDATA%\npm) is also user-writable — plain npm install -g billion-context.
This installs the bili command (bili-proxy is kept as an alias). Hitting
EACCES with an old root-owned prefix? Reinstall with --prefix=~/.local
(pass the flag again on any future npm reinstall of bili) — that is the
permanent fix; avoid sudo.
Prefer containers? A Dockerfile ships at the repository root — see
Running in Docker below.
Quickstart
Three ways to use it — pick one:
- Native plugin (most native):
bili plugin install <client>— bili becomes a plugin inside the client; start the client as usual. - Launcher (config-free): one
bili <client>command brings up the proxy and the client together — no real config file is ever touched. - URL change (most universal): prefix your client's baseURL with the proxy
origin +
/bili/.
Mechanism details behind these three options (plugin lifecycle, runtime-info protocol, injection priority) live in docs/technical-notes.md.
Ports, briefly (#1660): bili start (manual) owns 8787. Everything a lane
spawns for you (native hooks, launcher lanes) lives in a separate
self-managed zone starting at 18787 — collisions hop +1 and each lane
remembers its drift, so zero-config installs never fight you for a port,
and a deliberate bili start daemon is attached by default. Attach is
lane-aware (#2905): a hook rides its own lane's instance first, NEVER
another declared lane's — except lanes you list in
native.attachLanePriority — and the 8787 user-zone daemon stays the
LAST-resort fallback. An
upgrade-restart that finds the previous build still draining on the lane's
port waits for it to release (up to 5s) and rebinds the SAME port instead of
drifting (#1723); only a genuinely occupied port hops +1 — and that hop is
now logged loudly.
Option 1 — Native plugin (bili plugin install pi / omp / opencode / dsh / kimi / hermes / zcode)
The proxy lives inside the client: install once, then start the client exactly as you always do — no launcher command, no env vars, no fixed port, no URL edits. Supported today for pi, omp, opencode (1.x and 2.x), dsh, kimi, hermes and zcode:
bili plugin install pi # registers a "billion-context" entry in pi's settings (npm form when bili itself was npm-installed)
bili plugin install omp # registers an extensions entry in omp's config.yml (~/.omp/agent/config.yml)
bili plugin install opencode # registers the plugin in opencode's real config + disables native auto-compaction
bili plugin install dsh # runs 'dsh plugin --profile <name> add billion-context' for every existing profile
bili plugin install kimi # writes $KIMI_CODE_HOME/plugins/managed/billion-context/kimi.plugin.json (+ installed.json record); per-session routing block lands in config.toml on first start (Kimi Code >= 2.0.0)
bili plugin install hermes # copies the Python plugin into ~/.hermes/plugins/billion-context/ (+ machine-owned bili.json sidecar) and enables it via `hermes plugins enable billion-context`
bili plugin install zcode # writes hooks.enabled + a SessionStart hook + mcp.servers.bili into ~/.zcode/cli/config.json; per-session routing lands in the bigmodel provider store on first start
bili plugin remove <client> # undo (dsh removes through the same channel; config snapshots go to .bili-bak)
Where a client has its own plugin channel you can also install natively, skipping bili commands entirely:
dsh:
dsh plugin --profile <name> add billion-contextis the very commandbili plugin install dshdrives per profile (npm form only; a git checkout has no published entry — pasting a GitHub address installs source withoutdist/and the bundle never loads) — same end state either way (pnpm into the profile, bundled patch layer mounted by dsh itself); remove through the same channel. See the dsh section below.opencode: easiest is
bili plugin install opencode(it picks the right entry form for your host version). Hand-install without bili (npm form only; a git checkout has no published entry):OpenCode 1.x — add the bare npm name to your real config's plugin list:
"plugin": ["billion-context"]. The package publishesexports["./server"]→dist/agent/opencode-native.js, so opencode loads it through its own Npm.add machinery and the plugin self-spawns exactly like the bili-installed form.OpenCode 2.x — the bare name does NOT work: 2.x resolves it through the package
mainfield (dist/index.js, the CLI — no plugin definition), so the module load fails (#3976). Instead, put a one-line re-export in a directory and point the config'spluginslist at that directory:// <shim dir>/index.js export { default } from "<abs path to the global install>/dist/agent/opencode-native.js";{ "plugins": ["<shim dir>"] }
Either generation: do the two things the bili installer would have done for you too — set
"compaction": { "auto": false }in the same config (otherwise OpenCode's native auto-compaction double-compresses) and keep a manual backup of the file first.claude: this repository doubles as a Claude Code plugin marketplace —
/plugin marketplace add ranxianglei/billion-context, then/plugin install billion-context@billion-context, then run/billion-context:bili-setup(it drivesbili plugin install claudefor you and tells you to restart). Same end state as the bili installer; the plugin ships no hooks or MCP entries of its own, so nothing double-registers.
For pi / omp / kimi / claude there is no client-side channel — bili plugin install <client> writes their config entries for you (kimi's declarative
kimi.plugin.json + registry record, claude's managed settings block, …).
Notes:
- Native mode is mutually exclusive with the standalone in-process extensions (
billion-context-pi,opencode-acp) — the installer swaps the entries and snapshots the original config (.bili-bak). - pi: native installs wire in built-in subagents by default (
acp_delegate/acp_delegate_wait/acp_delegate_cancel) — how to disable them while keeping context compression: Pi built-in subagents. - OpenCode legacy sessions, V1/V2 shapes and caveats: OpenCode.
kimireports runtime-info at bootstrap only (static headers can't carry per-request window/model values); subagent tool calls are routed by the proxy's outbound tool-use witness ring (#1685) — no model-visible conversation id.hermes's native plugin is Python: it points hermes' httpx stack at the proxy via env vars after a health check and stamps per-request headers through anllm_requestmiddleware.codexis the one client a plugin install cannot make self-sufficient: codex routes model traffic via env only (no config-file routing seam for the default ChatGPT-login provider — a managedmodel_providersblock would force API-key auth and drop subscription login), and an MCP server cannot inject env into its parent.bili plugin install codexwrites a single[mcp_servers.bili]block into~/.codex/config.toml(command = node, args = dist/mcp.js) exposing the four ACP tools; at session start the shell resolves a proxy — envBILI_MCP_PROXY> the live-instance record (any lane's proxy or abili startdaemon) > the 8787 user-zone default (#1660 removed the install-time origin bake, #403) — nothing reachable →tools/listfails with -32003. So: start bili first (bili startor any client's lane proxy), export HTTPS_PROXY yourself if you also want compression, or usebili codexfor the zero-config full posture. Mechanics: docs/clients.md.claudehas a native posture (#964): managed settings block +SessionStarthook + MCP shell; the hook rides the self-managed port zone (#1660) and re-pins the managed URL to the live origin each session, so port drift self-heals. Since #1821 the MCP shell also watches the proxy mid-session (30 s tick × 3 failures) and respawns a dead lane proxy itself, so model requests recover within ~90 s instead of a new session (same semantics as the zcode lane). Opt out withBILI_NATIVE_CLAUDE=0(passthrough). Mechanics: docs/technical-notes.md. Known limitation (#2290): the Claude Desktop app's Code tab (CLAUDE_CODE_ENTRYPOINT=claude-desktop) sets its ownANTHROPIC_BASE_URLon its embedded Claude Code, overriding the managed one — desktop sessions bypass the proxy entirely while the managed block'sDISABLE_AUTO_COMPACT=1still applies, so they get neither bili compression nor native auto-compaction. The hook warns loudly at session start (and records it forbili doctor, which also flags machines where Claude Desktop is installed alongside the native lane); terminal sessions are unaffected. Field-tested 2026-10-06 (#2290): puttingHTTPS_PROXYin the settingsenvblock produced zero CONNECT traffic to a live MITM-enabled proxy on Claude Desktop 2.19675.1 / embedded CC 2.1.288 (Windows 11) — the variable reaches no network stack there, matching the CLI behavior of that era — note that newer builds (CC 2.1.301+) DO honourHTTPS_PROXYon the CLI, which is whybili clauderoutes via the/bili/base-URL rewrite — the launcher gives this lane no proxy and no CA at all, and a child that brings a proxy of its own gets the loopback hosts merged intoNO_PROXY/no_proxyso that leg stays direct regardless of its proxy behavior (#3446). As of that version there is no usable routing seam on the desktop Code tab; the lane stays tracked in #2290 in case upstream changes behavior. Account-bound features (#4693): because the lane routes via a customANTHROPIC_BASE_URL, Claude Code treats the session as third-party/API-key and disables features bound to your Anthropic account/Cloud — Remote Control (/remote-control, driving a local session from mobile/desktop/web) disappears from the command list while bili routes traffic. Compression is unaffected; run plainclaudewithout bili (orbili plugin remove claude) for sessions that need those features. This is inherent to the base-URL routing seam — no alternative interception point survives CC's undici stack (#2290) — not a config deletion. Desktop-only users can restore native auto-compact withbili plugin remove claude. Since #5095 the managed block also installs aSubagentStarthook (node <root>/dist/claude-subagent-hook.js) that mints a per-dispatch lane token into each subagent's context: subagents that ride the parent's session id with no agent-id headers (the VS Code extension build, #4352) key onto their own<id>|sub:…:t:<token>compression lanes, so parallel same-type dispatches stop merging (and the token disappears from later requests once the first message folds — those fall back to the fingerprint lane). Uninstall removes the hook with the managed block.zcodehas a native posture (#1145): managed~/.zcode/cli/config.jsonblock + per-session providerbaseURLrewrite. Full mechanics: docs/clients.md.jcodeandaiderhave no native mode (no plugin/MCP/tool-injection seam: #962, #1048) — usebili jcode/bili aider.copilot,ampandgooseare launcher-only (#1049); goose cannot be cert-MITMed (rustls trusts no CA file) and rides plain-HTTP base-URL redirects instead.
Option 2 — Launcher (bili pi / bili codex / bili claude / bili omp / bili opencode / bili hermes / bili dsh / bili codebuddy / bili qoder / bili trae / bili jcode / bili kimi / bili gemini / bili iflow / bili qwen / bili antigravity / bili mcode / bili aider / bili copilot / bili amp / bili goose / bili kiro)
The launcher wraps a client in one command: it starts a proxy on an
independent port (a fresh instance is always spawned — a port is never
reused), then points the client at it — certificate-based MITM where the
client honors proxy/CA env vars, or an isolated /bili/ config rewrite
where it doesn't. No real config file is ever edited; the client's own
config is READ to discover which HTTPS upstream hosts it talks to, and those
hosts are whitelisted for MITM so the proxy can TLS-terminate exactly them
and blind-tunnel everything else.
bili pi # launch pi through the proxy — file-free (#535): env + extension registerProvider, real ~/.pi untouched
bili codex # launch codex through the proxy
bili claude # launch claude through the proxy
bili omp # pi-style, file-free (#535): env + extension registerProvider + compaction cancel, real ~/.omp untouched
bili opencode # OpenCode (1.x & 2.x): full guide in the [OpenCode](docs/clients.md#opencode) section below
bili hermes # file-free (#535): hermes proxy env (HTTPS_PROXY + combined CA bundle via SSL_CERT_FILE) — https via CONNECT MITM, http via absolute-form forward proxy; real ~/.hermes untouched
bili dsh # deepseek-harness: full native plugin injected via --patch (#941) — real dsh tools, session-bound /acp + /acp-cache (plugin mode); non-loopback upstreams ride proxy envs, loopback keeps the overlay DSH_HOME rewrite (#535); dsh auto-compaction off everywhere — host plane via the bundle patch, web-profile presets at load time (#1772/#2474/#3354)
bili codebuddy # Tencent CodeBuddy Code CLI: CODEBUDDY_BASE_URL /bili/ rewrite (OpenAI chat completions wire), budget aligned via CODEBUDDY_AUTO_COMPACT_WINDOW; real ~/.codebuddy untouched
bili qoder # qoder CLI (`qoder`/`qodercli`): model endpoint is hardcoded https (no /bili/ rewrite possible) — cert-MITM via HTTPS_PROXY + NODE_EXTRA_CA_CERTS, default model hosts whitelisted (#653). Qoder Desktop is a separate BYOK/custom channel that speaks plaintext OpenAI over a `/bili/` prefix — point it at bili as a normal custom provider (not this MITM lane); its official-subscription traffic is encrypted and out of scope (#764)
bili trae # Trae CLI (ByteDance, closed Go binary, no base-URL override) — cert-MITM via HTTPS_PROXY + SSL_CERT_FILE, model host from TRAE_CLI_API_HOST or the default enterprise gateway (#655)
bili jcode # jcode (Rust agent harness) — env-only cert-MITM launch: HTTPS_PROXY + SSL_CERT_FILE, model host api.z.ai whitelisted, local loopback providers stay direct via NO_PROXY
bili kimi # Kimi Code CLI (Moonshot): standard proxy envs except an unconditional loopback bypass — https via cert-MITM, http via absolute-form forward proxy; loopback endpoints inventoried with a manual /bili/ hint (#757)
bili gemini # Gemini CLI (Google): GOOGLE_GEMINI_BASE_URL /bili/ rewrite to generativelanguage.googleapis.com (Google native wire), real ~/.gemini untouched
bili iflow # iFlow CLI: IFLOW_BASE_URL /bili/ rewrite to apis.iflow.cn/v1 (OpenAI chat-completions wire), real ~/.iflow untouched
bili qwen # Qwen Code (multi-protocol gemini-cli fork, no base-URL hook): cert-MITM via HTTPS_PROXY + NODE_EXTRA_CA_CERTS, default DashScope/Qwen model hosts whitelisted, custom relays via --mitm-domain
bili antigravity # Antigravity CLI (agy, Google): CLOUD_CODE_URL /bili/ rewrite of daily-cloudcode-pa.googleapis.com — the real default upstream (bare cloudcode-pa 400s on /v1internal:*, #3129); undocumented language-server env override, v2.19.1 binary-verified; wire recognized as Google native by path (#2115)
bili mcode # MiniMax Code CLI: same shape as kimi (proxy envs, unconditional loopback bypass, cert-MITM/absolute-form); session bound via X-Mavis-Session-Id (#1050)
bili aider # Aider (Python pair programmer): cert-MITM via HTTPS_PROXY + SSL_CERT_FILE/REQUESTS_CA_BUNDLE; endpoint from OPENAI_API_BASE / ANTHROPIC_BASE_URL / --openai-api-base / .aider.conf.yml (#1048)
bili copilot # Copilot CLI (GitHub, closed Go binary) — cert-MITM via HTTPS_PROXY + SSL_CERT_FILE, api.githubcopilot.com + per-plan subdomains whitelisted (#1049)
bili amp # Amp CLI (Sourcegraph, closed Go binary) — cert-MITM via HTTPS_PROXY + SSL_CERT_FILE, ampcode.com whitelisted (#1049)
bili goose # Goose (Block, Rust/reqwest): rustls release builds trust no CA file — openai/anthropic legs via OPENAI_HOST/ANTHROPIC_HOST, custom providers via a regenerated GOOSE_PATH_ROOT overlay (/bili/ rewrites, real config untouched) (#1049)
bili kiro # Kiro CLI (AWS): cert-MITM via HTTPS_PROXY + combined CA on SSL_CERT_FILE and NODE_EXTRA_CA_CERTS; only runtime.<region>.kiro.dev is decrypted; developer/system items re-voiced as user for the gateway; session = Kiro's conversation id (survives --resume)
bili pi --mitm-domain api.foo.com # add a domain to the MITM whitelist
Option 3 — URL change (/bili/ prefix)
Start the proxy:
bili
Then just prefix your client's existing baseURL with http://localhost:8787/bili/.
The full upstream URL is embedded in the path, so the proxy knows where to
forward without any config:
client baseURL before: https://api.openai.com/v1
client baseURL after: http://localhost:8787/bili/https://api.openai.com/v1
For more per-client configuration examples, see the web UI guide at http://localhost:8787.
Verify. With the proxy running and your config saved, check it answers and that your first real request shows compression activity in the log:
# Health check (proxy up + where it forwards)
curl -s http://localhost:8787/__bili/health
# → {"ok":true,"upstream":"https://api.anthropic.com"}
# Live session stats (after a real request)
curl -s http://localhost:8787/__bili/stats
Then send one message from your client and watch the log
(~/.local/state/billion-context/bili.log, also printed to stderr). You
should see a processTurn line per request, and once the conversation grows,
[acp-usage] round N input=X cached=Y (cache hit Z%) + a compress event.
Client deep dives
Everything that doesn't fit in one Quickstart line — how each client's lanes attach, what gets written where, and known limitations — lives in docs/clients.md: dsh · Kimi Code · Hermes · ZCode · Gemini family (Gemini CLI / iFlow CLI / Qwen Code / Antigravity) · cert-MITM clients that never compress (CONNECT blind tunnels, #897) · unrecognized endpoints going direct (#1290) · OpenCode (launcher / native / pure proxy, /acp status & rules, legacy opencode-acp sessions #920).
From source (development)
Risk:
masteris the development branch and can be broken at any moment — unreleased changes, failing builds, unverified fixes. Unless you are patching the code or testing an unreleased fix, install a release with npm as above.
git clone https://github.com/ranxianglei/billion-context.git
cd billion-context
npm ci
npm run build
npm link
bili --version
After npm link, bili works from any directory.
FAQ
How do I check my cache hit rate? Don't dig through logs — /acp-cache
prints a text summary report right in the client, headed by a clickable
Web UI link: open it for the web version of the session page — the cache
hit-rate line chart plus per-break attribution:
The report has four
blocks that pin things down at a glance: GRAND LEDGER (totals + hit% with
an explicit HEALTHY verdict; misses decomposed into new content /
compress re-pay / upstream-ttl-or-client-rewrite) · FOLD ECONOMICS (per-fold
economics: net tokens saved, paid-back verdicts) · LINE ITEMS (anomalies
only: hit<85% or miss≥5000). Rule of thumb: compression itself costs ≤2% —
a healthy session sits at 96–99%. When you see less, the attribution tells
you which of the usual suspects it was, in this order: ① upstream cache TTL
expiry (shows up as stable-prefix misses — the top-spikes line names idle
times), ② a model switch, ③ a bili bug (report it with the page attached),
④ other/unknown. /acp-cache [full] lists every fold & line; same report over
Since #1535 the report ends with a MODEL SWITCHES section: a mid-session
model change invalidates the provider's prefix cache, so the whole stable
prefix is re-billed on the next request — each switch's unexplained residual
(its ttl bucket minus new content) is charged to the switch instead of
masquerading as TTL expiry; per-event from → to, hit %, and attributed
tokens are listed (full lists every event, the summary the last 8), and
the web sessions table gains a matching model-switch column. Since #2131 the
same machinery fingerprints the outbound credential (a 12-hex sha256 of the
authorization/x-api-key-family headers — the raw key is never stored) and
attributes a key switch — the relay behind a stable URL rotated to a
different account — in a dedicated KEY SWITCHES section, a key switch:
line in CACHE INVALIDATION, and a 🔑 badge in the web sessions column.
HTTP: GET /__bili/cache-report; the raw per-request [acp-usage] lines still
land in the log file for deep dives.
What does /acp show? In clients with the native plugin (opencode, dsh,
hermes), /acp renders the ACP status panel of the current conversation
straight from
the proxy (session, blocks, compressible ranges, usage); before the first model
request it shows an idle notice instead. /acp-cache [full] prints the cache
report above.
Can I query sessions and config over the web? Yes — open
http://localhost:8787: an overview dashboard, session
list with per-session detail, live logs, a config editor, and an upstream
connectivity test. A Blog view renders notes and long-form articles inside the
panel itself — bundled with the install, refreshed from GitHub master on a
~1-hour throttle while online (fail-soft to the bundled copy when offline);
the full library also lives on GitHub Pages. Everything else is also plain JSON
for scripting (/__bili/stats, /__bili/sessions, /__bili/config, …).
When does compression happen? It is model-driven: the injected context
tools are called by the model as context grows, gentle growth nudges
(~50K-token steps by design, adjustable via compress.nudgeGrowthTokens)
prompt it along the way, and preflight fires as a hard backstop when the input
alone exceeds the window (#470). Watch it live with /acp or the web UI.
Does compression shrink my client's own conversation file? No — the most common surprise in very long sessions (#5796). Compression shrinks what is sent to the model: every outgoing request is rebuilt from live blocks plus summaries, so the wire stays small however long the conversation runs. The client's stored history is user data — bili never rewrites it. On dsh the host's auto-compaction is additionally disabled while bili is active (two compressors must not fight over one history, #1772/#2474/#3354), so the host-side file grows with every turn: each turn takes longer as it grows (measured: tens of seconds per turn at ~30K messages, #5796), and the practical convergence path is to start a new session when it gets heavy. The web session list badges sessions at 5,000+ requests (长会话/slow) as a reminder that this point has been reached.
Why does the first compaction wait until ~200k? Compaction does not
trigger on absolute window position but on growth intervals: by default the
first soft compaction fires 50k tokens past the boot content
(compress.nudgeGrowthTokens, flat and window-independent). With a boot around
100k — or with the growth step set to ~100k — the first compaction may wait
until ~200k. To make it fire earlier: 1) trim the system prompt, disable
unneeded tools, prune skills; 2) lower compress.nudgeGrowthTokens to ~50k;
3) enable lean mode. Worked example in
docs/configuration.md.
Is bili transparent? How do I turn it off? Unrecognized endpoints forward
unchanged (docs/clients.md), and every mode reverses cleanly:
bili plugin remove <client> for native installs, stop using the launcher
command / env vars / /bili/ prefix for the other two — traffic goes direct
again immediately.
Can I combine bili with a model-routing gateway (e.g. magpie)?
Yes — point the client at bili, and point bili at the gateway through the
/bili/ prefix, which embeds the gateway's absolute URL:
http://localhost:8787/bili/http://127.0.0.1:3425…. Example: Claude Code →
ANTHROPIC_BASE_URL=http://localhost:8787/bili/http://127.0.0.1:3425 +
ANTHROPIC_API_KEY=magpie; OpenAI-style clients →
OPENAI_BASE_URL=http://localhost:8787/bili/http://127.0.0.1:3425/v1 (+
OPENAI_API_KEY=magpie). Compression happens before routing — every request
that reaches any account in the gateway's routing groups is already compressed.
Credentials stay in the gateway: the client's dummy key rides through
untouched, and real keys never touch bili. Caveats: ① an account switch
mid-conversation invalidates the provider's prefix cache — inherent to
multi-account routing, not bili (watch it in the KEY SWITCHES section of
/acp-cache); ② routed model names are opaque to bili's window lookup —
declare their context windows under the providers key
http://127.0.0.1:3425 via models.<name>.context, or they run on the
fallback window (route-miss warning, #2317); ③ if the relay rejects bili's
Anthropic-lane cache_control stamps, set compat.noCacheControl; ④ loopback
gateways work out of the box; a gateway on a private non-loopback address may
additionally need network.allowPrivateHttpProviders (#2984); ⑤ for Claude Code
native plugin mode (bili plugin install claude), set
BILI_CLAUDE_UPSTREAM=http://127.0.0.1:3425 before install so the managed block
wraps the relay into ANTHROPIC_BASE_URL (or configure
ANTHROPIC_BASE_URL=http://127.0.0.1:8787/bili/http://127.0.0.1:3425 directly;
localhost and 127.0.0.1 both accepted, #5369). Verified end-to-end over all
three wires (OpenAI Chat / Responses / Anthropic Messages, including
cross-protocol conversion), #4433.
Does this work with opencodex (ocx) — the local Codex/Claude Code gateway with a ChatGPT account pool? Same pattern: point the client at bili, and point bili at ocx through the /bili/ prefix — OpenAI-style clients (chat completions and Responses, e.g. Codex) → OPENAI_BASE_URL=http://localhost:8787/bili/http://127.0.0.1:10100/v1; Anthropic-style clients → ANTHROPIC_BASE_URL=http://localhost:8787/bili/http://127.0.0.1:10100. A dummy key value satisfies clients that require one; it rides through untouched. Keep bili in front of ocx rather than behind it: ocx owns the provider credentials and the account pool at its final hop, so only this order keeps compression and pooling working together. Loopback needs no key between the two (ocx's default loopback bind reads no token); a non-loopback bind takes OPENCODEX_API_AUTH_TOKEN plus the x-opencodex-api-key header per its docs. Request models by ocx's provider/model id so routing is deterministic; as with any gateway, routed ids are opaque to bili's window lookup (declare them via models.<name>.context) and a mid-session account rebind invalidates the prefix cache (watch KEY SWITCHES in /acp-cache). Verified end-to-end over all three wires (OpenAI Chat / Responses / Anthropic Messages), bili 0.1.212 × ocx 2.97.0.
Where are logs and session data stored? Log:
~/.local/state/billion-context/bili.log (also mirrored to stderr); session
state: ~/.local/share/billion-context/ (XDG-overridable; Windows AV exclusion
#362, opt-in cleanup #1082) — full paths in docs/configuration.md.
Running the proxy
Running in Docker
A Dockerfile ships at the repository root — it installs the published npm
package at build time (the tested artifact; nothing is compiled in the image)
and runs the proxy as a service:
docker build -t billion-context .
docker run -d --name bili -p 8787:8787 \
-v bili-config:/root/.config/billion-context \
-v bili-data:/root/.local/share/billion-context \
-v bili-cache:/root/.cache/billion-context \
-v bili-state:/root/.local/state/billion-context \
billion-context
Then point clients' baseURL at http://<host>:8787/bili/… exactly like Option 3
above.
- Base:
node:24-slim— uses the builtinnode:sqlitestorage engine directly (no native-addon download). Don't switch to Node 20 bases: the self-downloaded SQLite engine requires Node ≥ 22 and crashes under 20 (#3341). - Updates: in-container self-update is off by default (
ACP_AUTO_UPDATE=0) — the sanctioned update channel for a containerized copy is rebuilding/pulling a newer image; an in-place self-update would be silently reverted to the image version on the next recreate. Opt back in with-e ACP_AUTO_UPDATE=1; pin a release at build time with--build-arg BILI_VERSION=0.1.197(defaultlatest). - Data: the four mounted dirs hold config, session state, update throttle/lock + cached SQLite engine, and logs respectively. Mount volumes or bind mounts so they survive container replacement; the image also declares them as
VOLUMEs so even a baredocker runkeeps them out of the container layer. - Security: the container listens on
0.0.0.0:8787so clients outside it can connect — there is no authentication, so expose the port only on a trusted network (same caveat as--host 0.0.0.0below). The web UI and/__bili/*JSON endpoints work unchanged through the mapped port. - Healthcheck: the image ships a
HEALTHCHECKprobing/__bili/healthon 8787 via node's builtin fetch (no curl needed in the slim base); if you override the listen port, remember the probe stays pinned to 8787.
Flags
bili --port 9000 # change listen port
bili --host 0.0.0.0 # listen on all interfaces (see host note below)
bili --debug # verbose logging (also: set "debug": true in config)
bili --passthrough # forward without compression (smoke-test mode)
bili --config ~/my-bili.json # use a different config file
bili -c dsh-test # named config → ~/.config/billion-context/dsh-test.json (per-model files)
bili update # check & install a newer version now (bypasses throttle)
bili doctor --fix # converge drifted dsh profile ledgers (manifest spec / lockfile record / bundle copy) to the on-disk version (#3496)
bili --no-auto-update # disable self-update for this run
bili --auto-restart-on-update # self-restart when a new version is installed (default off)
bili --bin /path/to/client # launcher: spawn this exact client binary instead of auto-detecting (env BILI_CLIENT_BIN)
Flags override env vars and the config file. bili --help lists them all.
Remote agents (--host)
By default the proxy binds 127.0.0.1 and only accepts loopback connections. To serve agents on other machines: bili --host 0.0.0.0 (or your LAN IP); remote agents point their model baseURL at http://<this-host>:<port>/bili/….
- MITM-mode
CONNECTthen also accepts remote clients — for whitelisted model hosts only. Blind tunnels to arbitrary hosts stay loopback-only, so the proxy can never be used as an open relay. /bili/<absolute-url>destination admission (#409): the proxy itself and link-local/metadata addresses are always denied; loopback/private destinations are allowed for local clients (self-hosted upstreams) and denied for remote clients unless listed inBILI_TUNNEL_ALLOWED_HOSTS(hostorhost:port, comma-separated). One exception (#1073): a local client relaying a management path (/__bili/*,/__acp/*) to a loopback IP-literal destination is forwarded unmarked; remote peers and hostname destinations keep the internal tunnel marker unconditionally, and management paths stay unreachable through the tunnel even via NAT hairpin. An absolute-form request addressed to the instance's own endpoint on a management path is served locally instead of tunneled (a forward-proxy-style health probe gets a real answer).- There is no authentication: only do this on a trusted LAN or behind a firewall. The
/__bili/management endpoints remain loopback-only. A startup[security]warning reminds you of the above.
Debugging
bili --debug (or env ACP_DEBUG=1, or "debug": true in config — flag > env > config) logs every processTurn (tag counts, token usage), the nudge decision (growth/usage/pendingT1/shouldInject), client headers, and SSE rewrites.
Log file
All logs tee to ~/.local/state/billion-context/bili.log by default (XDG state dir) and still print to stderr. Override with "logFile" in config or ACP_LOG_FILE (off disables the file). Auto-rotates at 10 MB (bili.log.old). Per-request
…
Community comments
Comments are public GitHub Discussions. Loading them connects to GitHub and Giscus; a GitHub account is required to post.