DeepSeek Harness Plugin

ranxianglei/billion-context

Stars ★ 7824 Downloads (30d) 3,333,136 Category Sessions & Messages Added 2026-09-24 npm billion-context

The official billion-context plugin: a context-compression plugin for small context windows (a 100K context is enough), token savings (5x fewer tokens), and month-long single sessions (billions of tokens).

Install

# from npm (prebuilt)

dsh plugin --profile web add billion-context

# from GitHub (first run asks for allowBuilds approval — follow the hint, retry)

dsh plugin --profile web add github:ranxianglei/billion-context

Any plugin you install runs third-party code with your own permissions — it can read your files, use your credentials, and reach the network, and tool approvals don’t sandbox it. GitHub-sourced plugins also run build scripts at install time — pnpm blocks those until you allow them, so an install can stop with ERR_PNPM_GIT_DEP_PREPARE_NOT_ALLOWED or ERR_PNPM_IGNORED_BUILDS; dsh prints the exact key to add under allowBuilds in your profile’s pnpm-workspace.yaml, and the install works on the next run. Allowing a build is a trust decision: only install sources you trust, and pin a commit (github:owner/repo#sha).

README


Cache health at a glance: a healthy session keeps a 96–99% prefix-cache hit rate — compression itself costs ≤2%. Sustained lower? Check attribution with /acp or /acp-cache (see FAQ); usual causes, in order: upstream cache TTL expiry · model switch · a bili bug (please report) · other/unknown.

Community

QQ Group: 1056132097 (full) 1108730198 (full) 1129868472 (open)


📄 Paper / Preprint

📝 The paper itself is open-sourced under the MIT License as part of the codebase (paper/). It is a living document — anyone may edit it; improvements are welcome via pull request.

A production-scale longitudinal study: 4.5 months, three hosts, 174,327 model calls, 18.76B cumulative input tokens (~24.7B across all hosts), zero window violations on 204,800-token models, marathon sessions of 8,584–12,049 calls.

📊 Independent Third-Party Benchmarks

  • 1,500-Item Cross-Benchmark Evaluation (LongMemEval-S · ContextBench · MemoryAgentBench) (中文版) — head-to-head comparison of billion-context (v0.1.202) against full context and 3 RAG arms (bm25, embedding, embedding-rerank) with DeepSeek V4.1 Flash, Qwen3-Embedding-8B, Qwen3-Reranker-4B, and Pi Agent: #1 on ContextBench (0.3097 Utilized F1, -58.3% cost vs full), +16.6% over best RAG on LongMemEval-S (79.20% vs 62.60%, -71.1% independent-session cost vs full), and 100% survival on 1.48M-token MemoryAgentBench TTL (30× steady-state compression where full fails 199/199).

billion-context sits between any agent and its model API, rewriting Anthropic/OpenAI streams with acp-kernel compression. The model decides when and what to compress into high-fidelity summaries — not a hard truncation limit.

Why

Long coding sessions blow up context. Each provider charges per token, and once you pass the context window the session degrades or dies. billion-context compresses consumed conversation into layered summaries so you can run a single session for days — billions of tokens through one context window.

Unlike a host's built-in summarizer, compression here is incremental, reversible, and prefix-cache friendly: summaries are written in small ranges, can be decompressed on demand, and the cache prefix stays intact.

How it works

How it works

Context-management tools

The proxy injects four context-management tools into the conversation; the model calls them itself as context grows, and the proxy executes compress server-side so folded ranges stay summarized in history until restored:

  • compress — fold a message range into a detailed summary.
  • decompress — restore a compressed range when exact details are needed again.
  • search_context — keyword search over compressed summaries and visible messages.
  • acp_status — context-usage overview plus which ranges are still compressible.

Which do I need?

Pick by your client:

Client Use
pi billion-context — bili pi (launcher) or bili plugin install pi (native); standalone billion-context-pi remains usable — details: docs/clients.md
opencode (1.x / 2.x) billion-context — bili opencode (launcher) or bili plugin install opencode (native); standalone opencode-acp remains usable on 1.x — full guide: OpenCode
omp billion-context via bili omp (built-in plugin) or bili plugin install omp (self-spawning native plugin, no launcher)
dsh bili dsh (launcher — full native plugin via --patch) or bili plugin install dsh ≡ dsh plugin --profile <name> add billion-context (one unified lane) — details: docs/clients.md
kimi bili plugin install kimi (self-spawning native, Kimi Code ≥ 2.0.0) or bili kimi (cert-MITM) or /bili/ prefix — details: docs/clients.md
hermes bili plugin install hermes (self-spawning native, Python plugin #958) or bili hermes (cert-MITM)
zcode (Z.ai / bigmodel coding plan) bili plugin install zcode (self-spawning native, #1145) or cert-MITM via the GUI's Settings → Network or /bili/ prefix — details: docs/clients.md
claude bili claude (launcher) or bili plugin install claude (native posture, #964 — managed settings block + session-owned proxy; see the notes below)
codex bili codex (launcher — the full zero-config posture) or bili plugin install codex (MCP-shell tools companion: start bili first — the shell never spawns a proxy and never routes codex's own traffic) — details: docs/clients.md
jcode billion-context via bili jcode (cert-MITM) or /bili/ prefix — no native mode (compiled Rust binary, no plugin seam, #962)
gemini (Gemini CLI) bili gemini (launcher, GOOGLE_GEMINI_BASE_URL /bili/ rewrite) or /bili/ prefix — launcher-only (no in-loop tool seam, #1043)
iflow (iFlow CLI) bili iflow (launcher, IFLOW_BASE_URL /bili/ rewrite) or /bili/ prefix
qwen (Qwen Code) bili qwen (launcher, cert-MITM) or /bili/ prefix
antigravity (Antigravity CLI / agy, Google) bili antigravity (launcher, CLOUD_CODE_URL /bili/ rewrite of daily-cloudcode-pa.googleapis.com — Antigravity's real default upstream, #3129) or /bili/ prefix — no plugin seam (closed Go language_server; its user-plugin surface is additive-only, #2115) — details: docs/clients.md
mcode (MiniMax Code) billion-context via bili mcode (cert-MITM) or /bili/ prefix — no native mode (event hooks only, no model-request seam, #1050)
aider billion-context via bili aider (cert-MITM) or /bili/ prefix — no native mode (shell-command-only hooks, no tool-injection seam, #1048)
copilot (GitHub Copilot CLI) bili copilot (launcher, cert-MITM) — closed Go binary, no plugin seam (#1049)
amp (Amp CLI) bili amp (launcher, cert-MITM) — closed Go binary, no plugin seam (#1049)
kiro (Kiro CLI, AWS) bili kiro (launcher, cert-MITM) — Rust shell + Node agent server, no plugin seam; model traffic is the Responses wire on runtime.<region>.kiro.dev, gateway quirks handled built-in — details: docs/clients.md
crush (Charm Crush) bili crush (launcher, cert-MITM) — open-source Go binary, no plugin seam; built-in provider hosts whitelisted, custom base_urls auto-discovered from crush.json (#2340)
zed (Zed editor) bili zed (launcher, cert-MITM) — open-source Rust editor, no plugin seam; reqwest honors HTTPS_PROXY, CA via SSL_CERT_FILE (Linux env probing); built-in provider hosts whitelisted, custom api_urls auto-discovered from settings.json; loopback providers (ollama/lmstudio) stay direct via NO_PROXY (#2340)
goose (Goose CLI) bili goose (launcher) — rustls trusts no CA file, so no cert-MITM: openai/anthropic legs via OPENAI_HOST/ANTHROPIC_HOST, custom providers via a regenerated GOOSE_PATH_ROOT overlay (#1049)
everything else (no context hook) billion-context — bili <client> (launcher, preferred) or /bili/ prefix

Native mode vs standalone extensions. The host-native plugins (bili plugin install …) and the standalone in-process extensions (billion-context-pi, opencode-acp) are mutually exclusive — both active means double compression. The installer makes the switch: it replaces the legacy entries (bare name, npm: alias, versioned, path form; array or object shape) and snapshots the original config to .bili-bak once; a project-local install is not touched — remove that one by hand. As a runtime safety net for manual installs, the native entries set BILLION_CONTEXT_NATIVE=<host> synchronously at load so a standalone extension can stand down at action time. On the pi side the marker needs billion-context-pi 0.1.72+, and the pi-native entry scans both pi settings files once its proxy is up and warns loudly when it spots a co-resident legacy entry the installer never saw — that warning is the only visible signal while an old billion-context-pi silently double-compresses.

Install

Linux / macOS — install with a user-level prefix (no sudo, no npm config changes, and bili's self-update never hits permission errors):

npm install -g billion-context --prefix=~/.local

The bili command lands in ~/.local/bin — already on PATH in most distros; if not, add export PATH="$HOME/.local/bin:$PATH" to ~/.bashrc or ~/.zshrc. On nvm or Homebrew Node the default prefix is already user-owned — a plain npm install -g billion-context works as-is. On Windows the default prefix (%APPDATA%\npm) is also user-writable — plain npm install -g billion-context.

This installs the bili command (bili-proxy is kept as an alias). Hitting EACCES with an old root-owned prefix? Reinstall with --prefix=~/.local (pass the flag again on any future npm reinstall of bili) — that is the permanent fix; avoid sudo.

Prefer containers? A Dockerfile ships at the repository root — see Running in Docker below.

Quickstart

Three ways to use it — pick one:

  • Native plugin (most native): bili plugin install <client> — bili becomes a plugin inside the client; start the client as usual.
  • Launcher (config-free): one bili <client> command brings up the proxy and the client together — no real config file is ever touched.
  • URL change (most universal): prefix your client's baseURL with the proxy origin + /bili/.

Mechanism details behind these three options (plugin lifecycle, runtime-info protocol, injection priority) live in docs/technical-notes.md.

Ports, briefly (#1660): bili start (manual) owns 8787. Everything a lane spawns for you (native hooks, launcher lanes) lives in a separate self-managed zone starting at 18787 — collisions hop +1 and each lane remembers its drift, so zero-config installs never fight you for a port, and a deliberate bili start daemon is attached by default. Attach is lane-aware (#2905): a hook rides its own lane's instance first, NEVER another declared lane's — except lanes you list in native.attachLanePriority — and the 8787 user-zone daemon stays the LAST-resort fallback. An upgrade-restart that finds the previous build still draining on the lane's port waits for it to release (up to 5s) and rebinds the SAME port instead of drifting (#1723); only a genuinely occupied port hops +1 — and that hop is now logged loudly.

Option 1 — Native plugin (bili plugin install pi / omp / opencode / dsh / kimi / hermes / zcode)

The proxy lives inside the client: install once, then start the client exactly as you always do — no launcher command, no env vars, no fixed port, no URL edits. Supported today for pi, omp, opencode (1.x and 2.x), dsh, kimi, hermes and zcode:

bili plugin install pi          # registers a "billion-context" entry in pi's settings (npm form when bili itself was npm-installed)
bili plugin install omp         # registers an extensions entry in omp's config.yml (~/.omp/agent/config.yml)
bili plugin install opencode    # registers the plugin in opencode's real config + disables native auto-compaction
bili plugin install dsh         # runs 'dsh plugin --profile <name> add billion-context' for every existing profile
bili plugin install kimi        # writes $KIMI_CODE_HOME/plugins/managed/billion-context/kimi.plugin.json (+ installed.json record); per-session routing block lands in config.toml on first start (Kimi Code >= 2.0.0)
bili plugin install hermes      # copies the Python plugin into ~/.hermes/plugins/billion-context/ (+ machine-owned bili.json sidecar) and enables it via `hermes plugins enable billion-context`
bili plugin install zcode       # writes hooks.enabled + a SessionStart hook + mcp.servers.bili into ~/.zcode/cli/config.json; per-session routing lands in the bigmodel provider store on first start
bili plugin remove <client>     # undo (dsh removes through the same channel; config snapshots go to .bili-bak)

Where a client has its own plugin channel you can also install natively, skipping bili commands entirely:

  • dsh: dsh plugin --profile <name> add billion-context is the very command bili plugin install dsh drives per profile (npm form only; a git checkout has no published entry — pasting a GitHub address installs source without dist/ and the bundle never loads) — same end state either way (pnpm into the profile, bundled patch layer mounted by dsh itself); remove through the same channel. See the dsh section below.

  • opencode: easiest is bili plugin install opencode (it picks the right entry form for your host version). Hand-install without bili (npm form only; a git checkout has no published entry):

    • OpenCode 1.x — add the bare npm name to your real config's plugin list: "plugin": ["billion-context"]. The package publishes exports["./server"] → dist/agent/opencode-native.js, so opencode loads it through its own Npm.add machinery and the plugin self-spawns exactly like the bili-installed form.

    • OpenCode 2.x — the bare name does NOT work: 2.x resolves it through the package main field (dist/index.js, the CLI — no plugin definition), so the module load fails (#3976). Instead, put a one-line re-export in a directory and point the config's plugins list at that directory:

      // <shim dir>/index.js
      export { default } from "<abs path to the global install>/dist/agent/opencode-native.js";
      
      { "plugins": ["<shim dir>"] }
      

    Either generation: do the two things the bili installer would have done for you too — set "compaction": { "auto": false } in the same config (otherwise OpenCode's native auto-compaction double-compresses) and keep a manual backup of the file first.

  • claude: this repository doubles as a Claude Code plugin marketplace — /plugin marketplace add ranxianglei/billion-context, then /plugin install billion-context@billion-context, then run /billion-context:bili-setup (it drives bili plugin install claude for you and tells you to restart). Same end state as the bili installer; the plugin ships no hooks or MCP entries of its own, so nothing double-registers.

For pi / omp / kimi / claude there is no client-side channel — bili plugin install <client> writes their config entries for you (kimi's declarative kimi.plugin.json + registry record, claude's managed settings block, …).

Notes:

  • Native mode is mutually exclusive with the standalone in-process extensions (billion-context-pi, opencode-acp) — the installer swaps the entries and snapshots the original config (.bili-bak).
  • pi: native installs wire in built-in subagents by default (acp_delegate / acp_delegate_wait / acp_delegate_cancel) — how to disable them while keeping context compression: Pi built-in subagents.
  • OpenCode legacy sessions, V1/V2 shapes and caveats: OpenCode.
  • kimi reports runtime-info at bootstrap only (static headers can't carry per-request window/model values); subagent tool calls are routed by the proxy's outbound tool-use witness ring (#1685) — no model-visible conversation id.
  • hermes's native plugin is Python: it points hermes' httpx stack at the proxy via env vars after a health check and stamps per-request headers through an llm_request middleware.
  • codex is the one client a plugin install cannot make self-sufficient: codex routes model traffic via env only (no config-file routing seam for the default ChatGPT-login provider — a managed model_providers block would force API-key auth and drop subscription login), and an MCP server cannot inject env into its parent. bili plugin install codex writes a single [mcp_servers.bili] block into ~/.codex/config.toml (command = node, args = dist/mcp.js) exposing the four ACP tools; at session start the shell resolves a proxy — env BILI_MCP_PROXY > the live-instance record (any lane's proxy or a bili start daemon) > the 8787 user-zone default (#1660 removed the install-time origin bake, #403) — nothing reachable → tools/list fails with -32003. So: start bili first (bili start or any client's lane proxy), export HTTPS_PROXY yourself if you also want compression, or use bili codex for the zero-config full posture. Mechanics: docs/clients.md.
  • claude has a native posture (#964): managed settings block + SessionStart hook + MCP shell; the hook rides the self-managed port zone (#1660) and re-pins the managed URL to the live origin each session, so port drift self-heals. Since #1821 the MCP shell also watches the proxy mid-session (30 s tick × 3 failures) and respawns a dead lane proxy itself, so model requests recover within ~90 s instead of a new session (same semantics as the zcode lane). Opt out with BILI_NATIVE_CLAUDE=0 (passthrough). Mechanics: docs/technical-notes.md. Known limitation (#2290): the Claude Desktop app's Code tab (CLAUDE_CODE_ENTRYPOINT=claude-desktop) sets its own ANTHROPIC_BASE_URL on its embedded Claude Code, overriding the managed one — desktop sessions bypass the proxy entirely while the managed block's DISABLE_AUTO_COMPACT=1 still applies, so they get neither bili compression nor native auto-compaction. The hook warns loudly at session start (and records it for bili doctor, which also flags machines where Claude Desktop is installed alongside the native lane); terminal sessions are unaffected. Field-tested 2026-10-06 (#2290): putting HTTPS_PROXY in the settings env block produced zero CONNECT traffic to a live MITM-enabled proxy on Claude Desktop 2.19675.1 / embedded CC 2.1.288 (Windows 11) — the variable reaches no network stack there, matching the CLI behavior of that era — note that newer builds (CC 2.1.301+) DO honour HTTPS_PROXY on the CLI, which is why bili claude routes via the /bili/ base-URL rewrite — the launcher gives this lane no proxy and no CA at all, and a child that brings a proxy of its own gets the loopback hosts merged into NO_PROXY/no_proxy so that leg stays direct regardless of its proxy behavior (#3446). As of that version there is no usable routing seam on the desktop Code tab; the lane stays tracked in #2290 in case upstream changes behavior. Account-bound features (#4693): because the lane routes via a custom ANTHROPIC_BASE_URL, Claude Code treats the session as third-party/API-key and disables features bound to your Anthropic account/Cloud — Remote Control (/remote-control, driving a local session from mobile/desktop/web) disappears from the command list while bili routes traffic. Compression is unaffected; run plain claude without bili (or bili plugin remove claude) for sessions that need those features. This is inherent to the base-URL routing seam — no alternative interception point survives CC's undici stack (#2290) — not a config deletion. Desktop-only users can restore native auto-compact with bili plugin remove claude. Since #5095 the managed block also installs a SubagentStart hook (node <root>/dist/claude-subagent-hook.js) that mints a per-dispatch lane token into each subagent's context: subagents that ride the parent's session id with no agent-id headers (the VS Code extension build, #4352) key onto their own <id>|sub:…:t:<token> compression lanes, so parallel same-type dispatches stop merging (and the token disappears from later requests once the first message folds — those fall back to the fingerprint lane). Uninstall removes the hook with the managed block.
  • zcode has a native posture (#1145): managed ~/.zcode/cli/config.json block + per-session provider baseURL rewrite. Full mechanics: docs/clients.md.
  • jcode and aider have no native mode (no plugin/MCP/tool-injection seam: #962, #1048) — use bili jcode / bili aider.
  • copilot, amp and goose are launcher-only (#1049); goose cannot be cert-MITMed (rustls trusts no CA file) and rides plain-HTTP base-URL redirects instead.

Option 2 — Launcher (bili pi / bili codex / bili claude / bili omp / bili opencode / bili hermes / bili dsh / bili codebuddy / bili qoder / bili trae / bili jcode / bili kimi / bili gemini / bili iflow / bili qwen / bili antigravity / bili mcode / bili aider / bili copilot / bili amp / bili goose / bili kiro)

The launcher wraps a client in one command: it starts a proxy on an independent port (a fresh instance is always spawned — a port is never reused), then points the client at it — certificate-based MITM where the client honors proxy/CA env vars, or an isolated /bili/ config rewrite where it doesn't. No real config file is ever edited; the client's own config is READ to discover which HTTPS upstream hosts it talks to, and those hosts are whitelisted for MITM so the proxy can TLS-terminate exactly them and blind-tunnel everything else.

bili pi                               # launch pi through the proxy — file-free (#535): env + extension registerProvider, real ~/.pi untouched
bili codex                            # launch codex through the proxy
bili claude                           # launch claude through the proxy
bili omp                              # pi-style, file-free (#535): env + extension registerProvider + compaction cancel, real ~/.omp untouched
bili opencode                         # OpenCode (1.x & 2.x): full guide in the [OpenCode](docs/clients.md#opencode) section below
bili hermes                           # file-free (#535): hermes proxy env (HTTPS_PROXY + combined CA bundle via SSL_CERT_FILE) — https via CONNECT MITM, http via absolute-form forward proxy; real ~/.hermes untouched
bili dsh                              # deepseek-harness: full native plugin injected via --patch (#941) — real dsh tools, session-bound /acp + /acp-cache (plugin mode); non-loopback upstreams ride proxy envs, loopback keeps the overlay DSH_HOME rewrite (#535); dsh auto-compaction off everywhere — host plane via the bundle patch, web-profile presets at load time (#1772/#2474/#3354)
bili codebuddy                        # Tencent CodeBuddy Code CLI: CODEBUDDY_BASE_URL /bili/ rewrite (OpenAI chat completions wire), budget aligned via CODEBUDDY_AUTO_COMPACT_WINDOW; real ~/.codebuddy untouched
bili qoder                            # qoder CLI (`qoder`/`qodercli`): model endpoint is hardcoded https (no /bili/ rewrite possible) — cert-MITM via HTTPS_PROXY + NODE_EXTRA_CA_CERTS, default model hosts whitelisted (#653). Qoder Desktop is a separate BYOK/custom channel that speaks plaintext OpenAI over a `/bili/` prefix — point it at bili as a normal custom provider (not this MITM lane); its official-subscription traffic is encrypted and out of scope (#764)
bili trae                             # Trae CLI (ByteDance, closed Go binary, no base-URL override) — cert-MITM via HTTPS_PROXY + SSL_CERT_FILE, model host from TRAE_CLI_API_HOST or the default enterprise gateway (#655)
bili jcode                            # jcode (Rust agent harness) — env-only cert-MITM launch: HTTPS_PROXY + SSL_CERT_FILE, model host api.z.ai whitelisted, local loopback providers stay direct via NO_PROXY
bili kimi                             # Kimi Code CLI (Moonshot): standard proxy envs except an unconditional loopback bypass — https via cert-MITM, http via absolute-form forward proxy; loopback endpoints inventoried with a manual /bili/ hint (#757)
bili gemini                           # Gemini CLI (Google): GOOGLE_GEMINI_BASE_URL /bili/ rewrite to generativelanguage.googleapis.com (Google native wire), real ~/.gemini untouched
bili iflow                            # iFlow CLI: IFLOW_BASE_URL /bili/ rewrite to apis.iflow.cn/v1 (OpenAI chat-completions wire), real ~/.iflow untouched
bili qwen                             # Qwen Code (multi-protocol gemini-cli fork, no base-URL hook): cert-MITM via HTTPS_PROXY + NODE_EXTRA_CA_CERTS, default DashScope/Qwen model hosts whitelisted, custom relays via --mitm-domain
bili antigravity                      # Antigravity CLI (agy, Google): CLOUD_CODE_URL /bili/ rewrite of daily-cloudcode-pa.googleapis.com — the real default upstream (bare cloudcode-pa 400s on /v1internal:*, #3129); undocumented language-server env override, v2.19.1 binary-verified; wire recognized as Google native by path (#2115)
bili mcode                            # MiniMax Code CLI: same shape as kimi (proxy envs, unconditional loopback bypass, cert-MITM/absolute-form); session bound via X-Mavis-Session-Id (#1050)
bili aider                            # Aider (Python pair programmer): cert-MITM via HTTPS_PROXY + SSL_CERT_FILE/REQUESTS_CA_BUNDLE; endpoint from OPENAI_API_BASE / ANTHROPIC_BASE_URL / --openai-api-base / .aider.conf.yml (#1048)
bili copilot                          # Copilot CLI (GitHub, closed Go binary) — cert-MITM via HTTPS_PROXY + SSL_CERT_FILE, api.githubcopilot.com + per-plan subdomains whitelisted (#1049)
bili amp                              # Amp CLI (Sourcegraph, closed Go binary) — cert-MITM via HTTPS_PROXY + SSL_CERT_FILE, ampcode.com whitelisted (#1049)
bili goose                            # Goose (Block, Rust/reqwest): rustls release builds trust no CA file — openai/anthropic legs via OPENAI_HOST/ANTHROPIC_HOST, custom providers via a regenerated GOOSE_PATH_ROOT overlay (/bili/ rewrites, real config untouched) (#1049)
bili kiro                             # Kiro CLI (AWS): cert-MITM via HTTPS_PROXY + combined CA on SSL_CERT_FILE and NODE_EXTRA_CA_CERTS; only runtime.<region>.kiro.dev is decrypted; developer/system items re-voiced as user for the gateway; session = Kiro's conversation id (survives --resume)
bili pi --mitm-domain api.foo.com     # add a domain to the MITM whitelist

Option 3 — URL change (/bili/ prefix)

Start the proxy:

bili

Then just prefix your client's existing baseURL with http://localhost:8787/bili/. The full upstream URL is embedded in the path, so the proxy knows where to forward without any config:

client baseURL before:  https://api.openai.com/v1
client baseURL after:   http://localhost:8787/bili/https://api.openai.com/v1

For more per-client configuration examples, see the web UI guide at http://localhost:8787.

Verify. With the proxy running and your config saved, check it answers and that your first real request shows compression activity in the log:

# Health check (proxy up + where it forwards)
curl -s http://localhost:8787/__bili/health
# → {"ok":true,"upstream":"https://api.anthropic.com"}

# Live session stats (after a real request)
curl -s http://localhost:8787/__bili/stats

Then send one message from your client and watch the log (~/.local/state/billion-context/bili.log, also printed to stderr). You should see a processTurn line per request, and once the conversation grows, [acp-usage] round N input=X cached=Y (cache hit Z%) + a compress event.

Client deep dives

Everything that doesn't fit in one Quickstart line — how each client's lanes attach, what gets written where, and known limitations — lives in docs/clients.md: dsh · Kimi Code · Hermes · ZCode · Gemini family (Gemini CLI / iFlow CLI / Qwen Code / Antigravity) · cert-MITM clients that never compress (CONNECT blind tunnels, #897) · unrecognized endpoints going direct (#1290) · OpenCode (launcher / native / pure proxy, /acp status & rules, legacy opencode-acp sessions #920).

From source (development)

Risk: master is the development branch and can be broken at any moment — unreleased changes, failing builds, unverified fixes. Unless you are patching the code or testing an unreleased fix, install a release with npm as above.

git clone https://github.com/ranxianglei/billion-context.git
cd billion-context
npm ci
npm run build
npm link
bili --version

After npm link, bili works from any directory.

FAQ

How do I check my cache hit rate? Don't dig through logs — /acp-cache prints a text summary report right in the client, headed by a clickable Web UI link: open it for the web version of the session page — the cache hit-rate line chart plus per-break attribution:

web session page: cache hit-rate chart + attribution The report has four blocks that pin things down at a glance: GRAND LEDGER (totals + hit% with an explicit HEALTHY verdict; misses decomposed into new content / compress re-pay / upstream-ttl-or-client-rewrite) · FOLD ECONOMICS (per-fold economics: net tokens saved, paid-back verdicts) · LINE ITEMS (anomalies only: hit<85% or miss≥5000). Rule of thumb: compression itself costs ≤2% — a healthy session sits at 96–99%. When you see less, the attribution tells you which of the usual suspects it was, in this order: ① upstream cache TTL expiry (shows up as stable-prefix misses — the top-spikes line names idle times), ② a model switch, ③ a bili bug (report it with the page attached), ④ other/unknown. /acp-cache [full] lists every fold & line; same report over Since #1535 the report ends with a MODEL SWITCHES section: a mid-session model change invalidates the provider's prefix cache, so the whole stable prefix is re-billed on the next request — each switch's unexplained residual (its ttl bucket minus new content) is charged to the switch instead of masquerading as TTL expiry; per-event from → to, hit %, and attributed tokens are listed (full lists every event, the summary the last 8), and the web sessions table gains a matching model-switch column. Since #2131 the same machinery fingerprints the outbound credential (a 12-hex sha256 of the authorization/x-api-key-family headers — the raw key is never stored) and attributes a key switch — the relay behind a stable URL rotated to a different account — in a dedicated KEY SWITCHES section, a key switch: line in CACHE INVALIDATION, and a 🔑 badge in the web sessions column. HTTP: GET /__bili/cache-report; the raw per-request [acp-usage] lines still land in the log file for deep dives.

What does /acp show? In clients with the native plugin (opencode, dsh, hermes), /acp renders the ACP status panel of the current conversation straight from the proxy (session, blocks, compressible ranges, usage); before the first model request it shows an idle notice instead. /acp-cache [full] prints the cache report above.

Can I query sessions and config over the web? Yes — open http://localhost:8787: an overview dashboard, session list with per-session detail, live logs, a config editor, and an upstream connectivity test. A Blog view renders notes and long-form articles inside the panel itself — bundled with the install, refreshed from GitHub master on a ~1-hour throttle while online (fail-soft to the bundled copy when offline); the full library also lives on GitHub Pages. Everything else is also plain JSON for scripting (/__bili/stats, /__bili/sessions, /__bili/config, …).

When does compression happen? It is model-driven: the injected context tools are called by the model as context grows, gentle growth nudges (~50K-token steps by design, adjustable via compress.nudgeGrowthTokens) prompt it along the way, and preflight fires as a hard backstop when the input alone exceeds the window (#470). Watch it live with /acp or the web UI.

Does compression shrink my client's own conversation file? No — the most common surprise in very long sessions (#5796). Compression shrinks what is sent to the model: every outgoing request is rebuilt from live blocks plus summaries, so the wire stays small however long the conversation runs. The client's stored history is user data — bili never rewrites it. On dsh the host's auto-compaction is additionally disabled while bili is active (two compressors must not fight over one history, #1772/#2474/#3354), so the host-side file grows with every turn: each turn takes longer as it grows (measured: tens of seconds per turn at ~30K messages, #5796), and the practical convergence path is to start a new session when it gets heavy. The web session list badges sessions at 5,000+ requests (长会话/slow) as a reminder that this point has been reached.

Why does the first compaction wait until ~200k? Compaction does not trigger on absolute window position but on growth intervals: by default the first soft compaction fires 50k tokens past the boot content (compress.nudgeGrowthTokens, flat and window-independent). With a boot around 100k — or with the growth step set to ~100k — the first compaction may wait until ~200k. To make it fire earlier: 1) trim the system prompt, disable unneeded tools, prune skills; 2) lower compress.nudgeGrowthTokens to ~50k; 3) enable lean mode. Worked example in docs/configuration.md.

Is bili transparent? How do I turn it off? Unrecognized endpoints forward unchanged (docs/clients.md), and every mode reverses cleanly: bili plugin remove <client> for native installs, stop using the launcher command / env vars / /bili/ prefix for the other two — traffic goes direct again immediately.

Can I combine bili with a model-routing gateway (e.g. magpie)? Yes — point the client at bili, and point bili at the gateway through the /bili/ prefix, which embeds the gateway's absolute URL: http://localhost:8787/bili/http://127.0.0.1:3425…. Example: Claude Code → ANTHROPIC_BASE_URL=http://localhost:8787/bili/http://127.0.0.1:3425 + ANTHROPIC_API_KEY=magpie; OpenAI-style clients → OPENAI_BASE_URL=http://localhost:8787/bili/http://127.0.0.1:3425/v1 (+ OPENAI_API_KEY=magpie). Compression happens before routing — every request that reaches any account in the gateway's routing groups is already compressed. Credentials stay in the gateway: the client's dummy key rides through untouched, and real keys never touch bili. Caveats: ① an account switch mid-conversation invalidates the provider's prefix cache — inherent to multi-account routing, not bili (watch it in the KEY SWITCHES section of /acp-cache); ② routed model names are opaque to bili's window lookup — declare their context windows under the providers key http://127.0.0.1:3425 via models.<name>.context, or they run on the fallback window (route-miss warning, #2317); ③ if the relay rejects bili's Anthropic-lane cache_control stamps, set compat.noCacheControl; ④ loopback gateways work out of the box; a gateway on a private non-loopback address may additionally need network.allowPrivateHttpProviders (#2984); ⑤ for Claude Code native plugin mode (bili plugin install claude), set BILI_CLAUDE_UPSTREAM=http://127.0.0.1:3425 before install so the managed block wraps the relay into ANTHROPIC_BASE_URL (or configure ANTHROPIC_BASE_URL=http://127.0.0.1:8787/bili/http://127.0.0.1:3425 directly; localhost and 127.0.0.1 both accepted, #5369). Verified end-to-end over all three wires (OpenAI Chat / Responses / Anthropic Messages, including cross-protocol conversion), #4433.

Does this work with opencodex (ocx) — the local Codex/Claude Code gateway with a ChatGPT account pool? Same pattern: point the client at bili, and point bili at ocx through the /bili/ prefix — OpenAI-style clients (chat completions and Responses, e.g. Codex) → OPENAI_BASE_URL=http://localhost:8787/bili/http://127.0.0.1:10100/v1; Anthropic-style clients → ANTHROPIC_BASE_URL=http://localhost:8787/bili/http://127.0.0.1:10100. A dummy key value satisfies clients that require one; it rides through untouched. Keep bili in front of ocx rather than behind it: ocx owns the provider credentials and the account pool at its final hop, so only this order keeps compression and pooling working together. Loopback needs no key between the two (ocx's default loopback bind reads no token); a non-loopback bind takes OPENCODEX_API_AUTH_TOKEN plus the x-opencodex-api-key header per its docs. Request models by ocx's provider/model id so routing is deterministic; as with any gateway, routed ids are opaque to bili's window lookup (declare them via models.<name>.context) and a mid-session account rebind invalidates the prefix cache (watch KEY SWITCHES in /acp-cache). Verified end-to-end over all three wires (OpenAI Chat / Responses / Anthropic Messages), bili 0.1.212 × ocx 2.97.0.

Where are logs and session data stored? Log: ~/.local/state/billion-context/bili.log (also mirrored to stderr); session state: ~/.local/share/billion-context/ (XDG-overridable; Windows AV exclusion #362, opt-in cleanup #1082) — full paths in docs/configuration.md.

Running the proxy

Running in Docker

A Dockerfile ships at the repository root — it installs the published npm package at build time (the tested artifact; nothing is compiled in the image) and runs the proxy as a service:

docker build -t billion-context .
docker run -d --name bili -p 8787:8787 \
  -v bili-config:/root/.config/billion-context \
  -v bili-data:/root/.local/share/billion-context \
  -v bili-cache:/root/.cache/billion-context \
  -v bili-state:/root/.local/state/billion-context \
  billion-context

Then point clients' baseURL at http://<host>:8787/bili/… exactly like Option 3 above.

  • Base: node:24-slim — uses the builtin node:sqlite storage engine directly (no native-addon download). Don't switch to Node 20 bases: the self-downloaded SQLite engine requires Node ≥ 22 and crashes under 20 (#3341).
  • Updates: in-container self-update is off by default (ACP_AUTO_UPDATE=0) — the sanctioned update channel for a containerized copy is rebuilding/pulling a newer image; an in-place self-update would be silently reverted to the image version on the next recreate. Opt back in with -e ACP_AUTO_UPDATE=1; pin a release at build time with --build-arg BILI_VERSION=0.1.197 (default latest).
  • Data: the four mounted dirs hold config, session state, update throttle/lock + cached SQLite engine, and logs respectively. Mount volumes or bind mounts so they survive container replacement; the image also declares them as VOLUMEs so even a bare docker run keeps them out of the container layer.
  • Security: the container listens on 0.0.0.0:8787 so clients outside it can connect — there is no authentication, so expose the port only on a trusted network (same caveat as --host 0.0.0.0 below). The web UI and /__bili/* JSON endpoints work unchanged through the mapped port.
  • Healthcheck: the image ships a HEALTHCHECK probing /__bili/health on 8787 via node's builtin fetch (no curl needed in the slim base); if you override the listen port, remember the probe stays pinned to 8787.

Flags

bili --port 9000              # change listen port
bili --host 0.0.0.0           # listen on all interfaces (see host note below)
bili --debug                 # verbose logging (also: set "debug": true in config)
bili --passthrough           # forward without compression (smoke-test mode)
bili --config ~/my-bili.json # use a different config file
bili -c dsh-test             # named config → ~/.config/billion-context/dsh-test.json (per-model files)
bili update                  # check & install a newer version now (bypasses throttle)
bili doctor --fix            # converge drifted dsh profile ledgers (manifest spec / lockfile record / bundle copy) to the on-disk version (#3496)
bili --no-auto-update        # disable self-update for this run
bili --auto-restart-on-update   # self-restart when a new version is installed (default off)
bili --bin /path/to/client   # launcher: spawn this exact client binary instead of auto-detecting (env BILI_CLIENT_BIN)

Flags override env vars and the config file. bili --help lists them all.

Remote agents (--host)

By default the proxy binds 127.0.0.1 and only accepts loopback connections. To serve agents on other machines: bili --host 0.0.0.0 (or your LAN IP); remote agents point their model baseURL at http://<this-host>:<port>/bili/….

  • MITM-mode CONNECT then also accepts remote clients — for whitelisted model hosts only. Blind tunnels to arbitrary hosts stay loopback-only, so the proxy can never be used as an open relay.
  • /bili/<absolute-url> destination admission (#409): the proxy itself and link-local/metadata addresses are always denied; loopback/private destinations are allowed for local clients (self-hosted upstreams) and denied for remote clients unless listed in BILI_TUNNEL_ALLOWED_HOSTS (host or host:port, comma-separated). One exception (#1073): a local client relaying a management path (/__bili/*, /__acp/*) to a loopback IP-literal destination is forwarded unmarked; remote peers and hostname destinations keep the internal tunnel marker unconditionally, and management paths stay unreachable through the tunnel even via NAT hairpin. An absolute-form request addressed to the instance's own endpoint on a management path is served locally instead of tunneled (a forward-proxy-style health probe gets a real answer).
  • There is no authentication: only do this on a trusted LAN or behind a firewall. The /__bili/ management endpoints remain loopback-only. A startup [security] warning reminds you of the above.

Debugging

bili --debug (or env ACP_DEBUG=1, or "debug": true in config — flag > env > config) logs every processTurn (tag counts, token usage), the nudge decision (growth/usage/pendingT1/shouldInject), client headers, and SSE rewrites.

Log file

All logs tee to ~/.local/state/billion-context/bili.log by default (XDG state dir) and still print to stderr. Override with "logFile" in config or ACP_LOG_FILE (off disables the file). Auto-rotates at 10 MB (bili.log.old). Per-request

…

Content from the project README on GitHub ↗

Links

More in this category

View the whole category →

Community comments

Comments are public GitHub Discussions. Loading them connects to GitHub and Giscus; a GitHub account is required to post.