Context compression

Overview

Every turn, an AI coding agent re-sends its whole conversation to the model — the test log it already read, the JSON payload it already parsed, the grep output it already acted on. You pay for that context again on every step.

Vibgrate CLI compresses that context before it reaches the model and keeps every original retrievable on your machine, so the model can pull back exactly the lines it needs instead of re-paying for all of them.

Compression is not a second product with its own commands. It is a mode of the local runtime you already start (vg serve), a flag on the installer you already use (vg install), and a section of the report you already read (vg savings). Inside vg code it is on by default.

vg serve --compress          # serve the map and compress context, one process
vg install claude --compress # point Claude Code at it (undo: vg uninstall claude)
vg savings                   # what it saved: today / 7 days / 30 days

Savings depend on the workload and are reported, not promised: vg savings --benchmark gives you a reproducible number on your own machine.

How a request is compressed

The pipeline walks each message and each content block and decides per block:

1. Exclusions first. System prompts, user text, assistant text, blocks already carrying a marker, retrieval results, error outputs, file reads and edits are left alone by default. In cache mode only the newest turn is eligible, so your provider's prompt cache keeps hitting; token mode makes every eligible block a candidate. The most recent messages keep their code intact.
2. Route by shape. A detector classifies the block — JSON, source code, search results, build output, diff, HTML, tabular, structured config, plain text — and hands it to the compressor built for that shape.
3. Lossless first. Repeated lines, grep and directory headings, path headings, diff index lines and config boilerplate fold into byte-reversible markers. Lossy compression runs on top only when it beats the fold by a clear margin.
4. Lossy, shape-aware. JSON arrays keep a head / middle / tail sample plus every error-looking item and the rows that match your question. Logs collapse near-duplicates and keep errors, stack traces, first and last lines. Grep output is capped per file. Diffs keep every changed line and cap context. Source code keeps signatures, imports, exports and docstrings and collapses long bodies. Prose is compressed extractively — a deterministic sentence score, no model.
5. Guarded. Errors, ids, hashes, URLs and test names may never be dropped; a result that is not smaller, or that fails the recoverability check, is rejected and the original forwarded. A per-request deadline forwards the original if compression runs long.
6. Retrievable. Each lossy block stores its original (secrets redacted) in a local store with a short TTL and adds a marker. When a request carries markers, the listener offers the model a vg_retrieve tool and answers it from the store without a client round-trip. You can do the same with vg serve retrieve <hash>.

Across turns the pipeline also replaces verbatim repeats of earlier tool output with a pointer, marks file reads that a later edit made stale, and holds fresh reads byte-exact for a few turns before they become eligible.

Profiles. coding (default) is cache-mode with reads and edits byte-exact; balanced lets older reads compress; aggressive is token-mode over everything eligible, for long autonomous runs; general is for non-coding chat. Pick one with --profile or VG_COMPRESS_PROFILE.

Turning it on

vg install <agent> --compress writes the agent's own base-URL config, then makes sure the listener it points at is running — it starts vg serve --compress --background or reuses the healthy one already on the port. For Claude Code it also adds a SessionStart hook to the same .claude/settings.json that restarts the listener when a new session finds it gone, so the routing survives a reboot. The hook needs vg on your PATH; with npx it is skipped and the install says so.

Agents --compress can route: claude, codex, cursor, aider, copilot-cli, opencode, cline, continue, goose, openhands, gemini, qwen, kimi, grok, crush, amp, factory, kiro, vibe, zcode, vscode. Agents that read a config file (Codex, Claude Code, Droid, OpenCode, Crush, Continue, Goose) get an atomic edit with a backup beside it, a marker recording exactly which fields changed, and an owner file so two concurrent sessions never undo each other. Agents that only read an environment variable have nothing to write durably — run them for one session with vg serve --compress <agent>; the install reports that rather than silently skipping.

The listener speaks the Anthropic Messages API, OpenAI Chat Completions and Responses, streaming and non-streaming. Every response carries x-vg-tokens-before, x-vg-tokens-after, x-vg-tokens-saved, x-vg-usd-saved and x-vg-transforms headers so a client can see what happened to its request. If a healthy listener is already on the port, a second vg serve --compress attaches to it instead of failing.

Seeing what it saved

vg savings                  # both sections: code-map queries and compression
vg savings --compression    # only compression: requests, tokens, dollars, by model / client / project
vg savings --benchmark      # measure the compressors offline on built-in fixtures
vg show savings --open      # the live page in your browser

The ledger holds numbers only — never message content — and nothing leaves your machine. See vg savings.

Memory and learned guardrails

Two smaller capabilities live on the same verbs, because they write the same things:

• vg serve memory — project-scoped memory shared across your agents: facts, preferences, rules, decisions, gotchas, commands and snippets, with an evidence count so a rule is promoted only after it has been seen enough times. vg serve --memory injects the top matches into each request and exposes memory_search / memory_save to the model. Scopes are project, user and global; storage is a local file per scope, searched with a lexical ranker — no model, works offline.
• vg install <agent> --learn — scans the assistant's past session logs for this project, detects loops, repeated errors and missing-context patterns, and writes a short guardrails block into its instructions file (preview by default; --apply writes). See vg install.

Settings

Everything beyond the flags above is a setting rather than a flag — around 130 VG_* variables covering upstream URLs, spend caps, rate limits, output shaping, logging and per-subsystem tuning. Read them with vg serve config and change one with vg serve config set KEY VALUE; hot knobs take effect on the next request. Precedence: flag → environment → settings.json → profile default. The most-used ones:

Privacy

Everything runs on your machine. The listener binds to loopback unless you pass a token, forwards your provider credentials untouched, strips its own headers before forwarding, never logs message bodies unless you ask, and redacts secret shapes before anything — stored originals, ledgers, logs, memory — is written to disk. Files are created 0600. There is no beacon and no update check unless you opt in, and DO_NOT_TRACK / VIBGRATE_TELEMETRY=0 win over any opt-in.

In your own code

The same pipeline is available programmatically from @vibgrate/cli, offline, with no listener running: compress() over a message array, withCompression() for the Anthropic and OpenAI SDK client shapes, and compressionMiddleware() for the Vercel AI SDK. See the programmatic API.

Related

• vg serve — the runtime the listener lives in, and the serve compress / retrieve / memory / config subcommands
• vg install — routing an agent, and --learn
• vg savings — the report, the benchmark, and the live page
• vg doctor — listener state, store size, routed agents, invalid settings
• Token-savings benchmarks — the measured code-map comparison