Token Saver (beta)
Token Saver is a local proxy on each computer that makes AI tools' requests smaller. It compacts only the newest content in a request (JSON, logs, test output, search results, diffs), so the prompt cache stays valid, and sends the request straight to the provider with the tool's own credentials.
Token Saver is in every plan, in beta. It is off by default.
How it works
- It runs on the computer, listening on
127.0.0.1:8786(or the next free port). Requests go from the tool to Token Saver, then directly to the provider. Server keys are never involved, and nothing goes through the tunnel. - It uses the tool's own sign-in or key. It never replaces your provider credentials.
- It keeps the prompt cache valid. Only tool output after the last assistant turn is compacted. Content already sent is sent the same way again, so the cached prefix of the conversation doesn't change.
- It fails open. Anything unexpected, such as a parse or transform error, sends the original request unchanged.
- It routes Claude Code and Codex automatically, with a backup of their settings and an exact undo. A tool already pointed at the model proxy or another address is reported as a conflict and left alone. Quitting SpanDock puts the tools' settings back; starting it routes them again.
Modes
| Mode | What it does |
|---|---|
| Off | The default. Nothing is routed. |
| Passthrough | Measures only: requests are forwarded unchanged. |
| Safe | Compacts the newest tool output. Lossless changes only for subscription sign-ins; with an API key and the local retrieval store on, also lossy trimming, with the original kept on the computer. |
Lossless changes include minifying JSON, stripping terminal colour codes, folding repeated lines and grouping search results. With an API key and the local retrieval store on, Safe mode can also trim long arrays, logs and diffs. Each trim leaves a marker that names the original, kept on the computer for 30 minutes and never synced to the server. Subscription sign-ins, and credentials Token Saver doesn't recognise, get lossless changes only, and no header is ever added.
If three requests in ten miss the prompt cache, the computer falls back to Passthrough by itself.
Try a dry run first
A dry run applies Safe mode's changes to a copy of each request for 1 to 72 hours, while the original goes to the provider. Afterwards, a side-by-side report shows what Token Saver would have saved. The contents stay on the computer. If the server may view the dry run's report, the computer shows a notice saying so.
Roll it out
- On the server's dashboard, open AI tools → Token Saver → Policy.
- Under the defaults for every client, choose Managed and a mode. Passthrough for a week is a good start.
- Roll it out to a percentage of computers, or to computers with chosen tags, and lock the mode if you want.
- Look at the fleet and the request log on the same page.
The kill switch on the Policy tab turns Token Saver off on every computer at once.
What it reports
Token Saver records one entry per request, with counts and names only: the tool, provider, model, mode, the changes applied, tokens before and after, the cost saved and how it was worked out, the latency added and the reason for any fallback. Never prompt or tool content.
- Savings are priced with the price catalogue. The prompt cache's own saving is shown on a separate line.
- Spend isn't counted twice. Token Saver's entries carry no model spend, so dashboards, budgets and the leaderboard count only the tool's own telemetry.
- The Token Saver page shows a computer its own totals and log, and the server the whole fleet's.
Limits
- Token Saver covers the Anthropic Messages, OpenAI Chat Completions and OpenAI Responses formats.
- Claude Code and Codex are routed automatically. A tool already routed through the model proxy isn't routed through Token Saver as well.