Token-saving tools for Claude Code
Plain Claude Code against six setups that promise to cut its token bill.
This benchmark was created using LangWatch. Sign up to create your own
Cost per session, at API list prices
- Plain Claude Code: $5.87 on average, sessions from $5.51 to $6.09, the baseline
- Code Mode: $4.40 on average, sessions from $4.08 to $4.99, −25% against plain
- rtk: $4.64 on average, sessions from $4.41 to $4.78, −21% against plain
- Ponytail: $5.02 on average, sessions from $4.06 to $6.03, −14% against plain
- Graphify: $5.13 on average, sessions from $4.72 to $5.65, −13% against plain
- Caveman: $5.83 on average, sessions from $4.79 to $7.31, −1% against plain
- EGM: $21.39 on average, sessions from $16.40 to $24.95, +265% against plain
- Plain Claude Code
- Cheaper than plain in every session
What each setup does, and what it changed
Each ran three or four times on the same task.
Plain Claude Code
Claude Code with nothing added: every setup below starts from it.
- Cost
- $5.87
- a session on average
- Judge score
- 7.0/10
- Serious bugs
- none
Code Mode experimental
Built into Claude Code: Claude writes a short script that runs several reads, edits and commands in one step.
- Cost
- −25%
- on average, lower in every session
- Judge score
- 7.0/10
- same average
- Serious bugs
- none
It works only in an interactive session, so that is how it ran. Plain, rerun that way, came out close to plain.
Claude Code docs
rtk 0.49.0
A command-line tool that sits between Claude and the shell and shortens command output, like test runs, git and file listings, before Claude reads it.
- Cost
- −21%
- on average, lower in every session
- Judge score
- 7.3/10
- 0.3 points higher
- Serious bugs
- none
GitHub · Apache-2.0
Ponytail 5.3.0
A Claude Code plugin with instructions to write as little new code as the task needs and to reply briefly.
- Cost
- −14%
- on average, lower in 3 of 4 sessions
- Judge score
- 6.8/10
- 0.2 points lower
- Serious bugs
- none
GitHub · MIT
Graphify 0.9.93
Builds a map of the codebase (which function calls which) that Claude can query instead of opening files one by one.
- Cost
- −13%
- on average, lower in 2 of 3 sessions
- Judge score
- 6.7/10
- 0.3 points lower
- Serious bugs
- 1 of 3 sessions
Claude called it at most once a session.
GitHub · Apache-2.0
Caveman 4.0.0
A Claude Code plugin that makes Claude reply with short, clipped sentences. Code and error messages stay word for word.
- Cost
- −1%
- on average, no clear change
- Judge score
- 7.5/10
- 0.5 points higher
- Serious bugs
- 3 of 4 sessions
GitHub · Apache-2.0
EGM 0.1.1
Google’s Elephant-Goldfish Model plugin: Claude writes a design document and reviewer agents check it before any code is written.
- Cost
- +265%
- on average, higher in every session
- Judge score
- 7.0/10
- same average
- Serious bugs
- 2 of 3 sessions
GitHub · Apache-2.0
Cost against the quality score
Each dot is a setup’s average. The grey band is plain Claude Code’s lowest to highest judge score over its four sessions; every tool lands inside it, so the differences may be noise.
- Plain’s lowest to highest judge score
- Plain Claude Code, the open diamond
Claude rereads the whole conversation on every turn
That reread, of code, command output, instructions and its own earlier turns, is 97.6% of all tokens. Claude’s own new words are 0.5% of the tokens.
Every token of every session, as 1,000 squares
- Claude’s own words
- 0.5% Claude’s own words
- new input, read for the first time
- 1.9% new input, read for the first time
- the conversation, read again every turn
- 97.6% the conversation, read again every turn
How we ran it
The task, the sandbox, how the code was graded and what the numbers can and cannot say.
Show the method
A session is one run of Claude Code from the task description to finished code. Every session rebuilt the same merged pull request, langwatch/langwatch#8663, so any difference between setups comes from the tool and not the task. The model was Claude Opus 5.5, and the $40 budget never cut a session short.
Each session ran in its own fresh checkout inside a sandbox, with GitHub blocked, so it could not see the merged code, the graders or any other session.
We graded the code the way a reviewer would, so a tool that saves money by writing worse code shows up: the pull request’s own hidden tests, behaviour checks against the merged code, the serious findings of an AI code reviewer (OpenAI Codex), and an AI judge (Codex too) that scored every change without knowing which setup wrote it.
The dollar figures are what the same model calls would cost at API list prices. The sessions ran on a Claude subscription, so no one paid them.
Plain Claude Code, Ponytail and Caveman ran three at a time on one laptop; Graphify, rtk and EGM ran one at a time on a newer Claude Code; Code Mode and plain’s interactive rerun ran two at a time. So wall time is the least comparable measure, and tokens, cost and quality are the ones to read.
Code Mode works only in an interactive session, so its three sessions ran interactively, on Claude Code 2.1.325. To check that this does not skew the comparison, plain ran three more times the same way: its cost, tokens and quality landed inside plain’s range, and Code Mode used fewer tokens than this rerun in every session.
Each setup ran three or four times on one task. Plain alone cost from $5.51 to $6.09 across its four sessions, so a result inside plain’s range is one we cannot tell apart from chance.
This page shows the latest run, 16 Nov 2026: 27 sessions with Claude Opus 5.5. We rerun it as tools ship new versions and add tools as they appear.
The rule: we call a tool cheaper only if even its priciest session cost less than plain’s cheapest one. Every other measure works the same way, and anything else counts as inside plain’s range.
The setups
Questions
Does Code Mode save tokens?
Yes. Against plain rerun interactively, as Code Mode must run, it used 44% fewer tokens, fewer in every session. It made fewer model calls, and each call rereads the whole conversation. Its cost was 21% lower, though not in every session; its card shows −25% cost against the original plain runs. Its code scored the same.
Why does rtk save money?
rtk shortens shell output, like test runs and git, before Claude reads it. Claude rereads the whole conversation on every turn, and those rereads are 97.6% of the tokens, so shorter output saves on every later turn. Its sessions cost 21% less, lower in every session.
Does making Claude talk less save money?
Not clearly. Ponytail cost 14% less on average, lower in 3 of 4 sessions, but it also asks Claude to write less code. Caveman, which only shortens the replies, showed no clear change.
Did any tool hurt code quality?
Not clearly. Every judge average is inside plain’s own range, with Graphify lowest at 6.7/10 against plain’s 7.0/10. Hidden tests and behaviour checks stayed inside plain’s range. The AI reviewer flagged a serious bug in at least one session of Graphify, Caveman and EGM, and in no plain session.
Is EGM worth what it costs?
Not on this task. It cost 3.6 times as much as plain on average, and more in every session. Its judge average, 7.0/10, is the same as plain’s, so the extra $15.52 a session bought no clear gain.
Did Claude use Graphify’s map?
Barely: at most once a session. Graphify’s sessions cost 13% less on average, lower in 2 of 3 sessions, so the saving may not come from the map at all.
Are three or four sessions enough?
Enough to see big, steady differences. Plain alone cost from $5.51 to $6.09 across four identical sessions, so we call a tool cheaper only if even its priciest session cost less than plain’s cheapest one.
How was code quality graded?
With the pull request’s own hidden tests, behaviour checks against the merged code, an AI code reviewer looking for serious bugs, and an AI judge that rated each change from 1 to 10 without knowing which tool wrote it.
What was the task?
Rebuilding a real merged pull request, langwatch/langwatch#8663 (“Serve the previous release's single-file datasets”, 1,466 changed lines in 23 files), with the Claude Opus 5.5 model. It is one task, so the results show how these tools behave on one mid-sized change.
What do the dollar amounts mean?
They are API list prices for every model call in the session. The sessions ran on a subscription, so the amounts compare setups and are not a bill.
Will you test more tools?
Yes. We rerun the benchmark as tools ship new versions and add tools as they appear.
See what your own Claude Code sessions cost.
Track your Claude Code usage with every turn, tool call and token, on a subscription too.
