This started based off of a hunch. We usually use OpenCode, but were 'forced' to use Claude Code for a while due to issues with Meridian. In that time, we saw the usage meter rise much, much more quickly than when using OpenCode.
This was the initial anecdotal evidence, but we undertook this small study to collect empirical data:
We added logging between the agentic coding tool (Claude Code and OpenCode) and Anthropic's endpoint, and captured all requests (and the returned usage blocks).
With one caveat (toward the end of the post) we found unambiguously that Claude Code was far more inefficient in terms of its cache strategy and its harness token usage than OpenCode.
Discussion (259 Comments)Read Original on HackerNews
If I let the main agent do the same task sequentially, it was no problem at all. I don't know if it's really just communication and orchestration that makes sub agents so inefficient, or if Anthropic figured that most people using sub agents pay per token on a big corporate account, so this is an easy way to make more money from tokenmaxxers.
That said, if your project is "do this well-planned thing on a bunch of things in parallel" then you should absolutely be instructing to have subagents "step down" to less curious models. Their output may well be more cohesive as a result!
I write a good prompt, paste the code then copy the output code and place it into my project.
So in the end I hand assemble and I only give it what it needs to know so no extra context wasted.
The human in the loop is of course the secret sauce but this way I am highly efficient, no vibecode and I work really fast too. Everything is audited.
A major complaint with AI code was that AIs struggle with complex codebases, don't respect existing conventions, reinvent functionality multiple times over, etc. So, newer high end AIs are tuned with the "explore/exploit" dial turned towards "explore".
You could probably get it to do things "quick and dirty" with prompting, but that, of course, requires prompting for it.
Or are you saying my sub agents burned so many tokens because they were all using Fable, whereas my main agent could do the same job with a lesser model?
Unless they are orthogonal they most likely require similar context anyway so multiple sub agent is just wasteful.
im on local only AI and subagents are only valuable when they avoid polluting the context with extraneous file reads and parallel exploration when fixes are linear.
as OP is on about, subagents burn tokens because they arnt a deterministic intelligent gatherer but like pooluring water into a maze hoping the exit will illuminate.
If you do it sequentially you only read those files approximately once, and everything hits the same prefix cache
If you plan on continuing on in the parent, and aren't going to necessarily be touching the systems the other agents are exploring, it can be worth it.
It's useful in certain situations where the parent context may need the "10,000 foot" view of something without going back in there. But subsystem-specific AGENTS.md/CLAUDE.md files are still superior and accomplish the same thing. The problem with those is they can become stale.
1. Agent("Test")
2. look at your token usage
3. Repeat a few times
I didn't check again as I type this message but am somewhat sure subagent doesn't cache system prompt as of maybe last week
The best you can do in such an environment is seek to introduce new features at the top tier, and then pull old features down the stack as the cost of those features has been amortized out, or to hurt your competitors by raising the ladder.
Yesterday, I wanted to review a complex piece after a large refactoring, and requested a review plan beforehand. The first step was 8 agents + one more to verify the findings (all Fable). Looks good, approved.
The verification step turned into an attempt to throw a party with 41 Fable verifiers.
It will find a way.
> CRITICAL: Do NOT spawn sub-agents for any reason. Perform all work in the main session. If a task is too large, ask me to break it down manually.
> This is a big task, and can easily get too large. However, sub-agents make the situation worse, and eat through our token budget way too fast. Do not use them.
> Take on manageable tasks. Don't try to do everything at once. When you start on a big task, break it down into smaller tasks, and make sure you finish each task before starting on the next one.
Or actually Claude put it there for me. Maybe it's a bit much, but it seems to work.
I feel like maybe it could have asked for clarification or something rather than go and try to calculate all the digits of pi all of a sudden.
Probably because the general purpose subagents inherit the parent model.
I tell Claude explicitly to use Explore subagents, which use Haiku only, now.
only if you don't specify which model should be used
Maybe when they realize there is need to change this they come up with a more configurable interface for us mere mortals who can't afford to gamble their house on a pay as you go subscription.
I run it basically 24/7 on a ~500k line repo, and only rarely run out of quota before the end of the week.
My experience with Claude Code was very good until about 2.5 months ago, and then it suddenly turned unbelievably terrible for me.
I have not and will hopefully never look back.
I still have PTSD from how ungodly terrible it was that last week of using it.
Please, for the sake of everyone suffering from actual PTSD: Don't. It's hard enough already for victims to communicate what difficulties they are facing without people watering down terminology like that.
Sorry just teasing.
Just tried Claude Code yesterday, and nope, it's the same old bad.
This is why the subscription plans are forced through the harness (the "OpenClaw Wars"): it creates a false equivalence in the minds of many customers between API tokens (latency sensitive, easy to measure) and Claude Code tokens (remnant backfill to stay to the right of the roofline, marginal cost often zero).
Selling sausage as sirloin is a great business if people go for it. And there's nothing inherently wrong with spot pricing, as long as you're honest about it...
What's happening this year, with secrecy and all, is saddening, but expected.
On the other hand, the newer variants also tend to benchmark higher so it's not quite a clean argument of "hey the new version eats more tokens"
Its rather frustrating, slower tokens and more tokens.
At this point I think Dario is just in his wellness retreat adjusting a revenue/profit dial.
Increase the price by 70% and then cut it by 50%, resulting in a 15% cut that sounds like a major deal.
It’s like that saying “What Andy giveth, Bill taketh away”, but in this case it is one company.
There is definitely a conflict of interest.
no amount of alignment will stop aomeone drom just shutting up.
Edit: consumer Claude subs are the 5%. I’d bet most all of CC subs lump in under enterprise.
I don't fully agree with the premise that they intentionally increase system prompts, but the enterprise plan usage is going to make that a huge income for Anthropic.
Also I think it’s well known that OpenAI is the much less expensive option (in tokens and $$). For the same $20 you get a lot more mileage.
Curious if folks have strong opinions about the overall UX of OpenCode vs CC…
And people paying $100 or $200 are much more likely to max it out for purely psychological reasons - it crosses that threshold where I want to see my money's worth in full. Whereas people on $20 subs are more likely to be there just to get access to better models and features, and are not necessarily even doing any substantial work.
Especially since compute is such a scarce resource.
OTOH, this makes typical subscriptions usages consume more tokens, which are included in their flat fee.
This sounds more like incompetence than malice.
To remind Goodhart's law: "When a measure becomes a target, it ceases to be a good measure".
..also to parent's point, yes the upsell is only appealing once user run's out of tokens.
Not sure if intentionally meant as a reference, but it gives "I use Arch btw" vibes.
and the interesting thing about system prompt wastage is its a cost that scales non linearly with subagent use.
I mean, that's a very weak argument? Isn't a much more plausible explanation that with your tooling you'll have more of a lock-in than with just your model?
They get lock-in, and through that lock-in are more effectively able to inflate token usage.
After reading PUSH_AX's valid comment: ``` This is like saying contractor (A) asked for $33,000 to undertake the work and contractor (B) asked for $7,000 Are we measuring and caring about the right thing? ``` We will update the post to include:
1) A more in-depth task. 2) Qualitative results comparison. 3) As soon as possible, a reproduction of the inputs and outputs.
I wonder if a lot of the 33k is context, like from recent conversations.
https://quesma.com/blog/the-true-cost-of-saying-hi-to-an-ai-...
Tokenflation seems very real: the number of tokens consumed by simple tasks keeps increasing.
I am a bit surprised that you're disappointed that it does exactly what you told it to - people usually have the opposite complaint.
If you're using it interactively and watching what it changes, I'd trigger the tests when you think it's needed. And if you want to go more hands-off, why not add try to encode the same nuance you'd use into the rule?
I read that this is because it wastes time looking through past conversations and other context to figure one what you might want it to do - a less ambiguous prompt would be better.
The entire agent system prompt can be seen here:
https://github.com/earendil-works/pi/blob/main/packages%2Fco...
Every time I read this comments I have the feeling you are talking about mcp or sub agents, otherwise this makes no sense at all.
What do you miss? I ask because I do some heavy work with pi + GLM 5.2 (using opencode Go subscription) and my workflow is plan -> implement.
Sure, but you have to add almost everything, no? It deliberately only comes with read, write, edit, and bash. My point wasn't that you can't add stuff, but that I'd just rather use an harness that's a bit more full featured from the start.
(Pi is a bit like old 3D printing where fettling the printer to work is a central part of the hobby. I'd rather just buy a Prusa.)
I guess the cache would only be invalid if the day changed or the root directory, which would technically happen infrequently enough.
But I'll investigate how that works in a session. You got me curious.
Incidentally, I also have zero supply chain attack surface as I have zero dependencies in my agent, just go stdlib. Pi, again, has 130+ transitive dependencies asking me to trust the security of my system to 150+ additional people I've never met in exchange for a bunch of bloat I do not want.
To learn yourself:
<$20 on a cloud AI api for a chunk of tokens and have the AI teach you. "help me write an AI Agent using (language) and walk me through the steps"
Realize that these agent are REPL/while loops that maintain a conversation state and then based upon the tagging syntax like <TOOL:bash:uptime>uptime for system run time</TOOL> and the agent extracts the tool and then does sub commands.
If going local, llama.cpp is going to be the more beginner friendly local inference engine that supports more processor types (AMD GPUs, Intel GPUs, CPUs, anything that supports Vulkan, not just Nvidia). LM Studio is a nice wrapper for this if you'd rather avoid cloning repo and compiling yourself, provided you don't mind closed source software; it's much less enshittified than Ollama.
If going local, you will also need model weights in the right format for your inference engine, and with a model that can fit on your hardware. This is going to be .GGUF files if you're using llama.cpp or a wrapper for it like LM Studio.
From there, pick a language, go look up the OpenAI /chat/completions API format (or Anthropic's "Responses" API format), create a DS or array or slice to store messages, and build a loop that accepts user input, formats it according to the API format, sends it to the inference server, retrieves and parses the response, adds the response to the DS/array/slice, and repeat.
There's a lot more beyond this - tool calling, other API formats (optionally), MCP servers, transport layers besides terminal stdin/stdout, permission models, starting with a system message, clearing your message stack correctly (hint: don't reset it mid tool-call), message compaction, web searching and page fetching, semantic search RAG over embeddings, memory layers - way too much to cover exhaustively in a single message.
I don’t like that Claude became more opaque around February, including the system prompts. 33k feels way too much.
I appreciate that Codex is open source and OpenAI has explicitly said using the subscription with other agents is ok. OpenAI has been much more consumer-friendly recently.
What matters even more is tooling quality. Bad/buggy tooling causes a lot more roundtrips that wipes out all gains from initial greedy approach.
A few months ago, I did a full benchmark run of 7 agents over 8 tasks (https://github.com/dirac-run/dirac has the data and traces). I cannot claim neutrality because of the obvious connection to one of those, but the data should be reproducible and useful. Importantly, Dirac wins there significantly on those tasks because they are mostly refactoring related (which is where approaches like hash-anchoring and AST parsing tend to shine)
In a pay per token situation, there is a huge conflict of interest with the harness provider and the token seller being the same party ... efficiency is less profitable.
I have accused claude code of trying to run up the meter on me and it confirmed I was absolutely right.
Except there’s a competitive incentive to either use less tokens or make the tokens go further
There is a world where, to hit the next quarters revenue projection, you add 2,000 tokens to the system prompt and "beat" expectations at the next earnings release.
Doesn't the model need at least a basic system prompt to understand what tools are available?
I used mitmproxy (setup assisted by Claude, natch) to capture Claude Code's entire initial system prompt and the whole thing was (I just double-checked) 162k of JSON.
This led me to start experimenting with Pi, OpenCode, and Hermes...
I was simply supporting the article's data - their reported 33k tokens is probably roughly 150-165k.
> When context gets too long, maki compacts history automatically: strips images, thinking blocks, and summarizes older turns.
Don’t the summaries of older turns effectively invalidate the context cache, such that you consume less tokens but more expensive tokens?
My $20 sub using gpt 5.6 sol thinking-off lasts for hours using pi.
Also, I have seriously used most harnesses - One feels like it's being built in a place that truly understands AI and where agentic engineering is headed. You might not like it, but peak performance exists in CC when it comes to orchestration of bulk parallel work / subagents. The open source agents are catching up or accell in different way (Im preferable to pi.dev), but I'm not sure they're architecting orchestration the right why.
Are we measuring and caring about the right thing?
If you don't use a subscription, and pay per token instead, you can easily move to another harness.
This is posed as some sort of discovery, but both Claude Code and OpenCode display token usage clearly after starting a chat or agent, and 30k and 7k is exactly what you see.
I find it hilarious people think they can build on slop or on captured output from quality engineers. Intent is the most important aspect, and no code base could ever capture that
I asked both a trivial question (summarize last commit). Opus cost 50 cents, Fable about $1.
That checks out because Fable's twice as much in the API (though I think its emphasis on correctness makes the difference larger for bigger tasks).
But, at $1 per question, I think I will stick to the subscription for now! I was certainly glad GPT-5.6-Sol is included in OpenAI's subscription, and I'm curious if they'll be able to do the same for GPT-6.
All the VC money appears to have run out a few weeks ago.
I haven't tested it on anything bigger but it doesn't seem to do the kind of proactive testing, that they do in bigger harnesses.
Codex at least has a system prompt that tells it not to consider a feature a complete until it has verified it. I'm not sure about Claude Code.
I suppose I could add that one line to the prompt, and it would get me much closer to agi :) I think Fable does this proactively even without a prompt, but I haven't tested that yet.
If Fable in my own harness is significantly cheaper than Claude Code, that would be very appealing. (I could actually afford to use it for most things!) But I think most of the cost comes from the testing it does. So we'll have to see.
[0] https://minimal-agent.com/
Competition is good.
That would depend entirely on what your device is. This sounds likely not to be an issue with the harness, but the capabilities of the models you've tried.
I experience almost no tool call failure using my nothing-special harness and DSv4 Flash.
Qwen 3.6 35B A3B and Qwen 3.6 27B can both do reliable tool calls on Pi at Q4_K_M using llama.cpp
Ultimately this combo worked:
1. https://pi.dev/packages/pi-tool-guard —- corrects key name synonyms and common structure errors, so tool calls succeed automatically (e.g if the model hallucinates old_str instead of oldText). It also wraps top level oldText/newText in an edits array if the tool didn’t do it.
2. https://pi.dev/packages/@aboutlo/pi-smart-edit - white-space-tolerant edits, as Qwen would sometimes add a fifth space to a four space indent
Hashline edit tools didn’t work well for me at all, they confused the model and it still failed to edit correctly. Also line removals would invalidate the rest of the file requiring re-reads. I tried pi-hashline-edit-pro, though I see it now keeps a database of hashes to help keep them stable across edits. Regardless Qwen kept thinking that the hashline prefixes were part of the source.
If you’re using API, on the other hand, there is absolutely no reason to use Claude Code, or Codex.
We already know company is not making any profit. To break even they need ppl to use a lot more tokens AND pay for them premium price.
We also know LLMs dont give such a huge productivity boost do warrant spending of THAT size.
At this point you only wait for more and more shady plays.
In our case the alternative was nothing so I'm happy to have it, but currently claude is not as competitive as I'd have maybe expected given the hype
It works great for long-horizon tasks, and feels like it saves a boatload of tokens.
I'm generally against this context pruning without prompting or details. Sleev is very opaque about how it works and definitely will bust your cache.
Supposed to be hacker news and half the posts are like "this harness steals this" like it cant be avoided.
These API costs are mad.
It pays to be marginally ahead of people stuck on open models.
If I self host a local model is there some way to make Android studio not time out after 10 minutes?
So not only is this article AI-written, but the testing was entirely done by AI, too? I can't see any other reason to use such an old model.
> Our traffic passes through a local LLM gateway that wraps requests in its own envelope, a constant we measured at roughly 6,200 tokens with bare calibration requests
Why do you need to do calibration requests to figure out how your own gateway is affecting requests?
> Its subagent lane did not complete cleanly through our gateway
> We attempted to toggle extended thinking in both harnesses and are declining to publish numbers. Our gateway applies its own thinking policy, neither harness's toggle demonstrably survived the path, and anything we quoted would be noise.
Why is your own gateway screwing with your testing?
Cost, mainly. The runs went through a Claude Max subscription rather than metered API billing, and pinning an older stable snapshot kept run-to-run comparisons clean and cheap. The fixed harness payload (system prompt plus tool schemas), so the headline numbers shouldn't change too much.
That said, happy to re-run the matrix on Fable and publish the diff; payload figures should barely move, tool-calling behaviour might.
Gateway:
Meridian (github.com/rynfar/meridian); proxy that bridges the Claude Code SDK to a standard Anthropic endpoint so a Claude Max subscription can drive OpenCode-et-al.
It's the auth route for all agent traffic on the machine, not something built for the benchmark.
Claude is much better in OpenCode then in Claude Code, OpenCode is just better than Claude Code. Claude Code feels like a complete mess to use comparatively.