Cost engineering: what we learned, and the bill we could not reconcile

When I first looked at AI cost in YuktiBOS, I thought the answer was a cheaper model. It was not. This post covers what two long research reports told us, what we built, and an uncomfortable fact: our own spend figures were wrong, and some may still be.

Why we did it

One article, a Jio telecom case study, cost Rs 180 and was unusable. Here is what happened. No source was ever opened. The market-choice step was skipped. Then a reviewer was paid four times to say "Unknown, not opened". Internal notes like "I have not yet opened any Jio source" were written into the text a reader would see.

Every check ran after writing. The money went into checking something that should not have been written yet.

What the research said

I gave two long research reports to Claude: one from ChatGPT on large-context workflows, one from Gemini on token cost in multi-agent systems. They agreed on most things. The ones we acted on:

1. Cost is a system-design problem, not a model problem. One request fans out into many calls, and each call re-sends the full brand and client context. Each step also re-reads everything before it, so input grows roughly with the square of the steps.
2. Stopping to ask the user for more budget is an anti-pattern. Hard caps are circuit breakers for runaway loops. They are not a cost strategy.
3. Caching is architecture. Order the prompt from static to dynamic: global rules, client context, tools, session state, the task. Cached reads cost about a tenth of normal. The cache silently breaks if the front of the prompt changes, for example with a timestamp.
4. Give each worker a minimal package. A fact-checker needs a rubric, claims, sources and the draft, not the whole brand book. The reports put this at roughly 25,000 down to 3,500 tokens per reviewer.
5. Revise by line, not by page. Reviewers return line-level fixes, code applies them, no full rewrites.
6. Route by task. Small models for classify, extract and check. Frontier models only for hard judgement.
7. Cap loops. At most two revision rounds, then a human queue.
8. Measure every call.

What we built

• A trace of every AI call: task, step, role, model, input tokens, cached input tokens, output tokens, cost and time. A "Cost and quality" report shows planned cost, actual cost and savings from the cache per piece.
• A measurement mode that raises the safety cap for a day so test runs complete and are fully traced.
• One prompt builder with a fixed block order, deterministic serialisation, and tests that fail if the cached front changes between two calls for the same client.
• A short brand card per client, built when a voice guide is approved.
• A research gate before the writer. Facts missing means one targeted retry, then write only what is sourced. Gaps go to notes, never into the body.
• Budget-first recipes: one brief, one research pass, one writer, one combined reviewer with line fixes. The cap finishes the piece instead of stopping to ask.
• A fixed evaluation set of five briefs, so every change can be tested on the same work.

The planned cost was Rs 12 to 20 for an article and Rs 2 to 3 for a post. Those were estimates, not results.

What we measured

From the blind review file, 225 pieces have a recorded cost. Their mean cost was Rs 19.06 and their total Rs 4,289. By model (pieces with a cost): Opus Rs 17.7 (n = 120), Sonnet Rs 16.5 (n = 37), GPT-5.5 Rs 22.4 (n = 32), Gemini Rs 23.4 (n = 36). I calculated those from the file.

These include best-of-3 runs, which cost two to three times a single writer without a clear gain in the human read. In the leaderboard from 3 November the cost per usable post was Rs 11.8 for the human brief on Opus and Rs 29.5 for the editor's brief best-of-3 on Opus.

Two of those cheaper-looking things did not hold up. Moving the brief to the smallest model made the most important step the weakest. I cut cost before measuring quality, and quality dropped. And the cheapest checker, loaded with about 65,000 characters of skill text, rewrote sentences and flattened the voice.

The bill we could not reconcile

On 3 November my Anthropic console showed $50.85 for the last seven days, about Rs 4,816. YuktiBOS's own roster said Rs 5,930. We were over-reporting by about 23%.

Two causes were found or suspected:

1. GPT and Gemini calls made by workers configured for Anthropic were being added to Anthropic's total. This is fixed. History before the fix stays overstated.
2. Open items I flagged for checking: whether cache tokens are priced twice (Anthropic's input count already excludes cache reads and writes); the price table for each model, which has no source recorded in the code; and the rupee rate. The code uses 83 rupees to the dollar. My own figures imply about 94.7, which would make every rupee cost about 14% low.

So the Rs 19.06 above is "as recorded", not audited. I would trust the ratios between variants more than the rupee totals.

I also saw credits drop from about $260 to about $60 in one month, which is why I want spend checked by API key, separating YuktiBOS from Claude Code.

What I would not claim

I have not published a cache hit rate, because I do not have a verified one. Caching was added, marked "not verified" in the plan, and a cache-hit column was later added to the leaderboard. The proof is a number I can show, and I do not have it in my notes yet.

What is still open

• Confirm the rupee rate, the price table and the cache token maths. Three items need my decision.
• Check whether the daily limit should reset at midnight or at 05:30 IST, as it does now.
• Move limits to dollars (done) and re-confirm after the reconciliation.
• State isolation is only partial; the combined reviewer still gets more than it needs.

Numbers at a glance