Your AI Markdown files are costing you more than tokens

Julio Falbo15 min read
Listen to it, read by Julio Falbo (19 min)
19:00

CLAUDE.md, AGENTS.md and dozens of skills later: why uncurated instructions are expensive and risky, and why the fix is to script everything that does not need to be random

When I wrote Markdown is the New API, one number stuck with me: 50,000. In that article, it was roughly how many tokens of context the official GitHub MCP server initially consumed. It needed them only to teach an agent to open issues and merge code. It later dropped to about 23,000. The gh CLI already did all of it.

That same tax is back. This time it came through a door we opened ourselves.

I will not pretend I have a dashboard screenshot for this one. What follows is a composite: one fictional repo built from a pattern that is easy to reproduce, with illustrative numbers. Run the audit at the end of “Why a pile of Markdown is dangerous” on your own repo and replace my numbers with yours.

Picture a monorepo one year into AI-assisted development. Three tools are in use, and each one reads its own files. Claude Code reads CLAUDE.md. Codex and several other agents read AGENTS.md. Cursor reads .cursor/rules.

Over that year, the root CLAUDE.md has grown to about 2,000 tokens. Four teams each wrote about 1,000 tokens of their own rules. Twelve Cursor rules are marked alwaysApply: true, at about 500 tokens each. Forty skills contribute a name and a description of about 75 tokens each.

Then someone wants every tool to see every rule. Claude Code’s support for AGENTS.md is confusing at best, so the usual bridge is a symlink or @ imports inside CLAUDE.md. A few imports pull in the four team files. More pull in the twelve always-on Cursor rules. A single Claude Code session now loads about 15,000 tokens before the user types a word.

This is the ceiling, not the typical case. No tool loads all of these files by default. Nested CLAUDE.md files load only when the agent works in that subdirectory. Cursor rules without alwaysApply: true load only when they match. A default setup sees a fraction of the pile. I use the ceiling for two reasons. Bridging is what teams reach for when their tools disagree. And round numbers make the arithmetic easy to check.

Now assume you build your own agent on the API and resend that context on every call, with no caching. An agent session is not one call. Every tool call sends the context again. Assume 40 model calls per session. That is 600,000 input tokens per session spent on instructions alone. At an illustrative price of $3 per million input tokens, that is $1.80 per session. Twenty engineers running ten sessions a day comes to about $360 a day. Over 22 working days, that is roughly $8,000 a month. None of it touched the actual task.

If you use Claude Code, Cursor or Codex, you are not paying that bill. They already apply prompt caching automatically, and I will come back to how much that saves. Caching shrinks the money. It does nothing for the other problem in the pile: three files disagree about which package manager to use, and nobody knows.

We fixed integration bloat with Markdown. Now we are drowning in it.

How the model actually reads your files

To see why this hurts, you need a rough picture of the machine.

Before 2017, most language models were recurrent neural networks (RNNs, and their cousin the LSTM). They read text one word at a time and carried a small memory forward. Picture reading a novel through a keyhole while keeping notes on a single sticky note. By page 300, page 1 barely fits. They were also slow to train. Word 50 had to wait until word 49 was processed.

Then Google researchers published Attention Is All You Need and introduced the Transformer. Its core idea is self-attention. In the original design, every word in the input looks at every other word at the same time and decides which ones matter. Back to the novel: the keyhole is gone. You see the whole page, and a highlighter connects “it” to the noun it refers to. Every position is processed in parallel during training, so these models could learn from enormous amounts of text.

Models like Claude and GPT use a decoder-only variant with one restriction. Each token looks only at itself and the tokens before it. This is called causal masking. It fits generation, because a token cannot attend to words that have not been written yet. It also tells you where your Markdown sits. Instruction files load before the user’s request, so every token the model writes can look back at all of them.

That brings us to the unit of work: the token. Every vendor has its own tokenization rules. A token is often a fragment of a word, and in English it averages about three quarters of a word, as I noted before. That works out to about 1.3 tokens per word. The composite numbers in this article round that to 1 token per word to keep the arithmetic easy. When you count your own files, use the avg of 1.3 factor the audit applies.

Generation is your phone’s autocomplete, run in a loop. The model never writes a sentence. It writes one token, appends it and predicts again. The probability of a whole sentence is factored into a chain of next-token probabilities:

Text
P("the build failed") = P("the")
                      × P("build"  | "the")
                      × P("failed" | "the build")

Each factor is conditioned on everything before it. That includes every Markdown file you loaded. At each step the model scores every token in its vocabulary and samples one.
Two consequences follow: In the classic attention design, compute grows roughly with the square of the context length and every loaded word shapes every word the model writes.

Why a pile of Markdown is dangerous

In Generalist vs T-Shaped I described rules files (CLAUDE.md, .cursor/rules/*.mdc) as the AI equivalent of an .eslintrc. When they are always on (in Cursor, that means alwaysApply: true), they are active without being invoked. That is their strength but is also why sprawl costs so much. Ambient text gets paid for on every turn of every session.

Four failures repeat.

When an agent ignores a rule, the common instinct is to escalate. Write a longer explanation. Add one more NEVER in capitals. Switch to a bigger model. Each of these makes the pile heavier. A longer rule costs tokens on every turn of every session. A NEVER is one more condition on every prediction, and often one more conflict with a file someone else owns. A bigger model reconciles contradictions more gracefully, at a higher price per token, and it still has to guess which file you meant. None of them removes the contradiction.

Rules that must always hold should leave Markdown entirely. Claude Code hooks run shell commands at lifecycle events such as PreToolUse and PostToolUse. "Always run the formatter" in a Markdown file is a request. A PostToolUsehook that runs the formatter is a guarantee. One is a sign saying "please buckle up" but the other is a car that will not start until you do.

Try yourself and start with a a quick audit.

Count what each tool actually loads, not the whole pile:

Bash
# Claude Code: the root file. Nested CLAUDE.md files load when the agent enters that directory.
# If CLAUDE.md uses @ imports, add the imported files to this count.
wc -w CLAUDE.md

# Cursor: only rules with alwaysApply: true are always on (with or without a space after the colon)
grep -lE 'alwaysApply:[[:space:]]*true' .cursor/rules/*.mdc 2>/dev/null | xargs cat | wc -w

# Skills: only name and description load up front.
# Project skills and personal skills both count. Add the skill directories of any installed plugins.
grep -hE '^(name|description):' .claude/skills/*/SKILL.md ~/.claude/skills/*/SKILL.md 2>/dev/null | wc -w

# Codex: concatenates every AGENTS.md from the repo root down to the current directory.
# List them all, then sum the ones on the path where you actually work.
find . -name AGENTS.md -not -path '*/node_modules/*' -exec wc -w {} +

The skills line matters more than it looks. In the composite repo, the forty descriptions are 3,000 of the 15,000 tokens, and they never show up in a count of rules files. Personal and plugin skills hide even better, because they do not live in the repo at all. The AGENTS.md line matters for the opposite reason. A plain wc -w AGENTS.md on the root file undercounts any repo with nested files.

wc -w counts words, not tokens. Multiply by about 1.3 to get English tokens. Put a budget per tool in CI and fail the build when someone exceeds it.

Context templates: the cheapest win on your bill

Providers offer prompt caching. They store the attention computation (the key-value pairs) for a prompt prefix. When the next request starts with exactly the same prefix, they reuse it. On Anthropic, cache reads cost 10% of the base input price. OpenAI caches automatically for prompts of 1,024 tokens or more.

Claude Code, Cursor and Codex already lean on this, which is why their normal users probably will never see the $8,000 figure. If you build your own agent on the API, the cache only pays off when you order the context for it.

Think of a kitchen that preps sauces before service. You prep once and plate many times. Change one early ingredient and the prep is thrown out.

So I build context from a template: a fixed layout that decides what goes where.

  1. Identity and hard constraints. Rarely change.
  2. Tool definitions and the skill index. Change per release.
  3. Project rules. Change per sprint.
  4. Dynamic suffix: date, retrieved documents, the user’s request.

Stable content goes on top. Volatile content goes at the bottom. One timestamp on line 1 turns every request into a cache miss.

Take the own-agent version of the composite repo and run it through that template. The 15,000 always-loaded tokens are identical on every call, so they become the cached prefix. The first call in a session writes the cache, and writing costs a little more than normal input. The other 39 calls read it at 10% of the base price. At the same illustrative $3 per million, 39 reads of 15,000 tokens cost about $0.18. Add the first write and the instruction cost lands around $0.25 per session instead of $1.80. The monthly figure falls from about $8,000 to roughly $1,000. Same files. Same model. Different order.

Two caveats. The cache expires after a few minutes without use, so a session with long pauses pays for the write again. And caching makes the pile cheaper, not smaller. The model still attends to all 15,000 tokens, and Lost in the Middle still applies. Do not treat caching as permission to skip the curation. That goes double for users of the off-the-shelf tools: the cache already took care of their money, so the conflicts are the cost they are still paying.

Verify it instead of trusting the template. Anthropic’s API reports cache_creation_input_tokens and cache_read_input_tokens in the usage block of every response (docs). Log both. If reads stay near zero, something volatile has crept into your prefix.

The AI Engineer Core Track on Udemy (enrollment required) recommends a fixed context in every prompt, including the current date. Its argument is that a few hundred tokens are not a material cost. I agree, with two changes. In an agent loop with dozens of always-loaded files, those hundreds become thousands per turn, multiplied by every tool call. And the date belongs in the suffix, not the prefix.

The best feature of AI is also its biggest weakness

At every step the model samples from a probability distribution. Temperature sets how adventurous that sampling is. This is where creativity comes from: a fresh angle, a better name, a phrasing you would not have found. It is also why the same request produces a different tool call on Tuesday.

Setting temperature to 0 does not save you. Horace He at Thinking Machines Lab showed that inference servers can return different outputs for the same prompt at temperature 0. Server load changes how requests are batched, and batching changes the floating-point arithmetic.

Now run the reliability math with illustrative numbers. A 10-step agent workflow where each step is right 95% of the time succeeds about 60% of the time (0.95^10). Make seven steps deterministic and the chain rises to about 86% (0.95^3). The model did not change.

That second number assumes the seven scripted steps are right every time. A script is only that reliable after it has been tested against its edge cases. Deterministic is not the same as correct, and the release notes script in the next section had four bugs to prove it. What determinism buys you is that a bug fails the same way on every run, so you can find it once and fix it for good.

Every prompt is a workflow

Stop reading a prompt as a request. Read it as a workflow, then ask of each step: does this need judgment, or just execution? The course’s five-step strategy starts with understanding the problem before picking a model. Writing the steps down is that understanding.

In functional programming, a pure function returns the same output for the same input and has no side effects. A vending machine is pure. Press B4 and you get the same snack every time. A barista is not. Use the barista for taste and the vending machine for everything else.

Consider “write the release notes for v2.3”. It hides several steps: find the previous tag, list merged PRs, group them by label, extract ticket IDs, format the changelog, write a friendly summary. Only the last step needs a language model. The rest is git and a few lines of shell. Left to the agent, each step is a chance to read files it does not need, retry or invent a ticket number.

Here is a sketch of the deterministic part:

Bash
#!/usr/bin/env bash
# scripts/release_notes.sh: everything in "write the release notes" that needs no judgment
# Usage: release_notes.sh <version> [previous_tag]
set -euo pipefail

version="$1"
# --exclude skips the new tag if it already points at HEAD.
# Without it, v2.3 becomes its own previous tag and the range is empty.
prev_tag="${2:-$(git describe --tags --abbrev=0 --exclude "$version")}"
# Match only your tracker's project keys. Replace PAY|OPS|WEB with yours.
# A generic [A-Z]{2,}-[0-9]+ also matches UTF-8, ISO-8601, SHA-256 and HTTP-2.
ticket_re='\b(PAY|OPS|WEB)-[0-9]+\b'
echo "## ${version} (changes since ${prev_tag})"
echo "### Changes"
# --first-parent prints one commit per PR on the main branch,
# whether the repo uses merge commits or squash merges.
git log "${prev_tag}..HEAD" --first-parent --pretty=format:'- %s'
echo
echo "### Tickets"
# grep exits 1 when nothing matches. Under pipefail that aborts the script,
# so a release with no ticket IDs needs the || true.
git log "${prev_tag}..HEAD" --pretty=format:'%s %b' \
  | { grep -oE "$ticket_re" || true; } | sort -u | sed 's/^/- /'

The first version of this sketch, in an earlier draft of this article, had four bugs. Under set -euo pipefail, grep returned code 1 on a release without ticket IDs, and the whole script aborted. It used --merges, which prints nothing in a repo that squash-merges, and squash merging is the default for many teams on GitHub. git describe picked up the new tag itself whenever the tag already pointed at HEAD, which left an empty range. The fourth was quieter. The ticket pattern was a generic [A-Z]{2,}-[0-9]+, so a commit that mentioned UTF-8 or SHA-256 put a fake ticket in the list. Nothing crashed. The output was simply wrong. The script still needs its own tests, and the eval section comes back to that.

In a merge-commit repo, --first-parent gives you subjects like "Merge pull request #123". For PR titles and labels, gh pr list --state merged --json title,labels is the better source, and grouping by label takes a few more lines. The point is the shape. The skill that wraps this script shrinks to three sentences: run scripts/release_notes.sh <version>, write a two-sentence summary on top, use only what the script printed. The ticket list is no longer something the model produces. It is something the model reads.

If you convert a workflow like this, measure three things before and after: input tokens per run, wall-clock time, and how often the output needs a human fix. Those three numbers decide whether the next workflow is worth converting.

Converting between different file formats is a solved problem. You don’t need an artificial intelligence to do it for you.

My position is that most skills should be thin wrappers around scripts. If a SKILL.md is longer than the script it describes, one of them is in the wrong place. This takes the SKILL.md plus CLI idea I credited to Peter Steinberger one step further. Do not just point the agent at a tool, give it a tool shaped exactly like the job.

You also get reuse. CI, cron, a human terminal and any agent can call a script. A paragraph of instructions only works inside one context window.

Evals become boring, and boring is good

An eval is a test for AI behavior. You feed an input and check the output against criteria. Some checks are deterministic: valid JSON, contains a string and etc, others use a second model as a judge with a rubric, such as “did it invent facts?”. You run evals in CI on every prompt or skill change, like unit tests.

Two tools cover most of this. For example, promptfoo uses YAML with assertions like contains, is-json, javascript and llm-rubric. DeepEval feels like pytest and ships metrics such as G-Eval and tool correctness.

A free-form agent is hard to evaluate because it has a thousand acceptable paths. A scripted workflow splits the problem in two. The script gets ordinary unit tests. For the release notes script, that means four fixture repos: one with no ticket IDs, one that squash-merges, one where the new tag already points at HEAD, and one whose commits mention UTF-8 and SHA-256. Those are exactly the four bugs above so, the agent eval then checks three things, and only one of them needs a judge.

YAML
prompts:
  - file://prompts/release_agent.txt
providers:
  - openai:gpt-4o-mini
tests:
  - vars:
      request: "Write the release notes for v2.3"
      script_output: file://fixtures/release_notes_v2.3.txt
    assert:
      - type: contains
        value: "scripts/release_notes.sh v2.3"
      - type: javascript
        value: |
          // Same project keys as scripts/release_notes.sh
          const re = /\b(PAY|OPS|WEB)-\d+\b/g;
          const allowed = new Set(context.vars.script_output.match(re) || []);
          return (output.match(re) || []).every(id => allowed.has(id));
      - type: llm-rubric
        value: "The summary makes no claims that are not supported by: {{script_output}}"

The fixture is a saved run of the script, so both the ticket check and the judge have ground truth to compare against. The pattern must match the one in the script. With the generic pattern, a correct summary that says “now handles UTF-8 filenames” would fail the check.

That javascript assertion targets the failure human review is worst at. An invented ticket ID looks exactly like a real one. PAY-1432 and PAY-1423 pass a skim. A reviewer checks tone and formatting. Nobody opens the tracker to confirm each number. The assertion does it every time, on every change to the prompt or the skill, and it never gets tired on a Friday.

An earlier draft of this eval gave that check to the llm-rubric judge. That is the exact mistake this article argues against. Whether every ID in the output appears in a list is set membership. It has one right answer, so it gets code. The judge keeps the one question that needs reading: does the summary claim anything the script did not print? The eval splits along the same line as the workflow. Code for what has one right answer. A model for what needs judgment.

This test is also a simplification. The prompt receives the script output up front, so the agent never really calls anything. The contains assertion only checks that the command appears in the text. If your agent uses tool calling, the command lands in the tool call, not in the text, and that assertion will fail on a correct run. In that setup, assert on the tool call itself: the script name and the version argument. Then feed the fixture back as the tool result and run the other two checks on the final answer.

What to do on Monday

This is for the engineer who owns the agent setup for a team.

  1. Count the always-loaded tokens per tool, including imports, nested AGENTS.md files and skill descriptions from the repo, your home directory and plugins. Enforce a budget in CI.
  2. Give every rule exactly one home and delete the duplicates. If you bridge files between tools, bridge one source of truth, not copies.
  3. Turn the root file into an index.
  4. Move every “must always” rule into a hook.
  5. If you run your own agent, reorder context for the cache: stable content first, volatile content last. Then log the cache read tokens and confirm it worked.
  6. Take your most-used skill, write out its steps and script every step that needs no judgment. Test the script against the edge cases first. Then write the eval, with code for every check that has one right answer and a judge only for the rest.

Let the model be non-deterministic only where you are paying for its judgment. Everywhere else, you are paying for its mistakes.

References

Share on LinkedIn

Originally published on Medium on Oct 14, 2026. Claps and comments live there.

Comments

No comments yet. Be the first.

We email you a 6-digit code to publish it. Your email is never shown.

← More articles · follow by RSS