LOOP
Claude Prompt Caching Hit 0 of 1,412 Calls: A Timestamp Killed It
I turned on Claude prompt caching, shipped it, and moved on to the next thing. A week later I opened the usage logs to admire my savings.
cache_read_input_tokens: 0. On every call. 1,412 calls in a row.
The API never returned an error. My cache_control blocks looked fine. Every request quietly paid full price for the same 6,000-token prefix, over and over, and the response told me so in a field I wasn't reading. This is the autopsy: three separate bugs, each enough to kill the cache by itself, and the logging that would have caught them on day one.
TL;DR
Claude prompt caching only hits when the request prefix (tools, then system, then messages) is byte-identical up to the cache breakpoint. One changed character before the breakpoint means a full miss.
My system prompt started with Current time: 09:41:07. That alone gave me a 0% hit rate.
My tool list was built from a Python set. String hashing is randomized per process, so each of my 4 workers sent the tools in a different order. Hit rate stopped at 66%.
One prompt variant was too short to cache at all. No error, no warning, just cache_creation_input_tokens: 0.
After fixing all three: 94% of calls hit the cache and input token cost per session fell 58%. Log cache_read_input_tokens on every call or you will never know.
What was I caching?
The system in question is Preterview, a voice interview practice tool I built and run (full disclosure: I built it, it's at preterview.com). Each session is a back-and-forth with an AI interviewer in one of three styles, and every turn sends the same big chunk of context: interviewer persona, scoring rubric, and the candidate's parsed resume.
That chunk is about 6,000 tokens and it doesn't change during a session. A session averages 12 to 14 turns. Re-sending 6,000 identical tokens 14 times is the textbook case for prompt caching, which is why I added it in about ten minutes and assumed it worked.
Here's roughly what the call looked like:
``` resp = client.messages.create( model=MODEL, max_tokens=600, tools=build_tools(session), system=[ { "type": "text", "text": build_system_prompt(session), "cache_control": {"type": "ephemeral"}, } ], messages=session.history, ) ```
Looks right. It was not right.
How does Claude prompt caching decide what's a hit?
Claude prompt caching matches on an exact prefix. The request is treated as one ordered sequence: tool definitions first, then the system prompt, then messages. A cache_control breakpoint says "cache everything up to here." On the next call, if every token up to that breakpoint is identical, you get a cache read billed at a tenth of the normal input rate. If anything differs, you pay full price plus a write premium (1.25x on the default 5-minute TTL) to create a new entry.
So "the system prompt is basically the same" counts for nothing. It's either byte-identical or it's a miss.
Bug 1: why did a timestamp give me a 0% cache hit rate?
Because it was the first line of the system prompt. Months earlier I'd added this so the interviewer could say "good morning" at the right time of day:
``` def build_system_prompt(session): return ( f"Current time: {datetime.now():%H:%M:%S}\n" f"{PERSONAS[session.style]}\n" f"{RUBRIC}\n" f"Candidate resume:\n{session.resume_text}" ) ```
Seconds precision. Every turn had a different first line, so every turn had a different prefix, so every turn wrote a fresh cache entry that nobody ever read. I was paying the 1.25x write premium on 1,412 calls to cache prompts that expired unused. Caching made it more expensive than not caching.
The fix: move anything volatile after the breakpoint. The time of day went into a short second system block with no cache_control:
``` system=[ {"type": "text", "text": stable_prefix(session), "cache_control": {"type": "ephemeral"}}, {"type": "text", "text": f"Local time: {now:%H:%M}"}, ] ```
I redeployed and checked the logs. Hit rate: 66%. Better. Not 90-something, which is what the math said it should be.
Bug 2: why did Python set ordering break my cache?
Because Python randomizes string hashing per process, so iterating a set of strings gives a different order in each process. My tool list came from this:
``` enabled = {"score_answer", "ask_followup", "end_interview"} tools = [TOOL_DEFS[name] for name in enabled] ```
Inside one process the order is stable. But I run 4 uvicorn workers, and each worker got its own hash seed. Tools come first in the cache prefix, so four orderings meant four different prefixes for the same session.
This also explains 66% exactly. A session's turns land on workers at random. Each worker misses once per session (the first time it sees that session), then hits. With 4 workers and 12 turns, the expected number of distinct workers touched is 4 × (1 − 0.75^12) ≈ 3.87. So about 3.9 misses per 12 turns, about 8.1 hits, about 68%. My measured 66% sat right on top of the prediction, which is the most satisfying thing a bug has done for me all year.
The fix is boring:
``` tools = [TOOL_DEFS[name] for name in sorted(enabled)] ```
I also started serializing tool schemas from a frozen JSON file instead of building them from dicts at runtime, so nothing upstream could reorder keys either.
Hit rate after redeploy: 91%.
Bug 3: why does a short prompt never get cached?
Because there's a minimum cacheable length per model, and below it the API silently skips caching. On the model I use it's 1,024 tokens; smaller models need more, so check the docs for yours.
Two of my three interviewer styles had long persona blocks. The third, a deliberately terse "rapid-fire" style, came to about 900 tokens before the resume got appended. For candidates who uploaded a one-page resume, the whole stable prefix stayed under the threshold. Those calls showed cache_creation_input_tokens: 0 and cache_read_input_tokens: 0. No error. The breakpoint simply did nothing.
I moved the shared rubric above the persona so every style's prefix starts with the same 2,000+ tokens. That pushed every variant over the minimum, and it means the rubric portion is shared across styles too.
Final hit rate: 94%. The remaining 6% is the first turn of each session, which has to write the cache, plus sessions where the candidate paused longer than 5 minutes and the entry expired.
How much did fixing prompt caching actually save?
Input token cost per session fell 58%. Here's the math on an average 14-turn session, counted in "full-price token equivalents":
Two things jump out. First, the broken version was 18% worse than having no caching at all, because of the write premium. Second, history is now the biggest line item, so the next step is a second breakpoint on the conversation itself. You get up to 4 breakpoints per request; I was using one.
Latency improved too. Time to first token on cached turns dropped noticeably, which matters a lot in a voice product where every few hundred milliseconds feels like an awkward silence. I don't have a clean before/after on that because other changes shipped the same week, so I won't put a number on it.
How do you check if Claude prompt caching is working?
Log the three usage fields on every response and alert when reads stay at zero. This is the code I should have written on day one:
``` u = resp.usage log.info( "cache", read=u.cache_read_input_tokens, write=u.cache_creation_input_tokens, uncached=u.input_tokens, prefix_hash=hashlib.sha256(prefix_bytes).hexdigest()[:12], ) ```
The prefix_hash line is the one that cracked bug 2. I hash the serialized tools plus the stable system block locally and log it. When two turns of the same session showed different hashes, I diffed the two request bodies and the reordered tool list was right there in the first 40 bytes.
A checklist I now run on any cached prompt:
Nothing volatile before the breakpoint. No timestamps, request IDs, random few-shot picks, or "user is on page X."
Deterministic ordering everywhere. Sort sets. Freeze tool schemas. Watch out for dict merges that depend on input order.
Check the length. If cache_creation_input_tokens is 0 on the first call, you're under the minimum.
Put shared stuff first. Content shared across variants goes at the top so it caches across all of them.
Alert on a 0 read rate. A cache that silently stops working looks exactly like a cache that never worked.
So why was my Claude prompt caching hit rate 0%?
Claude prompt caching requires a byte-identical prefix up to the cache_control breakpoint, and my prefix changed on every single call because the first line of the system prompt was a seconds-precision timestamp. Fixing that exposed two more silent failures: Python's per-process hash randomization reordered my tool definitions across 4 workers (capping hits at 66%, exactly what the math predicted), and one prompt variant fell under the minimum cacheable length. Moving volatile text after the breakpoint, sorting the tools and padding the shared prefix took the hit rate from 0% to 94% and cut input cost per session by 58%. None of it threw an error. The only signal was cache_read_input_tokens, so log it.
Written by the developer behind Preterview, an interview prep platform.
LOOP
LOOP
LOOP
LOOP
LOOP
LOOP
LOOP
LOOP
LOOP