01 / The Model
Prefix cache
The provider’s store for the unchanging beginning of your conversation, so it is not re-read from scratch on every request. What makes cache tokens cheap, and a reason to keep standing instructions stable.
How you’ll hear it
“Should we tweak the instructions every time?”
“Only if we must. Every change breaks the cache and we pay full price again.”
Avoid
Putting changing details, like today’s date, at the very top of long instructions.
The longer version
The prefix cache stores the processed beginning of a request. It only helps if that beginning is identical next time. Put stable content first and variable content last, and repeated requests become faster and cheaper.