04 / How It Goes Wrong
Attention budget
Each piece of context has a fixed amount of influence to spread around. Adding more context does not add more attention. It divides the same attention more thinly.
How you’ll hear it
“It missed the key clause.”
“It was one clause in forty pages. Its attention was spread thin.”
Avoid
Burying the important instruction in the middle of a long document.
The longer version
A model’s attention is finite and shared across everything in context. More material divides the same attention further. Put what matters most where it cannot be missed, and leave out what does not matter.