Prompt Caching / Static Prefix + Dynamic Suffix
Call #1
cache MISS
STATIC PREFIX · prefill + WRITE cache
user query
Call #2..N
cache HIT
STATIC PREFIX · reused, skip prefill
new query
Input cost on hit
full
→ -90%
Latency to first token
full
→ -50%
A cache hit needs an
identical prefix
. Order prompt blocks most-static first, dynamic last.