Speculative Decoding / Draft Proposes, Target Verifies
small
DRAFT
fast
t1
t2
t3
t4
t5
large
TARGET
parallel
t1
t2
t3
t4
t5
target checks all 5 in one parallel step / keeps the accepted prefix, discards the rest
0
tokens kept / step
2-3x
net speedup
One target pass yields
multiple tokens
using idle compute. Higher acceptance rate = bigger win.