Tag

#inference

Reports tagged inference.

Reports

1 report

Nov 2026

11-01 speculative-decoding How speculative decoding checks ahead—and when it saves time A cheaper model supplies candidate text for the larger model to check together; speedup depends on how much of the draft survives and how much the shared check costs. 10 cited · of 145 sources