- /882023-10-21
[DDIA 04] Chapter 4 Encoding and Evolution
- /872023-10-20
[DDIA 03] Chapter 3 Storage and Retrieval
- /862023-10-20
[DDIA 02] Chapter 2 Data Models and Query Languages
- /852026-02-10
Cascade Speculative Drafting for Even Faster LLM Inference
Cascade Speculative Drafting speeds up the drafting itself: smaller models draft for the draft model (vertical cascade), and lighter models handle later positions, which are less likely to be accepted (horizontal cascade).
- /842026-02-09
LayerSkip Enabling Early Exit Inference and Self-Speculative Decoding
LayerSkip trains an LLM with layer dropout and an early-exit loss on a shared LM head, so early layers can predict tokens for early-exit inference and self-speculative decoding in one model.
- /832026-02-09
3-Model Speculative Decoding
PyramidSD inserts a qualifier model between the draft and target models in speculative decoding, and why the qualifier only helps when it accepts tokens by distribution similarity instead of exact match.
- /822026-02-08
Swift On-the-fly Self-speculative Decoding For LLM Inference Acceleration
SWIFT searches for layers to skip on the fly using the already generated context, so the LLM serves as its own draft model for self-speculative decoding without any training.
- /812026-02-08
Draft & Verify Lossless Large Language Model Acceleration via Self-Speculative Decoding
Self-Speculative Decoding drafts tokens with the LLM itself by skipping intermediate layers and verifies them with the full model; the skipped layers are chosen offline by Bayesian optimization, introduced in a short primer.
- /802026-02-07
Hierarchical Speculative Decoding with Dynamic Windows for Efficient Language Model Inference
HSDDW stops drafting when draft entropy exceeds a threshold tuned from rejected tokens, and adds a middle model that pre-verifies drafts before the target model sees them.
- /792026-02-07
CAS-Spec Cascade Adaptive Self-Speculative Decoding for On-the-Fly Lossless Inference Acceleration of LLMs
CAS-Spec builds a cascade of draft models from the target model itself with layer sparsity, activation quantization and early exit, then routes between them and sets draft lengths adaptively, with no training.