- /862023-10-20
[DDIA 02] Chapter 2 Data Models and Query Languages
- /852026-02-10
Cascade Speculative Drafting for Even Faster LLM Inference
Cascade Speculative Drafting speeds up the drafting itself: smaller models draft for the draft model (vertical cascade), and lighter models handle later positions, which are less likely to be accepted (horizontal cascade).
- /842026-02-09
LayerSkip Enabling Early Exit Inference and Self-Speculative Decoding
LayerSkip trains an LLM with layer dropout and an early-exit loss on a shared LM head, so early layers can predict tokens for early-exit inference and self-speculative decoding in one model.
- /832026-02-09
3-Model Speculative Decoding
PyramidSD inserts a qualifier model between the draft and target models in speculative decoding, and why the qualifier only helps when it accepts tokens by distribution similarity instead of exact match.
- /822026-02-08
Swift On-the-fly Self-speculative Decoding For LLM Inference Acceleration
SWIFT searches for layers to skip on the fly using the already generated context, so the LLM serves as its own draft model for self-speculative decoding without any training.
- /812026-02-08
Draft & Verify Lossless Large Language Model Acceleration via Self-Speculative Decoding
Self-Speculative Decoding drafts tokens with the LLM itself by skipping intermediate layers and verifies them with the full model; the skipped layers are chosen offline by Bayesian optimization, introduced in a short primer.
- /802026-02-07
Hierarchical Speculative Decoding with Dynamic Windows for Efficient Language Model Inference
HSDDW stops drafting when draft entropy exceeds a threshold tuned from rejected tokens, and adds a middle model that pre-verifies drafts before the target model sees them.
- /792026-02-07
CAS-Spec Cascade Adaptive Self-Speculative Decoding for On-the-Fly Lossless Inference Acceleration of LLMs
CAS-Spec builds a cascade of draft models from the target model itself with layer sparsity, activation quantization and early exit, then routes between them and sets draft lengths adaptively, with no training.
- /782026-02-05
Revati Transparent GPU-Free Time-Warp Emulation for LLM Serving
Revati runs unmodified vLLM and SGLang without GPUs by intercepting CUDA calls and advancing a shared virtual clock by predicted kernel times, making configuration search cheap without a separate simulator.
- /772026-02-05
AIConfigurator Lightning-Fast Configuration Optimization for Multi-Framework LLM Serving
AIConfigurator profiles GEMM, attention, communication and memory primitives once per GPU and composes them to predict serving performance, searching static, aggregated and disaggregated deployments without benchmarking each one.