LLM Inference Papers Index

My reading notes.

012026#

1.10203-0209#

• Cascade Speculative Drafting for Even Faster LLM Inference
• CAS-Spec Cascade Adaptive Self-Speculative Decoding for On-the-Fly Lossless Inference Acceleration of LLMs
• Draft & Verify Lossless Large Language Model Acceleration via Self-Speculative Decoding
• Swift On-the-fly Self-speculative Decoding For LLM Inference Acceleration
• 3-Model Speculative Decoding
• Hierarchical Speculative Decoding with Dynamic Windows for Efficient Language Model Inference
• LayerSkip Enabling Early Exit Inference and Self-Speculative Decoding
• AIConfigurator Lightning-Fast Configuration Optimization for Multi-Framework LLM Serving
• Revati Transparent GPU-Free Time-Warp Emulation for LLM Serving

1.20127-0202#

• FlexPrefill A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference
• XAttention Block Sparse Attention with Antidiagonal Scoring
• SLED A Speculative LLM Decoding Framework for Efficient Edge Serving
• R-Stitch Dynamic Trajectory Stitching for Efficient Reasoning
• Estimating LLM Uncertainty with Evidence
• Entropy Adaptive Decoding Dynamic Model Switching for Efficient Inference
• Think Big, Generate Quick LLM-to-SLM for Fast Autoregressive Decoding

022025#

2.1Remaining#

• Beyond the 80 20 Rule High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning
• KVCache Cache in the Wild Characterizing and Optimizing KVCache Cache at a Large Cloud Provider

2.21111-1117#

• LServe Efficient Long-sequence LLM Serving with Unified Sparse Attention
• QServe W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
• Quest Query-Aware Sparsity for Efficient Long-Context LLM Inference
• Dynamic Sparse Attention on Mobile SoCs
• A dynamic parallel method for performance optimization on hybrid CPUs
• SmoothQuant Accurate and Efficient Post-Training Quantization for Large Language Models
• DuoAttention Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
• Efficient Streaming Language Models with Attention Sinks
• KTransformers: CPU/GPU Hybrid Inference for MoE Models

2.31104-1110#

• EAGLE Speculative Sampling Requires Rethinking Feature Uncertainty

2.41028-1103#

• Aegaeon Effective GPU Pooling for Concurrent LLM Serving on the Market
• DistServe Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
• Splitwise Efficient Generative LLM Inference Using Phase Splitting
• Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve

2.50826-0901#

• ELMS Elasticized Large Language Models On Mobile Devices
• Scaling Up On-Device LLMs via Active-Weight Swapping Between DRAM and Flash

2.60819-0825#

• STI Turbocharge NLP Inference at the Edge via Elastic Pipelining
• EdgeMoE Empowering Sparse Large Language Models on Mobile Devices
• LLM as a System Service on Mobile Devices
• SmallThinker A Family of Efficient Large Language Models Natively Trained for Local Deployment
• HeteroLLM Accelerating Large Language Model Inference on Mobile SoCs with Heterogeneous AI Accelerators
• A Survey of Resource-efficient LLM and Multimodal Foundation Models
• H2O Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models

2.70812-0818#

• KV-Runahead Scalable Causal LLM Inference by Parallel Key-Value Cache Generation
• Striped Attention Faster Ring Attention for Causal Transformers
• Ring Attention with Blockwise Transformers for Near-Infinite Context
• TPI-LLM Serving 70B-scale LLMs Efficiently on Low-resource Mobile Devices
• LLM.int8() 8-bit Matrix Multiplication for Transformers at Scale

2.80729-0804#

• Fast On-device LLM Inference with NPUs
• Deja Vu Contextual Sparsity for Efficient LLMs at Inference Time
• PowerInfer-2 Fast Large Language Model Inference on a Smartphone
• LLM in a flash Efficient Large Language Model Inference with Limited Memory
• PowerInfer Fast Large Language Model Serving with a Consumer-grade GPU

2.90722-0728#

• AWQ Activation-aware Weight Quantization for LLM Compression and Acceleration
• FlexGen High-Throughput Generative Inference of Large Language Models with a Single GPU
• LoRA Low-Rank Adaptation of Large Language Models
• SpecInfer Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification
• EdgeLLM Fast On-Device LLM Inference With Speculative Decoding
• Efficient Memory Management for Large Language Model Serving with PagedAttention

2.100715-0721#

• A Survey on Efficient Inference for Large Language Models

2.11−0714#

• Orca A Distributed Serving System for Transformer-Based Generative Models
• EdgeShard Efficient LLM Inference via Collaborative Edge Computing
• ServerlessLLM Locality-Enhanced Serverless Inference for Large Language Models

03Uncategorized#

WIP 🚧

• Efficient Function-as-a-Service for Large Language Models with TIDAL
• LLM Preliminaries
• paper-reviews
• CE-LSLM
• Native Sparse Attention Hardware-Aligned and Natively Trainable Sparse Attention
• CacheGen
• R2R
• CITER Collaborative Inference for Efficient Large Language Model Decoding with Token-Level Routing
• Accelerating Large-Scale Reasoning Model Inference with Sparse Self-Speculative Decoding
• SpecAttn Speculating Sparse Attention
• Token Level Routing Inference System for Edge Devices
• C2C
• Learning to Decode Collaboratively with Multiple Language Models