Channel index
Tags
- Channels
- 86
- Links
- 271
- Posts
- 158
- Peak
- LLM-Inference
Directory / by traffic
- CH-01LLM-Inference30CH-01 / LLM-Inference / 30 posts Latest: Decode GEMV on the CPU: Bandwidth, Compute or NUMA
- CH-02Paper Note18CH-02 / Paper Note / 18 posts Latest: Data Management in Microservices: State of the Practice, Challenges, and Research Directions
- CH-03Reading Note18CH-03 / Reading Note / 18 posts Latest: Build a Large Language Model (From Scratch) Reading Note
- CH-04Dev17CH-04 / Dev / 17 posts Latest: C++ Notes
- CH-05Design Data-Intensive Applications8CH-05 / Design Data-Intensive Applications / 8 posts Latest: [DDIA 09] Chapter 9 Consistency and Consensus
- CH-06Edge-Devices8CH-06 / Edge-Devices / 8 posts Latest: Dynamic Sparse Attention on Mobile SoCs
- CH-07100 Mistakes in Golang7CH-07 / 100 Mistakes in Golang / 7 posts Latest: [100 Mistakes in Golang 08-09] Chapter 8 & 9
- CH-08arXiv-247CH-08 / arXiv-24 / 7 posts Latest: A dynamic parallel method for performance optimization on hybrid CPUs
- CH-09arXiv-256CH-09 / arXiv-25 / 6 posts Latest: Estimating LLM Uncertainty with Evidence
- CH-10GPU6CH-10 / GPU / 6 posts Latest: CUDA Hardware Hierarchy: From SMs to Lanes
- CH-11KVCache6CH-11 / KVCache / 6 posts Latest: KVCache Cache in the Wild Characterizing and Optimizing KVCache Cache at a Large Cloud Provider
- CH-12LLM-Serving5CH-12 / LLM-Serving / 5 posts Latest: Revati Transparent GPU-Free Time-Warp Emulation for LLM Serving
- CH-13ML Compilers5CH-13 / ML Compilers / 5 posts Latest: TorchInductor Pipeline 与算子融合
- CH-14arXiv-234CH-14 / arXiv-23 / 4 posts Latest: Ring Attention with Blockwise Transformers for Near-Infinite Context
- CH-15Course4CH-15 / Course / 4 posts Latest: [Missing Semester 05] Lecture 5
- CH-16Hardware4CH-16 / Hardware / 4 posts Latest: Decode GEMV on the CPU: Bandwidth, Compute or NUMA
- CH-17Quantization4CH-17 / Quantization / 4 posts Latest: SmoothQuant Accurate and Efficient Post-Training Quantization for Large Language Models
- CH-18sparse-attention4CH-18 / sparse-attention / 4 posts Latest: XAttention Block Sparse Attention with Antidiagonal Scoring
- CH-19Talk4CH-19 / Talk / 4 posts Latest: Transaction Anomalies
- CH-20The Missing Semester4CH-20 / The Missing Semester / 4 posts Latest: [Missing Semester 05] Lecture 5
- CH-21torch.compile4CH-21 / torch.compile / 4 posts Latest: TorchInductor Pipeline 与算子融合
- CH-22ACL-243CH-22 / ACL-24 / 3 posts Latest: LayerSkip Enabling Early Exit Inference and Self-Speculative Decoding
- CH-23Commodity-GPU3CH-23 / Commodity-GPU / 3 posts Latest: LLM in a flash Efficient Large Language Model Inference with Limited Memory
- CH-24Distributed3CH-24 / Distributed / 3 posts Latest: Research: 6.824 Lab2B 中异常情况的分析
- CH-25ggml3CH-25 / ggml / 3 posts Latest: Decode GEMV on the CPU: Bandwidth, Compute or NUMA
- CH-26ICML-243CH-26 / ICML-24 / 3 posts Latest: Quest Query-Aware Sparsity for Efficient Long-Context LLM Inference
- CH-27Long-Context3CH-27 / Long-Context / 3 posts Latest: Quest Query-Aware Sparsity for Efficient Long-Context LLM Inference
- CH-28OSDI-243CH-28 / OSDI-24 / 3 posts Latest: DistServe Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
- CH-29Rust3CH-29 / Rust / 3 posts Latest: Using Rust
- CH-30Speculative-Decoding3CH-30 / Speculative-Decoding / 3 posts Latest: Cascade Speculative Drafting for Even Faster LLM Inference
- CH-31Tensor-Parallelism3CH-31 / Tensor-Parallelism / 3 posts Latest: TPI-LLM Serving 70B-scale LLMs Efficiently on Low-resource Mobile Devices
- CH-32TileLang3CH-32 / TileLang / 3 posts Latest: Learning TileLang with GEMV
- CH-33arXiv-26012CH-33 / arXiv-2601 / 2 posts Latest: Revati Transparent GPU-Free Time-Warp Emulation for LLM Serving
- CH-34auto-tuning2CH-34 / auto-tuning / 2 posts Latest: Revati Transparent GPU-Free Time-Warp Emulation for LLM Serving
- CH-35Distributed-Transactions2CH-35 / Distributed-Transactions / 2 posts Latest: Sonata Multi-Database Transactions Made Fast and Serializable
- CH-36ICML-232CH-36 / ICML-23 / 2 posts Latest: SmoothQuant Accurate and Efficient Post-Training Quantization for Large Language Models
- CH-37ICML-252CH-37 / ICML-25 / 2 posts Latest: XAttention Block Sparse Attention with Antidiagonal Scoring
- CH-38IEEE-TMC-252CH-38 / IEEE-TMC-25 / 2 posts Latest: EdgeMoE Empowering Sparse Large Language Models on Mobile Devices
- CH-39NeurIPS-252CH-39 / NeurIPS-25 / 2 posts Latest: CAS-Spec Cascade Adaptive Self-Speculative Decoding for On-the-Fly Lossless Inference Acceleration of LLMs
- CH-40NPU2CH-40 / NPU / 2 posts Latest: HeteroLLM Accelerating Large Language Model Inference on Mobile SoCs with Heterogeneous AI Accelerators
- CH-41Survey2CH-41 / Survey / 2 posts Latest: A Survey of Resource-efficient LLM and Multimodal Foundation Models
- CH-42VLDB-252CH-42 / VLDB-25 / 2 posts Latest: Sonata Multi-Database Transactions Made Fast and Serializable
- CH-43vLLM2CH-43 / vLLM / 2 posts Latest: vLLM Hybrid KV Cache
- CH-446.8241CH-44 / 6.824 / 1 post Latest: Research: 6.824 Lab2B 中异常情况的分析
- CH-45ACLFindings-251CH-45 / ACLFindings-25 / 1 post Latest: Hierarchical Speculative Decoding with Dynamic Windows for Efficient Language Model Inference
- CH-46arXiv-211CH-46 / arXiv-21 / 1 post Latest: LoRA Low-Rank Adaptation of Large Language Models
- CH-47arXiv-25021CH-47 / arXiv-2502 / 1 post Latest: Entropy Adaptive Decoding Dynamic Model Switching for Efficient Inference
- CH-48arXiv-25041CH-48 / arXiv-2504 / 1 post Latest: Token Level Routing Inference System for Edge Devices
- CH-49arXiv-25071CH-49 / arXiv-2507 / 1 post Latest: R-Stitch Dynamic Trajectory Stitching for Efficient Reasoning
- CH-50ASPLOS-231CH-50 / ASPLOS-23 / 1 post Latest: STI Turbocharge NLP Inference at the Edge via Elastic Pipelining
- CH-51ASPLOS-241CH-51 / ASPLOS-24 / 1 post Latest: SpecInfer Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification
- CH-52ATC-251CH-52 / ATC-25 / 1 post Latest: KVCache Cache in the Wild Characterizing and Optimizing KVCache Cache at a Large Cloud Provider
- CH-53C++1CH-53 / C++ / 1 post Latest: C++ Notes
- CH-54Concurrency1CH-54 / Concurrency / 1 post Latest: A Piece of: ThreadLocal
- CH-55CUDA1CH-55 / CUDA / 1 post Latest: CUDA Hardware Hierarchy: From SMs to Lanes
- CH-56FaaS1CH-56 / FaaS / 1 post Latest: Watchdogs in OpenFaaS
- CH-57Golang1CH-57 / Golang / 1 post Latest: A Piece of: Golang Profile
- CH-58ICLR-241CH-58 / ICLR-24 / 1 post Latest: Efficient Streaming Language Models with Attention Sinks
- CH-59ICLR-251CH-59 / ICLR-25 / 1 post Latest: Swift On-the-fly Self-speculative Decoding For LLM Inference Acceleration
- CH-60ICML-24-Workshop1CH-60 / ICML-24-Workshop / 1 post Latest: Think Big, Generate Quick LLM-to-SLM for Fast Autoregressive Decoding
- CH-61IEEE-IoT-J-251CH-61 / IEEE-IoT-J-25 / 1 post Latest: EdgeShard Efficient LLM Inference via Collaborative Edge Computing
- CH-62ISCA241CH-62 / ISCA24 / 1 post Latest: Splitwise Efficient Generative LLM Inference Using Phase Splitting
- CH-63Java1CH-63 / Java / 1 post Latest: A Piece of: ThreadLocal
- CH-64LLM-Tuning1CH-64 / LLM-Tuning / 1 post Latest: LoRA Low-Rank Adaptation of Large Language Models
- CH-65Math1CH-65 / Math / 1 post Latest: Math for ML
- CH-66Memory-Management1CH-66 / Memory-Management / 1 post Latest: Efficient Memory Management for Large Language Model Serving with PagedAttention
- CH-67Misc.1CH-67 / Misc. / 1 post Latest: LLM Generated Content
- CH-68MLSys-251CH-68 / MLSys-25 / 1 post Latest: QServe W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
- CH-69Model-Architecture1CH-69 / Model-Architecture / 1 post Latest: Qwen3.5 Dense Model Architecture
- CH-70MoE1CH-70 / MoE / 1 post Latest: EdgeMoE Empowering Sparse Large Language Models on Mobile Devices
- CH-71NeurIPS-241CH-71 / NeurIPS-24 / 1 post Latest: Cascade Speculative Drafting for Even Faster LLM Inference
- CH-72NeurIPS-25-Workshop1CH-72 / NeurIPS-25-Workshop / 1 post Latest: 3-Model Speculative Decoding
- CH-73NIPS-231CH-73 / NIPS-23 / 1 post Latest: H2O Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models
- CH-74NIPS20221CH-74 / NIPS2022 / 1 post Latest: LLM.int8() 8-bit Matrix Multiplication for Transformers at Scale
- CH-75OSDI-221CH-75 / OSDI-22 / 1 post Latest: Orca A Distributed Serving System for Transformer-Based Generative Models
- CH-76PyTorch1CH-76 / PyTorch / 1 post Latest: PyTorch Notes
- CH-77Schedule1CH-77 / Schedule / 1 post Latest: Orca A Distributed Serving System for Transformer-Based Generative Models
- CH-78SEC-251CH-78 / SEC-25 / 1 post Latest: SLED A Speculative LLM Decoding Framework for Efficient Edge Serving
- CH-79Sequence-Parallelism1CH-79 / Sequence-Parallelism / 1 post Latest: Ring Attention with Blockwise Transformers for Near-Infinite Context
- CH-80Serverless1CH-80 / Serverless / 1 post Latest: ServerlessLLM Locality-Enhanced Serverless Inference for Large Language Models
- CH-81SOSP-241CH-81 / SOSP-24 / 1 post Latest: PowerInfer Fast Large Language Model Serving with a Consumer-grade GPU
- CH-82SOSP-251CH-82 / SOSP-25 / 1 post Latest: KTransformers: CPU/GPU Hybrid Inference for MoE Models
- CH-83Sparsity1CH-83 / Sparsity / 1 post Latest: Scaling Up On-Device LLMs via Active-Weight Swapping Between DRAM and Flash
- CH-84Theoretical1CH-84 / Theoretical / 1 post Latest: Deja Vu Contextual Sparsity for Efficient LLMs at Inference Time
- CH-85Triton1CH-85 / Triton / 1 post Latest: Triton Notes: Tutorials and Kernel Paradigms
- CH-86TSC-241CH-86 / TSC-24 / 1 post Latest: TPI-LLM Serving 70B-scale LLMs Efficiently on Low-resource Mobile Devices
Point at a channel to read it
CH-01LLM-Inference30↑
- 2026-11-19Decode GEMV on the CPU: Bandwidth, Compute or NUMAKKZ-TN-158
- 2026-11-04vLLM Hybrid KV CacheKKZ-TN-156
- 2026-11-02NVLink vs PCIe in Practice: Bandwidth and LLM ServingKKZ-TN-153
- 2026-10-29ggml Q4_0 vec_dot WalkthroughKKZ-TN-151
- 2026-10-24KTransformers: CPU/GPU Hybrid Inference for MoE ModelsKKZ-TN-150
- 2026-03-18ggml Memory Layout and AllocationKKZ-TN-141
- 2025-11-15LServe Efficient Long-sequence LLM Serving with Unified Sparse AttentionKKZ-TN-112
- 2025-11-13DuoAttention Efficient Long-Context LLM Inference with Retrieval and Streaming HeadsKKZ-TN-109
- 2025-11-10EAGLE Speculative Sampling Requires Rethinking Feature UncertaintyKKZ-TN-106
- 2025-09-01Scaling Up On-Device LLMs via Active-Weight Swapping Between DRAM and FlashKKZ-TN-101
- 2025-08-27ELMS Elasticized Large Language Models On Mobile DevicesKKZ-TN-100
- 2025-08-25STI Turbocharge NLP Inference at the Edge via Elastic PipeliningKKZ-TN-099
- 2025-08-24HeteroLLM Accelerating Large Language Model Inference on Mobile SoCs with Heterogeneous AI AcceleratorsKKZ-TN-097
- 2025-08-24EdgeMoE Empowering Sparse Large Language Models on Mobile DevicesKKZ-TN-096
- 2025-08-17TPI-LLM Serving 70B-scale LLMs Efficiently on Low-resource Mobile DevicesKKZ-TN-092
- 2025-08-04LLM PreliminariesKKZ-TN-084
- 2025-08-04Fast On-device LLM Inference with NPUsKKZ-TN-083
- 2025-08-04Deja Vu Contextual Sparsity for Efficient LLMs at Inference TimeKKZ-TN-082
- 2025-07-30LLM in a flash Efficient Large Language Model Inference with Limited MemoryKKZ-TN-079
- 2025-07-29PowerInfer-2 Fast Large Language Model Inference on a SmartphoneKKZ-TN-078
- 2025-07-28PowerInfer Fast Large Language Model Serving with a Consumer-grade GPUKKZ-TN-077
- 2025-07-28AWQ Activation-aware Weight Quantization for LLM Compression and AccelerationKKZ-TN-075
- 2025-07-25SpecInfer Accelerating Large Language Model Serving with Tree-based Speculative Inference and VerificationKKZ-TN-074
- 2025-07-25FlexGen High-Throughput Generative Inference of Large Language Models with a Single GPUKKZ-TN-072
- 2025-07-23Efficient Memory Management for Large Language Model Serving with PagedAttentionKKZ-TN-071
- 2025-07-23EdgeLLM Fast On-Device LLM Inference With Speculative DecodingKKZ-TN-070
- 2025-07-20A Survey on Efficient Inference for Large Language ModelsKKZ-TN-069
- 2025-07-02Orca A Distributed Serving System for Transformer-Based Generative ModelsKKZ-TN-066
- 2025-07-01EdgeShard Efficient LLM Inference via Collaborative Edge ComputingKKZ-TN-065
- 2025-06-28ServerlessLLM Locality-Enhanced Serverless Inference for Large Language ModelsKKZ-TN-064
CH-02Paper Note18↑
- 2024-04-15Data Management in Microservices: State of the Practice, Challenges, and Research DirectionsKKZ-TN-034
- 2024-03-25Paper Note: Taking Omid to the Clouds: Fast, Scalable Transactions for Real-Time Cloud AnalyticsKKZ-TN-032
- 2024-03-22Paper Note: Omid, Reloaded: Scalable and Highly-Available Transaction ProcessingKKZ-TN-031
- 2024-01-22Paper Note: Towards Transaction as a ServiceKKZ-TN-029
- 2023-12-09Paper Note: Scalable Distributed Transactions across Heterogeneous StoresKKZ-TN-028
- 2023-12-08Paper Note: GRIT: Consistent Distributed Transactions across Polyglot Microservices with Multiple DatabasesKKZ-TN-027
- 2023-12-05Paper Note: How to Read a PaperKKZ-TN-026
- 2023-11-30Paper Note: Ad Hoc Transactions in Web Applications: The Good, the Bad, and the UglyKKZ-TN-024
- 2023-11-23Paper Note: Cobra: Making Transactional Key-Value Stores Verifiably SerializableKKZ-TN-021
- 2023-11-22Paper Note: Epoxy: ACID Transactions Across Diverse Data StoresKKZ-TN-020
- 2023-11-15Paper Note: Zab: High-performance broadcast for primary-backup systemsKKZ-TN-019
- 2023-11-12Paper Note: CAP Twelve Years Later: How the "Rules" have ChangedKKZ-TN-017
- 2023-11-06Paper Note: Chain ReplicationKKZ-TN-014
- 2023-11-04Paper Note: Time, clocks, and the orderingKKZ-TN-013
- 2023-10-31Paper Note: ChubbyKKZ-TN-010
- 2023-10-30Paper Note: ZooKeeperKKZ-TN-009
- 2023-10-30Paper Note: MapReduceKKZ-TN-008
- 2023-10-30Paper Note: BigTableKKZ-TN-007
CH-03Reading Note18↑
- 2025-05-20Build a Large Language Model (From Scratch) Reading NoteKKZ-TN-062
- 2025-03-31Tips and Guidance for WritingKKZ-TN-057
- 2025-03-19[100 Mistakes in Golang 08-09] Chapter 8 & 9KKZ-TN-053
- 2025-03-07[100 Mistakes in Golang 07] Chapter 7KKZ-TN-052
- 2025-03-04[100 Mistakes in Golang 06] Chapter 6KKZ-TN-051
- 2025-03-03[100 Mistakes in Golang 05] Chapter 5KKZ-TN-050
- 2025-02-20[100 Mistakes in Golang 03] Chapter 3KKZ-TN-049
- 2025-02-20[100 Mistakes in Golang 04] Chapter 4KKZ-TN-048
- 2025-02-20[100 Mistakes in Golang 02] Chapter 2KKZ-TN-047
- 2024-02-10Programming Rust Book NoteKKZ-TN-030
- 2023-11-13[DDIA 09] Chapter 9 Consistency and ConsensusKKZ-TN-018
- 2023-11-02[DDIA 08] Chapter 8 The Trouble with Distributed SystemsKKZ-TN-011
- 2023-10-29[DDIA 07] Chapter 7 TransactionsKKZ-TN-006
- 2023-10-24[DDIA 06] Chapter 6 PartitioningKKZ-TN-005
- 2023-10-23[DDIA 05] Chapter 5 ReplicationKKZ-TN-004
- 2023-10-21[DDIA 04] Chapter 4 Encoding and EvolutionKKZ-TN-003
- 2023-10-20[DDIA 03] Chapter 3 Storage and RetrievalKKZ-TN-002
- 2023-10-20[DDIA 02] Chapter 2 Data Models and Query LanguagesKKZ-TN-001
CH-04Dev17↑
- 2026-03-18C++ NotesKKZ-TN-140
- 2025-10-04Sync Dot Files with ChezmoiKKZ-TN-102
- 2025-08-08Git EssentialsKKZ-TN-087
- 2025-08-06VS Code EssentialsKKZ-TN-085
- 2025-05-26NumPy NotesKKZ-TN-063
- 2025-05-19Useful Websites and ExtensionsKKZ-TN-061
- 2025-04-23Common Golang MistakesKKZ-TN-060
- 2025-04-17CLI & Server NotesKKZ-TN-059
- 2025-04-04Jujutsu (jj) TutorialKKZ-TN-058
- 2025-03-30Pass the WallKKZ-TN-056
- 2025-03-29Docker Network IntroductionKKZ-TN-055
- 2025-02-04Handling Golang ErrorsKKZ-TN-046
- 2025-01-28Golang Struct and InterfacesKKZ-TN-045
- 2024-11-26From Clicks to Commands: A Windows User's Survival GuideKKZ-TN-043
- 2024-11-21macOS Setup & WorkflowKKZ-TN-042
- 2024-05-13A Piece of: LogsKKZ-TN-035
- 2023-11-25Note: WSL2 Mirrored 网络模式下异常情况总结KKZ-TN-022
CH-05Design Data-Intensive Applications8↑
- 2023-11-13[DDIA 09] Chapter 9 Consistency and ConsensusKKZ-TN-018
- 2023-11-02[DDIA 08] Chapter 8 The Trouble with Distributed SystemsKKZ-TN-011
- 2023-10-29[DDIA 07] Chapter 7 TransactionsKKZ-TN-006
- 2023-10-24[DDIA 06] Chapter 6 PartitioningKKZ-TN-005
- 2023-10-23[DDIA 05] Chapter 5 ReplicationKKZ-TN-004
- 2023-10-21[DDIA 04] Chapter 4 Encoding and EvolutionKKZ-TN-003
- 2023-10-20[DDIA 03] Chapter 3 Storage and RetrievalKKZ-TN-002
- 2023-10-20[DDIA 02] Chapter 2 Data Models and Query LanguagesKKZ-TN-001
CH-06Edge-Devices8↑
- 2025-11-11Dynamic Sparse Attention on Mobile SoCsKKZ-TN-108
- 2025-09-01Scaling Up On-Device LLMs via Active-Weight Swapping Between DRAM and FlashKKZ-TN-101
- 2025-08-27ELMS Elasticized Large Language Models On Mobile DevicesKKZ-TN-100
- 2025-08-25SmallThinker A Family of Efficient Large Language Models Natively Trained for Local DeploymentKKZ-TN-098
- 2025-08-04Fast On-device LLM Inference with NPUsKKZ-TN-083
- 2025-07-29PowerInfer-2 Fast Large Language Model Inference on a SmartphoneKKZ-TN-078
- 2025-07-23EdgeLLM Fast On-Device LLM Inference With Speculative DecodingKKZ-TN-070
- 2025-07-01EdgeShard Efficient LLM Inference via Collaborative Edge ComputingKKZ-TN-065
CH-07100 Mistakes in Golang7↑
- 2025-03-19[100 Mistakes in Golang 08-09] Chapter 8 & 9KKZ-TN-053
- 2025-03-07[100 Mistakes in Golang 07] Chapter 7KKZ-TN-052
- 2025-03-04[100 Mistakes in Golang 06] Chapter 6KKZ-TN-051
- 2025-03-03[100 Mistakes in Golang 05] Chapter 5KKZ-TN-050
- 2025-02-20[100 Mistakes in Golang 03] Chapter 3KKZ-TN-049
- 2025-02-20[100 Mistakes in Golang 04] Chapter 4KKZ-TN-048
- 2025-02-20[100 Mistakes in Golang 02] Chapter 2KKZ-TN-047
CH-08arXiv-247↑
- 2025-11-11A dynamic parallel method for performance optimization on hybrid CPUsKKZ-TN-107
- 2025-08-27ELMS Elasticized Large Language Models On Mobile DevicesKKZ-TN-100
- 2025-08-21A Survey of Resource-efficient LLM and Multimodal Foundation ModelsKKZ-TN-094
- 2025-08-18LLM as a System Service on Mobile DevicesKKZ-TN-093
- 2025-08-17Striped Attention Faster Ring Attention for Causal TransformersKKZ-TN-091
- 2025-07-29PowerInfer-2 Fast Large Language Model Inference on a SmartphoneKKZ-TN-078
- 2025-07-20A Survey on Efficient Inference for Large Language ModelsKKZ-TN-069
CH-09arXiv-256↑
- 2026-02-02Estimating LLM Uncertainty with EvidenceKKZ-TN-125
- 2025-11-15LServe Efficient Long-sequence LLM Serving with Unified Sparse AttentionKKZ-TN-112
- 2025-11-11Dynamic Sparse Attention on Mobile SoCsKKZ-TN-108
- 2025-09-01Scaling Up On-Device LLMs via Active-Weight Swapping Between DRAM and FlashKKZ-TN-101
- 2025-08-25SmallThinker A Family of Efficient Large Language Models Natively Trained for Local DeploymentKKZ-TN-098
- 2025-08-24HeteroLLM Accelerating Large Language Model Inference on Mobile SoCs with Heterogeneous AI AcceleratorsKKZ-TN-097
CH-10GPU6↑
- 2026-11-16CUDA Hardware Hierarchy: From SMs to LanesKKZ-TN-157
- 2026-11-04Learning TileLang with GEMVKKZ-TN-154
- 2026-11-02NVLink vs PCIe in Practice: Bandwidth and LLM ServingKKZ-TN-153
- 2026-10-30GPU Server Topology: NVLink, PCIe and NUMAKKZ-TN-152
- 2026-01-07Triton Notes: Tutorials and Kernel ParadigmsKKZ-TN-118
- 2025-08-24HeteroLLM Accelerating Large Language Model Inference on Mobile SoCs with Heterogeneous AI AcceleratorsKKZ-TN-097
CH-11KVCache6↑
- 2025-12-06KVCache Cache in the Wild Characterizing and Optimizing KVCache Cache at a Large Cloud ProviderKKZ-TN-116
- 2025-11-15LServe Efficient Long-sequence LLM Serving with Unified Sparse AttentionKKZ-TN-112
- 2025-11-13Quest Query-Aware Sparsity for Efficient Long-Context LLM InferenceKKZ-TN-111
- 2025-11-13Efficient Streaming Language Models with Attention SinksKKZ-TN-110
- 2025-11-13DuoAttention Efficient Long-Context LLM Inference with Retrieval and Streaming HeadsKKZ-TN-109
- 2025-08-21H2O Heavy-Hitter Oracle for Efficient Generative Inference of Large Language ModelsKKZ-TN-095
CH-12LLM-Serving5↑
- 2026-02-05Revati Transparent GPU-Free Time-Warp Emulation for LLM ServingKKZ-TN-128
- 2026-02-05AIConfigurator Lightning-Fast Configuration Optimization for Multi-Framework LLM ServingKKZ-TN-127
- 2025-11-03Splitwise Efficient Generative LLM Inference Using Phase SplittingKKZ-TN-105
- 2025-11-03DistServe Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model ServingKKZ-TN-104
- 2025-11-01Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-ServeKKZ-TN-103
CH-13ML Compilers5↑
CH-14arXiv-234↑
- 2025-08-17Ring Attention with Blockwise Transformers for Near-Infinite ContextKKZ-TN-090
- 2025-08-04Deja Vu Contextual Sparsity for Efficient LLMs at Inference TimeKKZ-TN-082
- 2025-07-28AWQ Activation-aware Weight Quantization for LLM Compression and AccelerationKKZ-TN-075
- 2025-07-23Efficient Memory Management for Large Language Model Serving with PagedAttentionKKZ-TN-071
CH-15Course4↑
CH-16Hardware4↑
CH-17Quantization4↑
- 2025-11-16SmoothQuant Accurate and Efficient Post-Training Quantization for Large Language ModelsKKZ-TN-114
- 2025-11-16QServe W4A8KV4 Quantization and System Co-design for Efficient LLM ServingKKZ-TN-113
- 2025-08-12LLM.int8() 8-bit Matrix Multiplication for Transformers at ScaleKKZ-TN-088
- 2025-07-28AWQ Activation-aware Weight Quantization for LLM Compression and AccelerationKKZ-TN-075
CH-18sparse-attention4↑
- 2026-01-29XAttention Block Sparse Attention with Antidiagonal ScoringKKZ-TN-124
- 2026-01-29FlexPrefill A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence InferenceKKZ-TN-123
- 2025-11-15LServe Efficient Long-sequence LLM Serving with Unified Sparse AttentionKKZ-TN-112
- 2025-11-13Quest Query-Aware Sparsity for Efficient Long-Context LLM InferenceKKZ-TN-111
CH-19Talk4↑
CH-20The Missing Semester4↑
CH-21torch.compile4↑
CH-22ACL-243↑
CH-23Commodity-GPU3↑
CH-24Distributed3↑
CH-25ggml3↑
CH-26ICML-243↑
CH-27Long-Context3↑
CH-28OSDI-243↑
- 2025-11-03DistServe Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model ServingKKZ-TN-104
- 2025-11-01Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-ServeKKZ-TN-103
- 2025-06-28ServerlessLLM Locality-Enhanced Serverless Inference for Large Language ModelsKKZ-TN-064