Channel index

Tags

Channels
86
Links
271
Posts
158
Peak
LLM-Inference
  1. CH-01LLM-Inference30CH-01 / LLM-Inference / 30 posts Latest: Decode GEMV on the CPU: Bandwidth, Compute or NUMA
  2. CH-02Paper Note18CH-02 / Paper Note / 18 posts Latest: Data Management in Microservices: State of the Practice, Challenges, and Research Directions
  3. CH-03Reading Note18CH-03 / Reading Note / 18 posts Latest: Build a Large Language Model (From Scratch) Reading Note
  4. CH-04Dev17CH-04 / Dev / 17 posts Latest: C++ Notes
  5. CH-05Design Data-Intensive Applications8CH-05 / Design Data-Intensive Applications / 8 posts Latest: [DDIA 09] Chapter 9 Consistency and Consensus
  6. CH-06Edge-Devices8CH-06 / Edge-Devices / 8 posts Latest: Dynamic Sparse Attention on Mobile SoCs
  7. CH-07100 Mistakes in Golang7CH-07 / 100 Mistakes in Golang / 7 posts Latest: [100 Mistakes in Golang 08-09] Chapter 8 & 9
  8. CH-08arXiv-247CH-08 / arXiv-24 / 7 posts Latest: A dynamic parallel method for performance optimization on hybrid CPUs
  9. CH-09arXiv-256CH-09 / arXiv-25 / 6 posts Latest: Estimating LLM Uncertainty with Evidence
  10. CH-10GPU6CH-10 / GPU / 6 posts Latest: CUDA Hardware Hierarchy: From SMs to Lanes
  11. CH-11KVCache6CH-11 / KVCache / 6 posts Latest: KVCache Cache in the Wild Characterizing and Optimizing KVCache Cache at a Large Cloud Provider
  12. CH-12LLM-Serving5CH-12 / LLM-Serving / 5 posts Latest: Revati Transparent GPU-Free Time-Warp Emulation for LLM Serving
  13. CH-13ML Compilers5CH-13 / ML Compilers / 5 posts Latest: TorchInductor Pipeline 与算子融合
  14. CH-14arXiv-234CH-14 / arXiv-23 / 4 posts Latest: Ring Attention with Blockwise Transformers for Near-Infinite Context
  15. CH-15Course4CH-15 / Course / 4 posts Latest: [Missing Semester 05] Lecture 5
  16. CH-16Hardware4CH-16 / Hardware / 4 posts Latest: Decode GEMV on the CPU: Bandwidth, Compute or NUMA
  17. CH-17Quantization4CH-17 / Quantization / 4 posts Latest: SmoothQuant Accurate and Efficient Post-Training Quantization for Large Language Models
  18. CH-18sparse-attention4CH-18 / sparse-attention / 4 posts Latest: XAttention Block Sparse Attention with Antidiagonal Scoring
  19. CH-19Talk4CH-19 / Talk / 4 posts Latest: Transaction Anomalies
  20. CH-20The Missing Semester4CH-20 / The Missing Semester / 4 posts Latest: [Missing Semester 05] Lecture 5
  21. CH-21torch.compile4CH-21 / torch.compile / 4 posts Latest: TorchInductor Pipeline 与算子融合
  22. CH-22ACL-243CH-22 / ACL-24 / 3 posts Latest: LayerSkip Enabling Early Exit Inference and Self-Speculative Decoding
  23. CH-23Commodity-GPU3CH-23 / Commodity-GPU / 3 posts Latest: LLM in a flash Efficient Large Language Model Inference with Limited Memory
  24. CH-24Distributed3CH-24 / Distributed / 3 posts Latest: Research: 6.824 Lab2B 中异常情况的分析
  25. CH-25ggml3CH-25 / ggml / 3 posts Latest: Decode GEMV on the CPU: Bandwidth, Compute or NUMA
  26. CH-26ICML-243CH-26 / ICML-24 / 3 posts Latest: Quest Query-Aware Sparsity for Efficient Long-Context LLM Inference
  27. CH-27Long-Context3CH-27 / Long-Context / 3 posts Latest: Quest Query-Aware Sparsity for Efficient Long-Context LLM Inference
  28. CH-28OSDI-243CH-28 / OSDI-24 / 3 posts Latest: DistServe Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
  29. CH-29Rust3CH-29 / Rust / 3 posts Latest: Using Rust
  30. CH-30Speculative-Decoding3CH-30 / Speculative-Decoding / 3 posts Latest: Cascade Speculative Drafting for Even Faster LLM Inference
  31. CH-31Tensor-Parallelism3CH-31 / Tensor-Parallelism / 3 posts Latest: TPI-LLM Serving 70B-scale LLMs Efficiently on Low-resource Mobile Devices
  32. CH-32TileLang3CH-32 / TileLang / 3 posts Latest: Learning TileLang with GEMV
  33. CH-33arXiv-26012CH-33 / arXiv-2601 / 2 posts Latest: Revati Transparent GPU-Free Time-Warp Emulation for LLM Serving
  34. CH-34auto-tuning2CH-34 / auto-tuning / 2 posts Latest: Revati Transparent GPU-Free Time-Warp Emulation for LLM Serving
  35. CH-35Distributed-Transactions2CH-35 / Distributed-Transactions / 2 posts Latest: Sonata Multi-Database Transactions Made Fast and Serializable
  36. CH-36ICML-232CH-36 / ICML-23 / 2 posts Latest: SmoothQuant Accurate and Efficient Post-Training Quantization for Large Language Models
  37. CH-37ICML-252CH-37 / ICML-25 / 2 posts Latest: XAttention Block Sparse Attention with Antidiagonal Scoring
  38. CH-38IEEE-TMC-252CH-38 / IEEE-TMC-25 / 2 posts Latest: EdgeMoE Empowering Sparse Large Language Models on Mobile Devices
  39. CH-39NeurIPS-252CH-39 / NeurIPS-25 / 2 posts Latest: CAS-Spec Cascade Adaptive Self-Speculative Decoding for On-the-Fly Lossless Inference Acceleration of LLMs
  40. CH-40NPU2CH-40 / NPU / 2 posts Latest: HeteroLLM Accelerating Large Language Model Inference on Mobile SoCs with Heterogeneous AI Accelerators
  41. CH-41Survey2CH-41 / Survey / 2 posts Latest: A Survey of Resource-efficient LLM and Multimodal Foundation Models
  42. CH-42VLDB-252CH-42 / VLDB-25 / 2 posts Latest: Sonata Multi-Database Transactions Made Fast and Serializable
  43. CH-43vLLM2CH-43 / vLLM / 2 posts Latest: vLLM Hybrid KV Cache
  44. CH-446.8241CH-44 / 6.824 / 1 post Latest: Research: 6.824 Lab2B 中异常情况的分析
  45. CH-45ACLFindings-251CH-45 / ACLFindings-25 / 1 post Latest: Hierarchical Speculative Decoding with Dynamic Windows for Efficient Language Model Inference
  46. CH-46arXiv-211CH-46 / arXiv-21 / 1 post Latest: LoRA Low-Rank Adaptation of Large Language Models
  47. CH-47arXiv-25021CH-47 / arXiv-2502 / 1 post Latest: Entropy Adaptive Decoding Dynamic Model Switching for Efficient Inference
  48. CH-48arXiv-25041CH-48 / arXiv-2504 / 1 post Latest: Token Level Routing Inference System for Edge Devices
  49. CH-49arXiv-25071CH-49 / arXiv-2507 / 1 post Latest: R-Stitch Dynamic Trajectory Stitching for Efficient Reasoning
  50. CH-50ASPLOS-231CH-50 / ASPLOS-23 / 1 post Latest: STI Turbocharge NLP Inference at the Edge via Elastic Pipelining
  51. CH-51ASPLOS-241CH-51 / ASPLOS-24 / 1 post Latest: SpecInfer Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification
  52. CH-52ATC-251CH-52 / ATC-25 / 1 post Latest: KVCache Cache in the Wild Characterizing and Optimizing KVCache Cache at a Large Cloud Provider
  53. CH-53C++1CH-53 / C++ / 1 post Latest: C++ Notes
  54. CH-54Concurrency1CH-54 / Concurrency / 1 post Latest: A Piece of: ThreadLocal
  55. CH-55CUDA1CH-55 / CUDA / 1 post Latest: CUDA Hardware Hierarchy: From SMs to Lanes
  56. CH-56FaaS1CH-56 / FaaS / 1 post Latest: Watchdogs in OpenFaaS
  57. CH-57Golang1CH-57 / Golang / 1 post Latest: A Piece of: Golang Profile
  58. CH-58ICLR-241CH-58 / ICLR-24 / 1 post Latest: Efficient Streaming Language Models with Attention Sinks
  59. CH-59ICLR-251CH-59 / ICLR-25 / 1 post Latest: Swift On-the-fly Self-speculative Decoding For LLM Inference Acceleration
  60. CH-60ICML-24-Workshop1CH-60 / ICML-24-Workshop / 1 post Latest: Think Big, Generate Quick LLM-to-SLM for Fast Autoregressive Decoding
  61. CH-61IEEE-IoT-J-251CH-61 / IEEE-IoT-J-25 / 1 post Latest: EdgeShard Efficient LLM Inference via Collaborative Edge Computing
  62. CH-62ISCA241CH-62 / ISCA24 / 1 post Latest: Splitwise Efficient Generative LLM Inference Using Phase Splitting
  63. CH-63Java1CH-63 / Java / 1 post Latest: A Piece of: ThreadLocal
  64. CH-64LLM-Tuning1CH-64 / LLM-Tuning / 1 post Latest: LoRA Low-Rank Adaptation of Large Language Models
  65. CH-65Math1CH-65 / Math / 1 post Latest: Math for ML
  66. CH-66Memory-Management1CH-66 / Memory-Management / 1 post Latest: Efficient Memory Management for Large Language Model Serving with PagedAttention
  67. CH-67Misc.1CH-67 / Misc. / 1 post Latest: LLM Generated Content
  68. CH-68MLSys-251CH-68 / MLSys-25 / 1 post Latest: QServe W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
  69. CH-69Model-Architecture1CH-69 / Model-Architecture / 1 post Latest: Qwen3.5 Dense Model Architecture
  70. CH-70MoE1CH-70 / MoE / 1 post Latest: EdgeMoE Empowering Sparse Large Language Models on Mobile Devices
  71. CH-71NeurIPS-241CH-71 / NeurIPS-24 / 1 post Latest: Cascade Speculative Drafting for Even Faster LLM Inference
  72. CH-72NeurIPS-25-Workshop1CH-72 / NeurIPS-25-Workshop / 1 post Latest: 3-Model Speculative Decoding
  73. CH-73NIPS-231CH-73 / NIPS-23 / 1 post Latest: H2O Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models
  74. CH-74NIPS20221CH-74 / NIPS2022 / 1 post Latest: LLM.int8() 8-bit Matrix Multiplication for Transformers at Scale
  75. CH-75OSDI-221CH-75 / OSDI-22 / 1 post Latest: Orca A Distributed Serving System for Transformer-Based Generative Models
  76. CH-76PyTorch1CH-76 / PyTorch / 1 post Latest: PyTorch Notes
  77. CH-77Schedule1CH-77 / Schedule / 1 post Latest: Orca A Distributed Serving System for Transformer-Based Generative Models
  78. CH-78SEC-251CH-78 / SEC-25 / 1 post Latest: SLED A Speculative LLM Decoding Framework for Efficient Edge Serving
  79. CH-79Sequence-Parallelism1CH-79 / Sequence-Parallelism / 1 post Latest: Ring Attention with Blockwise Transformers for Near-Infinite Context
  80. CH-80Serverless1CH-80 / Serverless / 1 post Latest: ServerlessLLM Locality-Enhanced Serverless Inference for Large Language Models
  81. CH-81SOSP-241CH-81 / SOSP-24 / 1 post Latest: PowerInfer Fast Large Language Model Serving with a Consumer-grade GPU
  82. CH-82SOSP-251CH-82 / SOSP-25 / 1 post Latest: KTransformers: CPU/GPU Hybrid Inference for MoE Models
  83. CH-83Sparsity1CH-83 / Sparsity / 1 post Latest: Scaling Up On-Device LLMs via Active-Weight Swapping Between DRAM and Flash
  84. CH-84Theoretical1CH-84 / Theoretical / 1 post Latest: Deja Vu Contextual Sparsity for Efficient LLMs at Inference Time
  85. CH-85Triton1CH-85 / Triton / 1 post Latest: Triton Notes: Tutorials and Kernel Paradigms
  86. CH-86TSC-241CH-86 / TSC-24 / 1 post Latest: TPI-LLM Serving 70B-scale LLMs Efficiently on Low-resource Mobile Devices
Point at a channel to read it

CH-01LLM-Inference30↑

  1. 2026-11-19
    Decode GEMV on the CPU: Bandwidth, Compute or NUMA
    KKZ-TN-158
  2. 2026-11-04
    vLLM Hybrid KV Cache
    KKZ-TN-156
  3. 2026-11-02
    NVLink vs PCIe in Practice: Bandwidth and LLM Serving
    KKZ-TN-153
  4. 2026-10-29
    ggml Q4_0 vec_dot Walkthrough
    KKZ-TN-151
  5. 2026-10-24
    KTransformers: CPU/GPU Hybrid Inference for MoE Models
    KKZ-TN-150
  6. 2026-03-18
    ggml Memory Layout and Allocation
    KKZ-TN-141
  7. 2025-11-15
    LServe Efficient Long-sequence LLM Serving with Unified Sparse Attention
    KKZ-TN-112
  8. 2025-11-13
    DuoAttention Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
    KKZ-TN-109
  9. 2025-11-10
    EAGLE Speculative Sampling Requires Rethinking Feature Uncertainty
    KKZ-TN-106
  10. 2025-09-01
    Scaling Up On-Device LLMs via Active-Weight Swapping Between DRAM and Flash
    KKZ-TN-101
  11. 2025-08-27
    ELMS Elasticized Large Language Models On Mobile Devices
    KKZ-TN-100
  12. 2025-08-25
    STI Turbocharge NLP Inference at the Edge via Elastic Pipelining
    KKZ-TN-099
  13. 2025-08-24
    HeteroLLM Accelerating Large Language Model Inference on Mobile SoCs with Heterogeneous AI Accelerators
    KKZ-TN-097
  14. 2025-08-24
    EdgeMoE Empowering Sparse Large Language Models on Mobile Devices
    KKZ-TN-096
  15. 2025-08-17
    TPI-LLM Serving 70B-scale LLMs Efficiently on Low-resource Mobile Devices
    KKZ-TN-092
  16. 2025-08-04
    LLM Preliminaries
    KKZ-TN-084
  17. 2025-08-04
    Fast On-device LLM Inference with NPUs
    KKZ-TN-083
  18. 2025-08-04
    Deja Vu Contextual Sparsity for Efficient LLMs at Inference Time
    KKZ-TN-082
  19. 2025-07-30
    LLM in a flash Efficient Large Language Model Inference with Limited Memory
    KKZ-TN-079
  20. 2025-07-29
    PowerInfer-2 Fast Large Language Model Inference on a Smartphone
    KKZ-TN-078
  21. 2025-07-28
    PowerInfer Fast Large Language Model Serving with a Consumer-grade GPU
    KKZ-TN-077
  22. 2025-07-28
    AWQ Activation-aware Weight Quantization for LLM Compression and Acceleration
    KKZ-TN-075
  23. 2025-07-25
    SpecInfer Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification
    KKZ-TN-074
  24. 2025-07-25
    FlexGen High-Throughput Generative Inference of Large Language Models with a Single GPU
    KKZ-TN-072
  25. 2025-07-23
    Efficient Memory Management for Large Language Model Serving with PagedAttention
    KKZ-TN-071
  26. 2025-07-23
    EdgeLLM Fast On-Device LLM Inference With Speculative Decoding
    KKZ-TN-070
  27. 2025-07-20
    A Survey on Efficient Inference for Large Language Models
    KKZ-TN-069
  28. 2025-07-02
    Orca A Distributed Serving System for Transformer-Based Generative Models
    KKZ-TN-066
  29. 2025-07-01
    EdgeShard Efficient LLM Inference via Collaborative Edge Computing
    KKZ-TN-065
  30. 2025-06-28
    ServerlessLLM Locality-Enhanced Serverless Inference for Large Language Models
    KKZ-TN-064

CH-02Paper Note18↑

  1. 2024-04-15
    Data Management in Microservices: State of the Practice, Challenges, and Research Directions
    KKZ-TN-034
  2. 2024-03-25
    Paper Note: Taking Omid to the Clouds: Fast, Scalable Transactions for Real-Time Cloud Analytics
    KKZ-TN-032
  3. 2024-03-22
    Paper Note: Omid, Reloaded: Scalable and Highly-Available Transaction Processing
    KKZ-TN-031
  4. 2024-01-22
    Paper Note: Towards Transaction as a Service
    KKZ-TN-029
  5. 2023-12-09
    Paper Note: Scalable Distributed Transactions across Heterogeneous Stores
    KKZ-TN-028
  6. 2023-12-08
    Paper Note: GRIT: Consistent Distributed Transactions across Polyglot Microservices with Multiple Databases
    KKZ-TN-027
  7. 2023-12-05
    Paper Note: How to Read a Paper
    KKZ-TN-026
  8. 2023-11-30
    Paper Note: Ad Hoc Transactions in Web Applications: The Good, the Bad, and the Ugly
    KKZ-TN-024
  9. 2023-11-23
    Paper Note: Cobra: Making Transactional Key-Value Stores Verifiably Serializable
    KKZ-TN-021
  10. 2023-11-22
    Paper Note: Epoxy: ACID Transactions Across Diverse Data Stores
    KKZ-TN-020
  11. 2023-11-15
    Paper Note: Zab: High-performance broadcast for primary-backup systems
    KKZ-TN-019
  12. 2023-11-12
    Paper Note: CAP Twelve Years Later: How the "Rules" have Changed
    KKZ-TN-017
  13. 2023-11-06
    Paper Note: Chain Replication
    KKZ-TN-014
  14. 2023-11-04
    Paper Note: Time, clocks, and the ordering
    KKZ-TN-013
  15. 2023-10-31
    Paper Note: Chubby
    KKZ-TN-010
  16. 2023-10-30
    Paper Note: ZooKeeper
    KKZ-TN-009
  17. 2023-10-30
    Paper Note: MapReduce
    KKZ-TN-008
  18. 2023-10-30
    Paper Note: BigTable
    KKZ-TN-007

CH-03Reading Note18↑

  1. 2025-05-20
    Build a Large Language Model (From Scratch) Reading Note
    KKZ-TN-062
  2. 2025-03-31
    Tips and Guidance for Writing
    KKZ-TN-057
  3. 2025-03-19
    [100 Mistakes in Golang 08-09] Chapter 8 & 9
    KKZ-TN-053
  4. 2025-03-07
    [100 Mistakes in Golang 07] Chapter 7
    KKZ-TN-052
  5. 2025-03-04
    [100 Mistakes in Golang 06] Chapter 6
    KKZ-TN-051
  6. 2025-03-03
    [100 Mistakes in Golang 05] Chapter 5
    KKZ-TN-050
  7. 2025-02-20
    [100 Mistakes in Golang 03] Chapter 3
    KKZ-TN-049
  8. 2025-02-20
    [100 Mistakes in Golang 04] Chapter 4
    KKZ-TN-048
  9. 2025-02-20
    [100 Mistakes in Golang 02] Chapter 2
    KKZ-TN-047
  10. 2024-02-10
    Programming Rust Book Note
    KKZ-TN-030
  11. 2023-11-13
    [DDIA 09] Chapter 9 Consistency and Consensus
    KKZ-TN-018
  12. 2023-11-02
    [DDIA 08] Chapter 8 The Trouble with Distributed Systems
    KKZ-TN-011
  13. 2023-10-29
    [DDIA 07] Chapter 7 Transactions
    KKZ-TN-006
  14. 2023-10-24
    [DDIA 06] Chapter 6 Partitioning
    KKZ-TN-005
  15. 2023-10-23
    [DDIA 05] Chapter 5 Replication
    KKZ-TN-004
  16. 2023-10-21
    [DDIA 04] Chapter 4 Encoding and Evolution
    KKZ-TN-003
  17. 2023-10-20
    [DDIA 03] Chapter 3 Storage and Retrieval
    KKZ-TN-002
  18. 2023-10-20
    [DDIA 02] Chapter 2 Data Models and Query Languages
    KKZ-TN-001

CH-04Dev17↑

  1. 2026-03-18
    C++ Notes
    KKZ-TN-140
  2. 2025-10-04
    Sync Dot Files with Chezmoi
    KKZ-TN-102
  3. 2025-08-08
    Git Essentials
    KKZ-TN-087
  4. 2025-08-06
    VS Code Essentials
    KKZ-TN-085
  5. 2025-05-26
    NumPy Notes
    KKZ-TN-063
  6. 2025-05-19
    Useful Websites and Extensions
    KKZ-TN-061
  7. 2025-04-23
    Common Golang Mistakes
    KKZ-TN-060
  8. 2025-04-17
    CLI & Server Notes
    KKZ-TN-059
  9. 2025-04-04
    Jujutsu (jj) Tutorial
    KKZ-TN-058
  10. 2025-03-30
    Pass the Wall
    KKZ-TN-056
  11. 2025-03-29
    Docker Network Introduction
    KKZ-TN-055
  12. 2025-02-04
    Handling Golang Errors
    KKZ-TN-046
  13. 2025-01-28
    Golang Struct and Interfaces
    KKZ-TN-045
  14. 2024-11-26
    From Clicks to Commands: A Windows User's Survival Guide
    KKZ-TN-043
  15. 2024-11-21
    macOS Setup & Workflow
    KKZ-TN-042
  16. 2024-05-13
    A Piece of: Logs
    KKZ-TN-035
  17. 2023-11-25
    Note: WSL2 Mirrored 网络模式下异常情况总结
    KKZ-TN-022

CH-05Design Data-Intensive Applications8↑

  1. 2023-11-13
    [DDIA 09] Chapter 9 Consistency and Consensus
    KKZ-TN-018
  2. 2023-11-02
    [DDIA 08] Chapter 8 The Trouble with Distributed Systems
    KKZ-TN-011
  3. 2023-10-29
    [DDIA 07] Chapter 7 Transactions
    KKZ-TN-006
  4. 2023-10-24
    [DDIA 06] Chapter 6 Partitioning
    KKZ-TN-005
  5. 2023-10-23
    [DDIA 05] Chapter 5 Replication
    KKZ-TN-004
  6. 2023-10-21
    [DDIA 04] Chapter 4 Encoding and Evolution
    KKZ-TN-003
  7. 2023-10-20
    [DDIA 03] Chapter 3 Storage and Retrieval
    KKZ-TN-002
  8. 2023-10-20
    [DDIA 02] Chapter 2 Data Models and Query Languages
    KKZ-TN-001

CH-06Edge-Devices8↑

  1. 2025-11-11
    Dynamic Sparse Attention on Mobile SoCs
    KKZ-TN-108
  2. 2025-09-01
    Scaling Up On-Device LLMs via Active-Weight Swapping Between DRAM and Flash
    KKZ-TN-101
  3. 2025-08-27
    ELMS Elasticized Large Language Models On Mobile Devices
    KKZ-TN-100
  4. 2025-08-25
    SmallThinker A Family of Efficient Large Language Models Natively Trained for Local Deployment
    KKZ-TN-098
  5. 2025-08-04
    Fast On-device LLM Inference with NPUs
    KKZ-TN-083
  6. 2025-07-29
    PowerInfer-2 Fast Large Language Model Inference on a Smartphone
    KKZ-TN-078
  7. 2025-07-23
    EdgeLLM Fast On-Device LLM Inference With Speculative Decoding
    KKZ-TN-070
  8. 2025-07-01
    EdgeShard Efficient LLM Inference via Collaborative Edge Computing
    KKZ-TN-065

CH-07100 Mistakes in Golang7↑

  1. 2025-03-19
    [100 Mistakes in Golang 08-09] Chapter 8 & 9
    KKZ-TN-053
  2. 2025-03-07
    [100 Mistakes in Golang 07] Chapter 7
    KKZ-TN-052
  3. 2025-03-04
    [100 Mistakes in Golang 06] Chapter 6
    KKZ-TN-051
  4. 2025-03-03
    [100 Mistakes in Golang 05] Chapter 5
    KKZ-TN-050
  5. 2025-02-20
    [100 Mistakes in Golang 03] Chapter 3
    KKZ-TN-049
  6. 2025-02-20
    [100 Mistakes in Golang 04] Chapter 4
    KKZ-TN-048
  7. 2025-02-20
    [100 Mistakes in Golang 02] Chapter 2
    KKZ-TN-047

CH-08arXiv-247↑

  1. 2025-11-11
    A dynamic parallel method for performance optimization on hybrid CPUs
    KKZ-TN-107
  2. 2025-08-27
    ELMS Elasticized Large Language Models On Mobile Devices
    KKZ-TN-100
  3. 2025-08-21
    A Survey of Resource-efficient LLM and Multimodal Foundation Models
    KKZ-TN-094
  4. 2025-08-18
    LLM as a System Service on Mobile Devices
    KKZ-TN-093
  5. 2025-08-17
    Striped Attention Faster Ring Attention for Causal Transformers
    KKZ-TN-091
  6. 2025-07-29
    PowerInfer-2 Fast Large Language Model Inference on a Smartphone
    KKZ-TN-078
  7. 2025-07-20
    A Survey on Efficient Inference for Large Language Models
    KKZ-TN-069

CH-09arXiv-256↑

  1. 2026-02-02
    Estimating LLM Uncertainty with Evidence
    KKZ-TN-125
  2. 2025-11-15
    LServe Efficient Long-sequence LLM Serving with Unified Sparse Attention
    KKZ-TN-112
  3. 2025-11-11
    Dynamic Sparse Attention on Mobile SoCs
    KKZ-TN-108
  4. 2025-09-01
    Scaling Up On-Device LLMs via Active-Weight Swapping Between DRAM and Flash
    KKZ-TN-101
  5. 2025-08-25
    SmallThinker A Family of Efficient Large Language Models Natively Trained for Local Deployment
    KKZ-TN-098
  6. 2025-08-24
    HeteroLLM Accelerating Large Language Model Inference on Mobile SoCs with Heterogeneous AI Accelerators
    KKZ-TN-097

CH-10GPU6↑

  1. 2026-11-16
    CUDA Hardware Hierarchy: From SMs to Lanes
    KKZ-TN-157
  2. 2026-11-04
    Learning TileLang with GEMV
    KKZ-TN-154
  3. 2026-11-02
    NVLink vs PCIe in Practice: Bandwidth and LLM Serving
    KKZ-TN-153
  4. 2026-10-30
    GPU Server Topology: NVLink, PCIe and NUMA
    KKZ-TN-152
  5. 2026-01-07
    Triton Notes: Tutorials and Kernel Paradigms
    KKZ-TN-118
  6. 2025-08-24
    HeteroLLM Accelerating Large Language Model Inference on Mobile SoCs with Heterogeneous AI Accelerators
    KKZ-TN-097

CH-11KVCache6↑

  1. 2025-12-06
    KVCache Cache in the Wild Characterizing and Optimizing KVCache Cache at a Large Cloud Provider
    KKZ-TN-116
  2. 2025-11-15
    LServe Efficient Long-sequence LLM Serving with Unified Sparse Attention
    KKZ-TN-112
  3. 2025-11-13
    Quest Query-Aware Sparsity for Efficient Long-Context LLM Inference
    KKZ-TN-111
  4. 2025-11-13
    Efficient Streaming Language Models with Attention Sinks
    KKZ-TN-110
  5. 2025-11-13
    DuoAttention Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
    KKZ-TN-109
  6. 2025-08-21
    H2O Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models
    KKZ-TN-095

CH-12LLM-Serving5↑

  1. 2026-02-05
    Revati Transparent GPU-Free Time-Warp Emulation for LLM Serving
    KKZ-TN-128
  2. 2026-02-05
    AIConfigurator Lightning-Fast Configuration Optimization for Multi-Framework LLM Serving
    KKZ-TN-127
  3. 2025-11-03
    Splitwise Efficient Generative LLM Inference Using Phase Splitting
    KKZ-TN-105
  4. 2025-11-03
    DistServe Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
    KKZ-TN-104
  5. 2025-11-01
    Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
    KKZ-TN-103

CH-13ML Compilers5↑

  1. 2026-11-04
    TorchInductor Pipeline 与算子融合
    KKZ-TN-155
  2. 2026-10-20
    torch.compile Walkthrough
    KKZ-TN-149
  3. 2026-08-11
    ML Compilers Overview
    KKZ-TN-145
  4. 2026-08-10
    vLLM and torch.compile
    KKZ-TN-144
  5. 2026-08-09
    Piecewise CUDA Graph
    KKZ-TN-143

CH-14arXiv-234↑

  1. 2025-08-17
    Ring Attention with Blockwise Transformers for Near-Infinite Context
    KKZ-TN-090
  2. 2025-08-04
    Deja Vu Contextual Sparsity for Efficient LLMs at Inference Time
    KKZ-TN-082
  3. 2025-07-28
    AWQ Activation-aware Weight Quantization for LLM Compression and Acceleration
    KKZ-TN-075
  4. 2025-07-23
    Efficient Memory Management for Large Language Model Serving with PagedAttention
    KKZ-TN-071

CH-15Course4↑

  1. 2024-10-17
    [Missing Semester 05] Lecture 5
    KKZ-TN-040
  2. 2024-10-15
    [Missing Semester 04] Lecture 4
    KKZ-TN-039
  3. 2024-10-11
    [Missing Semester 02] Lecture 2
    KKZ-TN-038
  4. 2024-10-09
    [Missing Semester 01] Lecture 1
    KKZ-TN-037

CH-16Hardware4↑

  1. 2026-11-19
    Decode GEMV on the CPU: Bandwidth, Compute or NUMA
    KKZ-TN-158
  2. 2026-11-16
    CUDA Hardware Hierarchy: From SMs to Lanes
    KKZ-TN-157
  3. 2026-11-02
    NVLink vs PCIe in Practice: Bandwidth and LLM Serving
    KKZ-TN-153
  4. 2026-10-30
    GPU Server Topology: NVLink, PCIe and NUMA
    KKZ-TN-152

CH-17Quantization4↑

  1. 2025-11-16
    SmoothQuant Accurate and Efficient Post-Training Quantization for Large Language Models
    KKZ-TN-114
  2. 2025-11-16
    QServe W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
    KKZ-TN-113
  3. 2025-08-12
    LLM.int8() 8-bit Matrix Multiplication for Transformers at Scale
    KKZ-TN-088
  4. 2025-07-28
    AWQ Activation-aware Weight Quantization for LLM Compression and Acceleration
    KKZ-TN-075

CH-18sparse-attention4↑

  1. 2026-01-29
    XAttention Block Sparse Attention with Antidiagonal Scoring
    KKZ-TN-124
  2. 2026-01-29
    FlexPrefill A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference
    KKZ-TN-123
  3. 2025-11-15
    LServe Efficient Long-sequence LLM Serving with Unified Sparse Attention
    KKZ-TN-112
  4. 2025-11-13
    Quest Query-Aware Sparsity for Efficient Long-Context LLM Inference
    KKZ-TN-111

CH-19Talk4↑

  1. 2024-03-31
    Transaction Anomalies
    KKZ-TN-033
  2. 2023-12-04
    Talk about Postgres Visibility Check Rules
    KKZ-TN-025
  3. 2023-11-07
    Talk about Consistency and Consensus
    KKZ-TN-015
  4. 2023-11-02
    Talk about Redlock
    KKZ-TN-012

CH-20The Missing Semester4↑

  1. 2024-10-17
    [Missing Semester 05] Lecture 5
    KKZ-TN-040
  2. 2024-10-15
    [Missing Semester 04] Lecture 4
    KKZ-TN-039
  3. 2024-10-11
    [Missing Semester 02] Lecture 2
    KKZ-TN-038
  4. 2024-10-09
    [Missing Semester 01] Lecture 1
    KKZ-TN-037

CH-21torch.compile4↑

  1. 2026-11-04
    TorchInductor Pipeline 与算子融合
    KKZ-TN-155
  2. 2026-10-20
    torch.compile Walkthrough
    KKZ-TN-149
  3. 2026-08-11
    ML Compilers Overview
    KKZ-TN-145
  4. 2026-08-10
    vLLM and torch.compile
    KKZ-TN-144

CH-22ACL-243↑

  1. 2026-02-09
    LayerSkip Enabling Early Exit Inference and Self-Speculative Decoding
    KKZ-TN-134
  2. 2026-02-08
    Draft & Verify Lossless Large Language Model Acceleration via Self-Speculative Decoding
    KKZ-TN-131
  3. 2025-07-30
    LLM in a flash Efficient Large Language Model Inference with Limited Memory
    KKZ-TN-079

CH-23Commodity-GPU3↑

  1. 2025-07-30
    LLM in a flash Efficient Large Language Model Inference with Limited Memory
    KKZ-TN-079
  2. 2025-07-28
    PowerInfer Fast Large Language Model Serving with a Consumer-grade GPU
    KKZ-TN-077
  3. 2025-07-25
    FlexGen High-Throughput Generative Inference of Large Language Models with a Single GPU
    KKZ-TN-072

CH-24Distributed3↑

  1. 2023-11-08
    Research: 6.824 Lab2B 中异常情况的分析
    KKZ-TN-016
  2. 2023-11-07
    Talk about Consistency and Consensus
    KKZ-TN-015
  3. 2023-11-02
    Talk about Redlock
    KKZ-TN-012

CH-25ggml3↑

  1. 2026-11-19
    Decode GEMV on the CPU: Bandwidth, Compute or NUMA
    KKZ-TN-158
  2. 2026-10-29
    ggml Q4_0 vec_dot Walkthrough
    KKZ-TN-151
  3. 2026-03-18
    ggml Memory Layout and Allocation
    KKZ-TN-141

CH-26ICML-243↑

  1. 2025-11-13
    Quest Query-Aware Sparsity for Efficient Long-Context LLM Inference
    KKZ-TN-111
  2. 2025-11-10
    EAGLE Speculative Sampling Requires Rethinking Feature Uncertainty
    KKZ-TN-106
  3. 2025-08-17
    KV-Runahead Scalable Causal LLM Inference by Parallel Key-Value Cache Generation
    KKZ-TN-089

CH-27Long-Context3↑

  1. 2025-11-13
    Quest Query-Aware Sparsity for Efficient Long-Context LLM Inference
    KKZ-TN-111
  2. 2025-11-13
    Efficient Streaming Language Models with Attention Sinks
    KKZ-TN-110
  3. 2025-11-13
    DuoAttention Efficient Long-Context LLM Inference with Retrieval and Streaming Heads
    KKZ-TN-109

CH-28OSDI-243↑

  1. 2025-11-03
    DistServe Disaggregating Prefill and Decoding for Goodput-optimized Large Language Model Serving
    KKZ-TN-104
  2. 2025-11-01
    Taming Throughput-Latency Tradeoff in LLM Inference with Sarathi-Serve
    KKZ-TN-103
  3. 2025-06-28
    ServerlessLLM Locality-Enhanced Serverless Inference for Large Language Models
    KKZ-TN-064

CH-29Rust3↑

  1. 2025-03-26
    Using Rust
    KKZ-TN-054
  2. 2024-12-26
    Rust Exercises Notes
    KKZ-TN-044
  3. 2024-02-10
    Programming Rust Book Note
    KKZ-TN-030

CH-30Speculative-Decoding3↑

  1. 2026-02-10
    Cascade Speculative Drafting for Even Faster LLM Inference
    KKZ-TN-135
  2. 2025-07-25
    SpecInfer Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification
    KKZ-TN-074
  3. 2025-07-23
    EdgeLLM Fast On-Device LLM Inference With Speculative Decoding
    KKZ-TN-070

CH-31Tensor-Parallelism3↑

  1. 2025-08-17
    TPI-LLM Serving 70B-scale LLMs Efficiently on Low-resource Mobile Devices
    KKZ-TN-092
  2. 2025-08-17
    Striped Attention Faster Ring Attention for Causal Transformers
    KKZ-TN-091
  3. 2025-08-17
    KV-Runahead Scalable Causal LLM Inference by Parallel Key-Value Cache Generation
    KKZ-TN-089

CH-32TileLang3↑

  1. 2026-11-04
    Learning TileLang with GEMV
    KKZ-TN-154
  2. 2026-08-18
    TileLang Programming Model
    KKZ-TN-147
  3. 2026-08-17
    Production Ready TileLang Ops Framework
    KKZ-TN-146

CH-33arXiv-26012↑

  1. 2026-02-05
    Revati Transparent GPU-Free Time-Warp Emulation for LLM Serving
    KKZ-TN-128
  2. 2026-02-05
    AIConfigurator Lightning-Fast Configuration Optimization for Multi-Framework LLM Serving
    KKZ-TN-127

CH-34auto-tuning2↑

  1. 2026-02-05
    Revati Transparent GPU-Free Time-Warp Emulation for LLM Serving
    KKZ-TN-128
  2. 2026-02-05
    AIConfigurator Lightning-Fast Configuration Optimization for Multi-Framework LLM Serving
    KKZ-TN-127

CH-35Distributed-Transactions2↑

  1. 2025-07-13
    Sonata Multi-Database Transactions Made Fast and Serializable
    KKZ-TN-068
  2. 2025-07-11
    Concurrency Control as a Service
    KKZ-TN-067

CH-36ICML-232↑

  1. 2025-11-16
    SmoothQuant Accurate and Efficient Post-Training Quantization for Large Language Models
    KKZ-TN-114
  2. 2025-07-25
    FlexGen High-Throughput Generative Inference of Large Language Models with a Single GPU
    KKZ-TN-072

CH-37ICML-252↑

  1. 2026-01-29
    XAttention Block Sparse Attention with Antidiagonal Scoring
    KKZ-TN-124
  2. 2026-01-29
    FlexPrefill A Context-Aware Sparse Attention Mechanism for Efficient Long-Sequence Inference
    KKZ-TN-123

CH-38IEEE-TMC-252↑

  1. 2025-08-24
    EdgeMoE Empowering Sparse Large Language Models on Mobile Devices
    KKZ-TN-096
  2. 2025-07-23
    EdgeLLM Fast On-Device LLM Inference With Speculative Decoding
    KKZ-TN-070

CH-39NeurIPS-252↑

  1. 2026-02-07
    CAS-Spec Cascade Adaptive Self-Speculative Decoding for On-the-Fly Lossless Inference Acceleration of LLMs
    KKZ-TN-129
  2. 2026-01-07
    Beyond the 80 20 Rule High-Entropy Minority Tokens Drive Effective Reinforcement Learning for LLM Reasoning
    KKZ-TN-119

CH-40NPU2↑

  1. 2025-08-24
    HeteroLLM Accelerating Large Language Model Inference on Mobile SoCs with Heterogeneous AI Accelerators
    KKZ-TN-097
  2. 2025-08-04
    Fast On-device LLM Inference with NPUs
    KKZ-TN-083

CH-41Survey2↑

  1. 2025-08-21
    A Survey of Resource-efficient LLM and Multimodal Foundation Models
    KKZ-TN-094
  2. 2025-07-20
    A Survey on Efficient Inference for Large Language Models
    KKZ-TN-069

CH-42VLDB-252↑

  1. 2025-07-13
    Sonata Multi-Database Transactions Made Fast and Serializable
    KKZ-TN-068
  2. 2025-07-11
    Concurrency Control as a Service
    KKZ-TN-067

CH-43vLLM2↑

  1. 2026-11-04
    vLLM Hybrid KV Cache
    KKZ-TN-156
  2. 2026-08-10
    vLLM and torch.compile
    KKZ-TN-144

CH-446.8241↑

  1. 2023-11-08
    Research: 6.824 Lab2B 中异常情况的分析
    KKZ-TN-016

CH-45ACLFindings-251↑

  1. 2026-02-07
    Hierarchical Speculative Decoding with Dynamic Windows for Efficient Language Model Inference
    KKZ-TN-130

CH-46arXiv-211↑

  1. 2025-07-25
    LoRA Low-Rank Adaptation of Large Language Models
    KKZ-TN-073

CH-47arXiv-25021↑

  1. 2026-01-07
    Entropy Adaptive Decoding Dynamic Model Switching for Efficient Inference
    KKZ-TN-120

CH-48arXiv-25041↑

  1. 2025-12-01
    Token Level Routing Inference System for Edge Devices
    KKZ-TN-115

CH-49arXiv-25071↑

  1. 2026-02-02
    R-Stitch Dynamic Trajectory Stitching for Efficient Reasoning
    KKZ-TN-126

CH-50ASPLOS-231↑

  1. 2025-08-25
    STI Turbocharge NLP Inference at the Edge via Elastic Pipelining
    KKZ-TN-099

CH-51ASPLOS-241↑

  1. 2025-07-25
    SpecInfer Accelerating Large Language Model Serving with Tree-based Speculative Inference and Verification
    KKZ-TN-074

CH-52ATC-251↑

  1. 2025-12-06
    KVCache Cache in the Wild Characterizing and Optimizing KVCache Cache at a Large Cloud Provider
    KKZ-TN-116

CH-53C++1↑

  1. 2026-03-18
    C++ Notes
    KKZ-TN-140

CH-54Concurrency1↑

  1. 2023-11-29
    A Piece of: ThreadLocal
    KKZ-TN-023

CH-55CUDA1↑

  1. 2026-11-16
    CUDA Hardware Hierarchy: From SMs to Lanes
    KKZ-TN-157

CH-56FaaS1↑

  1. 2024-08-25
    Watchdogs in OpenFaaS
    KKZ-TN-036

CH-57Golang1↑

  1. 2024-10-23
    A Piece of: Golang Profile
    KKZ-TN-041

CH-58ICLR-241↑

  1. 2025-11-13
    Efficient Streaming Language Models with Attention Sinks
    KKZ-TN-110

CH-59ICLR-251↑

  1. 2026-02-08
    Swift On-the-fly Self-speculative Decoding For LLM Inference Acceleration
    KKZ-TN-132

CH-60ICML-24-Workshop1↑

  1. 2026-01-07
    Think Big, Generate Quick LLM-to-SLM for Fast Autoregressive Decoding
    KKZ-TN-121

CH-61IEEE-IoT-J-251↑

  1. 2025-07-01
    EdgeShard Efficient LLM Inference via Collaborative Edge Computing
    KKZ-TN-065

CH-62ISCA241↑

  1. 2025-11-03
    Splitwise Efficient Generative LLM Inference Using Phase Splitting
    KKZ-TN-105

CH-63Java1↑

  1. 2023-11-29
    A Piece of: ThreadLocal
    KKZ-TN-023

CH-64LLM-Tuning1↑

  1. 2025-07-25
    LoRA Low-Rank Adaptation of Large Language Models
    KKZ-TN-073

CH-65Math1↑

  1. 2026-03-18
    Math for ML
    KKZ-TN-142

CH-66Memory-Management1↑

  1. 2025-07-23
    Efficient Memory Management for Large Language Model Serving with PagedAttention
    KKZ-TN-071

CH-67Misc.1↑

  1. 2025-08-08
    LLM Generated Content
    KKZ-TN-086

CH-68MLSys-251↑

  1. 2025-11-16
    QServe W4A8KV4 Quantization and System Co-design for Efficient LLM Serving
    KKZ-TN-113

CH-69Model-Architecture1↑

  1. 2026-08-19
    Qwen3.5 Dense Model Architecture
    KKZ-TN-148

CH-70MoE1↑

  1. 2025-08-24
    EdgeMoE Empowering Sparse Large Language Models on Mobile Devices
    KKZ-TN-096

CH-71NeurIPS-241↑

  1. 2026-02-10
    Cascade Speculative Drafting for Even Faster LLM Inference
    KKZ-TN-135

CH-72NeurIPS-25-Workshop1↑

  1. 2026-02-09
    3-Model Speculative Decoding
    KKZ-TN-133

CH-73NIPS-231↑

  1. 2025-08-21
    H2O Heavy-Hitter Oracle for Efficient Generative Inference of Large Language Models
    KKZ-TN-095

CH-74NIPS20221↑

  1. 2025-08-12
    LLM.int8() 8-bit Matrix Multiplication for Transformers at Scale
    KKZ-TN-088

CH-75OSDI-221↑

  1. 2025-07-02
    Orca A Distributed Serving System for Transformer-Based Generative Models
    KKZ-TN-066

CH-76PyTorch1↑

  1. 2026-01-15
    PyTorch Notes
    KKZ-TN-122

CH-77Schedule1↑

  1. 2025-07-02
    Orca A Distributed Serving System for Transformer-Based Generative Models
    KKZ-TN-066

CH-78SEC-251↑

  1. 2025-12-07
    SLED A Speculative LLM Decoding Framework for Efficient Edge Serving
    KKZ-TN-117

CH-79Sequence-Parallelism1↑

  1. 2025-08-17
    Ring Attention with Blockwise Transformers for Near-Infinite Context
    KKZ-TN-090

CH-80Serverless1↑

  1. 2025-06-28
    ServerlessLLM Locality-Enhanced Serverless Inference for Large Language Models
    KKZ-TN-064

CH-81SOSP-241↑

  1. 2025-07-28
    PowerInfer Fast Large Language Model Serving with a Consumer-grade GPU
    KKZ-TN-077

CH-82SOSP-251↑

  1. 2026-10-24
    KTransformers: CPU/GPU Hybrid Inference for MoE Models
    KKZ-TN-150

CH-83Sparsity1↑

  1. 2025-09-01
    Scaling Up On-Device LLMs via Active-Weight Swapping Between DRAM and Flash
    KKZ-TN-101

CH-84Theoretical1↑

  1. 2025-08-04
    Deja Vu Contextual Sparsity for Efficient LLMs at Inference Time
    KKZ-TN-082

CH-85Triton1↑

  1. 2026-01-07
    Triton Notes: Tutorials and Kernel Paradigms
    KKZ-TN-118

CH-86TSC-241↑

  1. 2025-08-17
    TPI-LLM Serving 70B-scale LLMs Efficiently on Low-resource Mobile Devices
    KKZ-TN-092