Notes on research, systems and engineering.
- Latest doc
- KKZ-TN-158
- Entries
- 158
- Latest entry
- 2026-11-19
- Activity / 12 mo
- /1582026-11-19
Decode GEMV on the CPU: Bandwidth, Compute or NUMA
Decode GEMV on a Core Ultra 9, an M4 Pro and a dual-socket Xeon: how close each gets to its memory bandwidth, and what holds it back.
- /1572026-11-16
CUDA Hardware Hierarchy: From SMs to Lanes
Where the threads of one CUDA launch run on an RTX A6000, where their data lives, and why some memory access patterns are slow.
- /1562026-11-04
vLLM Hybrid KV Cache
Why vLLM's "GPU KV cache size" for Qwen3.8-27B is not a count of token slots and changes with max_model_len: GDN state, attention pages and one shared block pool.
- /1552026-11-04
TorchInductor Pipeline 与算子融合
跟着一个 10 个算子的例子走完 TorchInductor 的四个阶段:Graph Preprocessing、Lowering、Scheduling、Codegen,重点是 Scheduler 怎么用 can_fuse、score_fusion 和 fuse_nodes 决定融合,并与 TVM 的 FuseOps 对比。
- /1542026-11-04
Learning TileLang with GEMV
用 GEMV 入门 TileLang:同一个 FP16 GEMV 写十四遍,依次用到 T.Kernel、线程绑定、shared / local / fragment 三种分配、T.copy、T.vectorized、T.Pipelined 和 reducer,以及 warp、合并访存、128-bit 加载这些与 CPU 不同的 GPU 访存特性。
- /1532026-11-02
NVLink vs PCIe in Practice: Bandwidth and LLM Serving
Peer-to-peer and NCCL bandwidth measured on an RTX A6000 and an A100 server, and what NVLink and PCIe change when vLLM serves Qwen3.8-27B with tensor parallelism.
- /1522026-10-30
GPU Server Topology: NVLink, PCIe and NUMA
What NV#, PIX, PXB, PHB, NODE and SYS in nvidia-smi topo -m mean on the board of a two-socket GPU server, with two real machines.
- /1512026-10-29
ggml Q4_0 vec_dot Walkthrough
How ggml's CPU backend computes a Q4_0 matrix-vector product during decode, split across threads, with and without repacking.
- /1502026-10-24
KTransformers: CPU/GPU Hybrid Inference for MoE Models
How KTransformers (SOSP '25) runs the 671B DeepSeek-V3 on one GPU and a dual-socket Xeon, and why the CPU side is the hard part.
- /1492026-10-20
torch.compile Walkthrough
用 ((a + b) * c.relu()).sum(dim=-1) 跟踪 torch.compile 的每个阶段:Dynamo FX Graph、AOT ATen Graph、TorchInductor IR(TensorBox / StorageBox / Pointwise / Reduction / realize)、Scheduler 与生成的 kernel。