NVLink vs PCIe in Practice: Bandwidth and LLM Serving

Peer-to-peer and NCCL bandwidth measured on an RTX A6000 and an A100 server, and what NVLink and PCIe change when vLLM serves Qwen3.8-27B with tensor parallelism.

GPU Server Topology: NVLink, PCIe and NUMA

What NV#, PIX, PXB, PHB, NODE and SYS in nvidia-smi topo -m mean on the board of a two-socket GPU server, with two real machines.

ggml Q4_0 vec_dot Walkthrough

How ggml's CPU backend computes a Q4_0 matrix-vector product during decode, split across threads, with and without repacking.

KTransformers: CPU/GPU Hybrid Inference for MoE Models

How KTransformers (SOSP '25) runs the 671B DeepSeek-V3 on one GPU plus a dual-socket Xeon: AMX-aware CPU kernels, a single CUDA Graph for the whole decode step, NUMA-aware tensor parallelism, and Expert Deferral, which overlaps CPU and GPU by letting some experts arrive one layer late.

torch.compile Walkthrough

用 ((a + b) * c.relu()).sum(dim=-1) 跟踪 torch.compile 的每个阶段：Dynamo FX Graph、AOT ATen Graph、TorchInductor IR（TensorBox / StorageBox / Pointwise / Reduction / realize）、Scheduler 与生成的 kernel。

Qwen3.5 Dense Model Architecture

TileLang Programming Model

Production Ready TileLang Ops Framework

vLLM and torch.compile

Piecewise CUDA Graph