KTransformers: CPU/GPU Hybrid Inference for MoE Models

How KTransformers (SOSP '25) runs the 671B DeepSeek-V3 on one GPU plus a dual-socket Xeon: AMX-aware CPU kernels, a single CUDA Graph for the whole decode step, NUMA-aware tensor parallelism, and Expert Deferral, which overlaps CPU and GPU by letting some experts arrive one layer late.

torch.compile Walkthrough

用 ((a + b) * c.relu()).sum(dim=-1) 跟踪 torch.compile 的每个阶段：Dynamo FX Graph、AOT ATen Graph、TorchInductor IR（TensorBox / StorageBox / Pointwise / Reduction / realize）、Scheduler 与生成的 kernel。

Qwen3.5 Dense Model Architecture

TileLang Programming Model

Production Ready TileLang Ops Framework

vLLM and torch.compile

Piecewise CUDA Graph

Probability and Statistics

Matrix Math

ggml with C++