AI Infra Handbooks
From Python, LLM internals and CUDA to inference systems: one path to an LLM inference role
Fourteen interlinked handbooks, 327 chapters, with self-tests, runnable code, exercises and answers. The English translation is in progress; pages not translated yet show the Chinese original.
See the roadmap: 327 chapters over 17 weeks, with the key chapters for each kind of role →
The library
Fourteen handbooks, plus interview questions, exercises, a learning roadmap and a job-hunting sprint plan
Advanced Python
For people who already write Python: from code that works to code that is idiomatic and solid
Advanced C++
Modern C++ for AI infrastructure: read inference libraries and write correct, efficient systems code that survives the sanitizers
CS Fundamentals
The operating systems, architecture, networking, distributed systems and algorithms an inference engineer needs, each chapter starting from something real in an inference system
Math Fundamentals
The math behind models and inference systems, in its own book to look up when you need it; every result is measured on a real Qwen3-0.6B
LLM Internals
The groundwork for inference optimization: from language models and the Transformer all the way to inference serving
PyTorch in a Hurry
Eight chapters to get fluent with PyTorch: tensors and shapes, autograd, models and the training loop, debugging and speed
Advanced CUDA
From GPU architecture to hand-written high-performance kernels, for AI infrastructure and inference optimization roles
Train a Small Model
Start from an empty directory on your own computer and train a small language model that writes in the style of a classical novel
Distributed Training
The training-side parallelism inference engineers need too: starting from the memory budget, implement DDP, ZeRO, and tensor, pipeline, context and expert parallelism from scratch, each matched against a single GPU
Inference Systems
Build an inference engine from scratch, read vLLM and SGLang, and master distributed inference and performance engineering
Transformer by Hand
Start from one main.py and write Qwen3.5's hybrid architecture one file per chapter, matching Hugging Face at every step
mini-sglang from Scratch
Start from a 126-line minimal engine and build a complete inference engine one file per chapter in the official module layout; every step runs, and the output matches HF token by token
Image & Video Generation
Inference and serving for diffusion and flow-matching models: pipeline anatomy, compute and memory budgets, few-step sampling, caching, quantization, multi-GPU parallelism, video inference and scheduling for generation services
SGLang Design Evolution
More than 19,000 commits as primary sources: the problem behind each design, the commit that introduced it, what it replaced and how it evolved, from the paper and the first commit to today
Reading SGLang-Omni
The multi-stage inference runtime for omni, TTS and ASR models: the Coordinator, stages and schedulers, how SGLang is embedded, how data moves between stages; three case studies, then finding problems and submitting a PR
Inference interview questions
Answer outlines for 85 frequent questions, coding questions, full reference answers to 10 system design questions and 4 mock interviews; the LLM and CUDA books have their own question banks
Exercises (Chinese)
292 coding exercises for the fifteen books: write code and run the tests in your browser; CUDA exercises run on a GPU emulator that checks coalescing and bank conflicts, and everything can be judged locally on macOS or WSL2
Playground
Write Python in the browser and run it right away, no exercise needed: numpy, matplotlib, plus the CUDA and Triton emulators; with code completion and shareable links
Flashcards
2082 flashcards drawn from the self-tests, exercises and interview banks of the fifteen books, scheduled by spaced repetition; filter by book or export to Anki
Learning roadmap
The 327 chapters of the fifteen books laid out over 17 weeks, matching the sprint plan week by week: core, recommended and optional chapters at a glance, key chapters for framework, optimization and platform roles, and progress tracking
17-week sprint plan
A job-hunting plan for inference systems roles: the skill model, what the three kinds of roles emphasize, weekly study and exercises, frontier topics, portfolio and open-source work, system design and the interview schedule
Where to start
Each stop prepares the next, and the books link to each other; the interview questions and a portfolio wrap it up
Advanced Python
Know the language well: data model, typing, testing, concurrency and profiling
LLM Internals
Understand what the model computes, and where each inference optimization applies
Train a Small Model
Walk through the corpus, the tokenizer, the model and the training loop by hand on your own computer
Advanced CUDA
Make the GPU fast: the memory hierarchy, classic kernels, FlashAttention
Advanced C++
RAII, memory pools, memory order, lock-free queues: read and write the C++ in inference libraries
Inference Systems
Build an inference engine from scratch, read vLLM / SGLang, distributed inference and performance engineering
Distributed Training
DDP, ZeRO, TP / PP / CP / EP and RL training systems: each kind of training parallelism has an inference counterpart
mini-sglang from Scratch
Put every concept into one complete codebase and turn it into a portfolio project
CS Fundamentals
Operating systems, architecture, networking and algorithms: fill in whatever a stop needs, and review before interviews
Math Fundamentals
Linear algebra, probability, information theory, floating point and performance math: when a derivation stops you, look up that section
Chapter-by-chapter roadmap → · Exercises for every chapter → · 17-week sprint plan → · Learning on a Mac: setup →