Inference systems roles a 17-week sprint plan
The target is an inference systems / AI infrastructure role at a leading LLM company. Four phases: first build solid foundations in C++, CUDA, LLMs and inference engines, then go deep into framework source code and distributed inference, then master frontier inference systems and build distinctive projects, and finally concentrate on interviews. Every week maps to handbook chapters, exercises and checkable deliverables.
The goal: "hireable" in an interviewer's eyes
Interviews for inference systems roles usually revolve around six abilities. Each lists what you should be able to do when you meet the bar: not "have read about it", but write it on a whiteboard, explain it with numbers and answer follow-up questions.
Four levels of learning goals, each one checkable
"I studied it" does not count. The goal for every topic is written as one of four levels: explain, write it right, make it fast, deliver; every item on the weekly checklists is labeled with its level and maps to a verification method below. The later the phase, the larger the share of L3 and L4.
Mock interview scoring: correctness (any mistakes), depth (how many layers of follow-up you can take), numbers (can you give orders of magnitude and estimates), communication (conclusion first, then details, clearly structured), each 1–4, and a pass means at least 3 on all four. The denominator of the performance bars is the bandwidth and compute you measure on your own machine in week 2, not the peak on the spec sheet.
What three kinds of inference roles emphasize
Inference systems roles fall roughly into three kinds, with different day-to-day work and interview emphasis. The plan follows the frameworks track as its main line while covering the core of kernel optimization and inference platforms; when you apply, adjust the focus of the last weeks to the job description.
All three kinds of roles test coding, fundamentals and system design; the job posting is the final word on requirements.
Standing out: a main line plus three amplifiers
The main line is the basics every candidate must clear. On top of that, amplify systems engineering skill into something distinctive: inference systems are full of "systems" problems (scheduling, storage, networking, runtimes, observability), which is exactly where many algorithm- or kernel-focused candidates are weak.
Week by week
Designed for about 30 hours a week. Each week has main study, exercises, algorithm problems, deliverables and a checklist; tick off the whole checklist before moving to the next week. Skip what you already know after the chapter self-tests and exercises, and give the time saved to projects and algorithms.
Four milestones
Each milestone is a mock hiring review: check yourself on time, and if you are not there, stop and fill the gaps. Do not carry holes into the next phase, because interview follow-ups always find them.
Projects and open-source contributions
The most convincing things on a résumé are "built, measured, merged". The three projects below complement each other: one proves you understand the whole inference engine, one shows distinctive systems engineering, and one proves you can work down at the GPU level. Open-source PRs let interviewers read your code directly.
Courses and tutorials used
Public AI systems courses and tutorials were compared, and the strongest part of each is built into the weekly plan (links marked external in the weeks). Their verification methods in particular (implement from scratch with built-in tests, real-GPU leaderboards, estimation exercises with answers) are part of this plan's checks.
These materials filled in for parts of this site that were not written yet (distributed training, C++ and so on); for external materials, the authors' latest versions are authoritative.
Interview prep
Interviews usually have four parts: coding (algorithms + hand-written components), fundamentals with follow-ups, system design, and deep dives into your projects. Practice each part separately, and do mock interviews.
Algorithms: about 200 problems, week by week
Finish LeetCode Hot 100 first, then the topics below; 25 minutes per problem, and if you cannot solve it, read the solution and redo it 3 days later.
Hand-written components: what interviews ask you to write on the spot
Each must be written in 20–40 minutes as a runnable version with correct edge cases (most have matching exercises).
System design: 10 must-practice questions
Answer in the structure "requirements and scale → capacity estimates → architecture → critical paths (scheduling / KV / communication) → failures and scaling → metrics", with numbers for every question.
Papers and technical reports: a close-reading list
Reading closely means you can answer "why is it designed this way, what does it cost, what are the numbers, what if it were X instead". Starred items are the work interviews ask about most, and must be read in detail.
Applications and interview rhythm
Résumé and presenting projects
Weekly rhythm
About 30 hours a week. With more time, scale up "main study" and "projects" proportionally, and do not cut the number of algorithm problems.
Supporting content
The content-building list that filled the gaps in the skill model above, by priority, now all done: fifteen handbooks, 292 exercises and four assignments, covering Python, C++, LLM internals, CUDA and PyTorch internals, distributed training, inference engines (including communication and storage, two frontier topics, production and the ecosystem) and mini-sglang.