LLM Tutorials: How Language Models Run

01

Published

Inside an LLM · part 1 of 5

How an LLM generates one token

One token from input to output: tokenization, embeddings, twenty-four transformer layers, the LM head, temperature and sampling. CPU work is blue, GPU work is green, and two 3-D benches show the tensor moving through the pipeline. Part one of Inside an LLM, a five-part series built on one reference model, Qwen2.5-0.5B, so every number lines up from the first page to the last.

Read part 1

Inside an LLM · part 2 of 5

Making the sentence stand up

Type a sentence and watch it stand up into a tensor: one row per token, through the ids, the embedding lookup and all 24 layers, until only the last row is left.

Read part 2

02

Coming soon

Parts 3–5 of Inside an LLM are drafted and under technical review.

part 3The decode phase Prefill versus decode, every tensor shape inside one layer, and what a step costs in FLOPs and bandwidth.
part 4Attention in detail Q, K and V, 14 query heads against 2 KV heads, RoPE, and what the cache holds byte by byte.
part 5The LM head, magnified 151,936 dot products, what a logit is, temperature as a division, top-k, top-p and the draw.

03

Planned for the future

plannedLLM inference optimization Batching, paged attention, speculative decoding, and where the throughput actually comes from.
plannedQuantization What int8, int4 and fp8 do to weights, activations and the cache, and how to measure the accuracy you lose.
plannedvLLM internals The scheduler, the block manager, and the path a request takes through a real serving stack.