LLaVA connects CLIP visual features to a language model with a lightweight projector. This article explains visual tokens, two-stage training, the data contract, compute costs, and failure boundaries.
LoRA freezes pretrained weights and learns a low-rank update. This article explains the matrix factorization, the real training ledger, adapter merging, QLoRA, and the limits of the low-rank assumption.
Speculative decoding lets a cheap model draft several tokens and a large model verify them in one pass. This article explains the serial bottleneck, rejection-sampling correction, and the real conditions for a speedup.
PagedAttention does not necessarily make the attention kernel faster, yet it enables much higher serving throughput in vLLM. This article explains how paging, block tables, on-demand allocation, and copy-on-write turn KV-cache memory into a first-class systems problem.
DPO replaces a separate reward model and PPO loop with a pairwise classification loss. The preference signal has not vanished: it is encoded in how the policy's probabilities move relative to a reference model. This article derives that change of variables and examines its limits.
