Recent Articles
7/20 - Parallel Decoding: Predicting More Than One Future at a Time
A clear guide to multi-token heads, candidate trees, Medusa-style verification, and the difference between parallel and speculative decoding.
6/20 - Early Exit Decoding: Stop Computing Once the Answer Is Clear
How early-exit language models use intermediate layers, confidence, and full-model verification to reduce inference work without hiding quality risk.
5/20 - Batch Inference: When Throughput Matters More Than Immediacy
A practical guide to offline LLM batch inference, job design, bucketing, retries, idempotency, cost, and throughput-oriented scheduling.
4/20 - PagedAttention: Virtual Memory for the KV Cache
How PagedAttention applies block-based virtual-memory ideas to dynamic KV caches, improving utilization, sharing, and serving throughput.
3/20 - FlashAttention: Why Moving Fewer Bytes Beats Doing Fewer FLOPs
An intuitive and rigorous explanation of FlashAttention, tiling, online softmax, IO awareness, exactness, and practical deployment limits.
2/20 - Speculative Decoding: Let a Small Model Guess, Let a Large Model Judge
How speculative decoding accelerates autoregressive generation while preserving the target model distribution, with acceptance math and production tradeoffs.
1/20 - KV Caching: The Memory That Makes Token Generation Possible
A beginner-friendly and technically deep guide to KV caching, memory sizing, prefix reuse, eviction, isolation, and production metrics.
Production LLM Systems Tutorial 1: End-to-End Application Design
A practical tutorial for designing an end-to-end LLM application with gateway, orchestration, retrieval, tools, inference, caching, telemetry, and failure handling.
Production LLM Systems Tutorial 2: Latency, Cost, and Quality
A practical tutorial on the latency, cost, and quality trade-offs behind model routing, caching, batching, quantization, speculative decoding, and prompt compression.
Production LLM Systems Tutorial 3: Scalable Inference Architecture
A tutorial on scalable LLM inference with vLLM, TensorRT-LLM, SGLang, KV cache management, parallelism, autoscaling, routing, and multi-tenant serving.