Recent Articles
From Prefill to Decode: Disaggregated Inference as a Distributed Systems Problem
Why splitting prefill and decode can improve LLM serving, and why the real challenge is KV transfer, topology, scheduling, and SLO-aware operation.
The Rust Case for AI Gateways: Backpressure, Streaming, and Failure Isolation
Why an AI gateway sits on a systems boundary where Rust's ownership, async, cancellation, and no-GC profile become practical advantages.
Why Round-Robin Dies in LLM Serving: KV-Aware Routing Explained
A deep but practical explanation of why LLM routing must account for KV cache overlap, prefill cost, decode load, and SLO risk instead of simply rotating requests across workers.
What AI-Native Talent Looks Like in 2026: A Recruiter's Field Guide
A practical guide for talent leaders on identifying AI-native candidates: workflow literacy, judgment, AI-assisted execution, and human skills that still matter.
TensorRT-LLM vs vLLM vs SGLang: Choosing an Inference Engine for Production
A practical comparison of TensorRT-LLM, vLLM, and SGLang across performance, portability, structured generation, cache reuse, deployment, and operations.
Dynamo Is Not an Inference Engine. It Is the Control Plane for Tokens
Why NVIDIA Dynamo is best understood as the distributed control plane around LLM inference engines, not as another engine competing with vLLM, SGLang, or TensorRT-LLM.
The AI Hiring Playbook for 2026: Skills, Signals, and Fewer Shiny Job Titles
A practical talent-acquisition playbook for hiring in the AI era: skills-first scorecards, better work samples, recruiter judgment, and fewer inflated AI job titles.
Inference Is Not HTTP: The Case for a Purpose-Built Gateway in Rust
A systems case for purpose-built inference gateways: token-aware routing, cache affinity, cancellation-safe streaming, Rust, and GPU-aware operations.
KV-Aware Routing: How Cache Locality Changes Load Balancing for LLMs
Why LLM load balancing needs cache locality, prefix reuse, token cost, and runtime state instead of simple round-robin.
Tokenomics for Engineers: Measuring Throughput per Dollar Instead of Tokens per Second
A practical framework for measuring inference economics: tokens per second, TTFT, TPOT, utilization, quality, energy, and cost per useful token.