Recent Articles
Why Agentic AI Is Bringing CPUs Back Into the Spotlight
Why agentic AI makes CPUs more important again, and why modern CPU+GPU platforms are starting to look like the natural shape of the AI cloud.
YC's 2026 Startup Map: AI Has Left the Chatbox
A talent and market-trends read on Y Combinator's latest Requests for Startups: AI-native services, company brains, agent software, chips, hardware, medicine, agriculture, space, and enterprise workflows.
Your RAG Demo Passed. Your RAG System Needs a Judge: RAGAS, Humans, and Evidence
A deep guide to evaluating RAG systems with RAGAS, retrieval metrics, faithfulness checks, human evals, golden datasets, and production feedback loops.
Agentic AI Needs Smarter Inference: Hints, Priority, and Cache Lifecycle
Why agentic workloads need inference runtimes that understand priority, expected output length, speculative prefill, and KV cache lifecycle instead of treating every request the same.
Draft Tokens or Smaller Numbers? Speculative Decoding vs Quantization in Production
A practical comparison of speculative decoding and quantization: what each optimizes, where each fails, how they interact, and what to measure before rollout.
KV Cache at Fleet Scale: The Memory System Hiding Inside Every LLM Platform
A deep systems guide to KV cache capacity, PagedAttention, prefix reuse, eviction, offload, routing, and fleet-level cache management for LLM inference.
The Cache Has Layers: Prompt Caching, Semantic Caching, and When Each One Betrays You
A production guide to prompt caching, context caching, semantic response caching, exact caching, and the tradeoffs that decide latency, cost, freshness, and correctness.
Autoscaling LLMs by TTFT and TPOT, Not CPU Utilization
Why LLM serving needs autoscaling based on first-token and per-token latency, and how Dynamo Planner points toward SLO-aware capacity control.
From H100 to Blackwell: What Actually Changes for Inference Architects
A practical architecture view of the shift from H100/H200 to Blackwell: memory, precision, NVLink scale-up, MoE, software, and cost per token.
Speculative Decoding in Production: When Draft Tokens Help and When They Hurt
A practical guide to speculative decoding: why it speeds up autoregressive generation, how to measure acceptance rate, and when it creates operational complexity.