Recent Articles
Production LLM Systems Tutorial 4: RAG and Data Pipelines
A practical tutorial for building a production RAG pipeline with ingestion, chunking, embeddings, hybrid search, reranking, metadata filters, and index versioning.
Production LLM Systems Tutorial 5: Monitoring and Observability
A tutorial for monitoring LLM applications across system metrics, quality signals, drift, tracing, privacy, and cost attribution.
Production LLM Systems Tutorial 6: Evaluation and A/B Testing
A tutorial for building offline evals, online experiments, regression gates, judge calibration, RAGAS-style metrics, and release workflows for LLM systems.
Production LLM Systems Tutorial 7: Security and Prompt Injection
A practical tutorial for defending LLM systems against direct and indirect prompt injection, data exfiltration, unsafe tool calls, and privilege escalation.
Production LLM Systems Tutorial 8: Human-in-the-Loop Workflows
A tutorial for designing human-in-the-loop LLM workflows with confidence routing, escalation queues, review UX, active learning, and approval gates.
Production LLM Systems Tutorial 9: Cost Optimization
A tutorial on reducing LLM application cost with routing, caching, prompt budgeting, batch processing, quantization, attribution, and token guardrails.
Production LLM Systems Tutorial 10: Versioning and Disaster Recovery
A tutorial for versioning models, prompts, embeddings, retrieval indexes, tools, and policies while designing fallback, rollback, and graceful degradation for LLM systems.
Agents Need Seatbelts: Guardrails and Infinite-Loop Detection for Tool-Using AI
A systems guide to agent guardrails, tool authorization, loop detection, budgets, recursion limits, no-progress checks, and production safety controls.
Your Token Bill Has a Leak: Cost Monitoring for Hidden LLM Waste
A production guide to hidden token leaks: retries, tool loops, unused RAG context, schema bloat, cached-token accounting, reasoning tokens, and cost observability.
Reduce LLM Inference Cost by 60% Without Serving Stale Answers
A production architecture for semantic response caching, freshness gates, prompt/KV reuse, and cache-aware routing when users ask the same questions in different words.