Tag
Everything tagged #inference
3 articles, newest first.
InferenceFeb 12, 2025 · 3 min read
CAG Over RAG, When Speed Is the Constraint
RAG searches for information on every turn. CAG pre-computes the KV cache. Where Time-to-First-Token and conversational latency matter, caching flips the architecture.
003 min read
InferenceJan 30, 2025 · 3 min read
LPUs and What They Do That GPUs Do Not
Language Processing Units are purpose-built for language models rather than general compute. The specialization shows up in where the weights live.
103 min read
InferenceDec 10, 2024 · 2 min read
Deploying AI Models with NVIDIA NIM: Production LLMs as Microservices
Running self-hosted LLMs used to mean stitching vLLM, Triton, and CUDA drivers by hand. NVIDIA NIM packages TensorRT-LLM in standardized, production-ready OCI containers.
002 min read
