Opening the archive
Everything published here so far. Filter by topic, or sort by what gets read most.
How Apple built predictive code completion directly into Xcode 16 using the Apple Neural Engine. Zero cloud roundtrips, offline privacy, and Swift specialization.
RAG searches for information on every turn. CAG pre-computes the KV cache. Where Time-to-First-Token and conversational latency matter, caching flips the architecture.
Tokens, vectors, attention, and next-token prediction. What is actually happening underneath something that feels like a conversation.
Language Processing Units are purpose-built for language models rather than general compute. The specialization shows up in where the weights live.
ChatGPT did not cause the wildfires. The conversation that formed around the claim is still worth having.
There are not enough GPUs to go around, so the company behind ChatGPT is looking at making the hardware itself.
746 views
728 views
684 views
356 views
