You ask ChatGPT to write a poem and it delivers. You ask Claude to summarize research and it gives you coherent analysis. It feels like conversing with a thinking entity.
Here is what actually happens. Your prompt, "How do LLMs think?", fragments into tokens: "How", "do", "LL", "Ms", "think", "?". Each token becomes a high-dimensional vector, a list of numbers like 0.12, -0.45, 0.88. That vector is the token's position in a vast semantic space where meaning becomes geometry, and similar words cluster together mathematically.
Attention
Those vectors enter the Transformer architecture, whose superpower is self-attention. Take the sentence "The robot picked up the red ball because it was heavy." To work out what "it" refers to, the attention mechanism calculates relevance scores between every token. "Ball" scores high. "Robot" matters. "Red" is peripheral. This is what lets the model grasp long-range dependencies across an entire sequence.
Prediction, not composition
The model does not compose a response. It predicts probabilities. For each position it calculates odds across its whole vocabulary, so "They" might be 28 percent, "Large" 22 percent, "These" 15 percent. It selects a high-probability token, adds it to the sequence, and repeats the entire process. Token by token, the response is built out of pure probability.
So how do they solve complex problems if they are only predicting words? Two reasons. Scale matters: training on massive datasets teaches models patterns complex enough to mirror logic and reasoning, producing emergent abilities that only appear above a certain size. And chain-of-thought prompting works because asking a model to think step by step forces it to generate intermediate reasoning tokens, which creates structured pathways for prediction that mimic logical thought.
LLMs possess no consciousness and no genuine understanding. They are highly optimized probabilistic machines that learned language patterns so well that their output mimics thinking. Understanding that mechanism, next-token prediction guided by patterns, is essential to using them effectively and responsibly. It turns the mystery into something sophisticated but fundamentally bounded.
