Series · 5 parts

PyTorch

Building the framework up from its one real primitive — the tensor — through autograd and the training loop.

Engineering

Sep 08, 2026 – Sep 11, 2026 · 29 min to read the lot · 0 views

Start with part oneStart with part one See them in the archive

The parts, in order

EngineeringSep 08, 2026 · 8 min read

PyTorch: From Tensor to Neural Network

People think PyTorch is enormous. It isn't. It's one data structure and a pile of operations on it — and once that clicks, every model you'll ever read looks obvious. Here's the ground floor.

0018 min read

EngineeringSep 09, 2026 · 6 min read

Autograd: How PyTorch Learns

Every training loop on Earth is basically the same four lines. The one that does the real work — loss.backward() — is a graph PyTorch builds while your code runs and then walks backwards. Let's take it apart.

0016 min read

EngineeringSep 10, 2026 · 5 min read

nn.Module: Building Networks That Hold Together

nn.Sequential is training wheels. Real models are custom Modules — with skip connections, shared weights, and branches. Here's how nn.Module actually works, and every footgun it hides.

0005 min read

EngineeringSep 11, 2026 · 5 min read

Datasets and DataLoaders: Keeping the GPU Fed

You bought a fast GPU and it's sitting at 30% utilisation. The model isn't the bottleneck — the input pipeline is. Here's how Dataset and DataLoader work, and how to stop starving the accelerator.

0005 min read

EngineeringSep 11, 2026 · 5 min read

Training for Real: GPUs, Mixed Precision, and Schedules

The five-line loop from Part 2 trains a toy. A real run adds a device, mixed precision for a near-free 2×, a learning-rate schedule, gradient clipping, and checkpoints you can actually resume from. Here's the whole thing.

0005 min read