Behind every trained model, the attention layers, the giant language models, the small on-device ones, is a single training algorithm: gradient descent. It sounds like calculus you'd rather forget, but the idea is a ball rolling downhill, and the code is about three lines. Build it once and "training a model" stops being mysterious.
The one idea: follow the slope downhill
Suppose you have a function that measures how wrong your model is, the loss. Training means finding the inputs that make the loss as small as possible: the bottom of the valley. You can't see the whole landscape, but at any point you can feel the slope under your feet. So you take a small step in the downhill direction, and repeat. Roll downhill, step by step, until you reach the bottom.
The slope is the gradient (the derivative): it points in the direction the function increases fastest. So to go down, you step in the opposite direction of the gradient. That's the whole algorithm.
Minimize a parabola
