In the last post, we talked about 2012 — the year deep learning stopped being an academic curiosity and started winning. But before we can appreciate why that moment mattered, we need to understand what these networks actually do, mechanically, when you feed them a number. Not the hand-wavy "it's like a brain" version. The actual math, the actual shapes, the actual reason a stack of matrix multiplications can approximate almost any function you throw at it.
So let's go back to the simplest possible neural network — one that isn't even really a network yet — and build up from there.
The perceptron: one line, one decision
Strip away everything, and a perceptron does exactly one thing: it draws a straight line (or a plane, or a hyperplane, depending on dimension) and asks which side are you on?
Given an input vector x and a weight vector w, the perceptron computes:
