But what is a neural network?

This first chapter builds the object itself. The problem is deliberately small — read a handwritten digit off a 28×28 grid of pixels — and the point is that the mechanism which solves it is the same mechanism running inside every model later in this course. Nothing is assumed beyond arithmetic and a willingness to look at a matrix.

What this chapter covers

• A neuron as one number between 0 and 1 — its activation
• Why the 784 pixels of the image become the first layer
• Weights, the weighted sum, and the bias that shifts the threshold
• Squishing the sum with a sigmoid, and why ReLU displaced it
• An entire layer transition written as one matrix–vector product, σ(Wa + b)
• The hope that hidden layers learn edges, then loops, then digits

Take that 13,002 seriously. Chapter 2 is a search through a 13,002-dimensional space, and the size of the number is exactly what makes the search interesting rather than trivial.