Archive
45 posts, newest first. Page 1 of 5.
Before we build anything, we need one idea: to a computer, a picture is a grid of numbers. A small cartoon dog, saved as a text file, shows this directly. Every one of its numbers is readable.
Given a grid of numbers, name what the picture shows. It sounds simple. Seven specific things make it one of the hardest problems computers ever faced.
Before we build anything clever, let's try the dumbest possible approach to image classification. It's simpler than it sounds, and it teaches us something crucial about why this problem is hard.
Before you can tell whether your classifier works, you have to measure it. The obvious way to measure has a trap in it, one the CS231n notes warn about. Here's the trap and the fix the course recommends.
We've tried memorizing examples (nearest neighbor). Now let's try something smarter: score each image and pick the highest score. It's simple, interpretable, and it teaches us how learning actually works.
A classifier can guess. But how do you measure how badly it guessed? Loss functions turn 'wrongness' into a number, and that number is what drives learning.
Part 6 takes hinge loss and softmax as given. But why these formulas and not some other arbitrary ones? Why the max? Why the '+1'? Why raise e to a power instead of just taking absolute values? Here's where they come from.
We have scores. We have loss. Now we need to adjust weights to reduce that loss. Gradient descent is the algorithm that does this, and it works by walking downhill in the dark.
Gradient descent needs gradients. But computing them directly would be impossibly slow. Backpropagation is the trick that makes it fast.
A single linear classifier is rigid. But what if you stack multiple layers, each adding non-linearity? That gives you a neural network.