Paper #1, Part 11: Memorizing vs. Learning (Overfitting and Regularization)
A network can memorize training data perfectly. But memorized answers don't generalize. Overfitting is the enemy, and regularization is how you fight it.
Category
Fundamentals
Core mechanics of how machine learning models actually learn.
Paper #1, Part 10: Garbage In, Garbage Out (Why Data Prep Matters)
Before a network can learn from pixels, those pixels need to be put on a consistent scale. It's a small step that makes training much more stable.
Paper #1, Part 9: Neurons and Networks (Stacking Simple Decisions Into Something Smart)
A single linear classifier is rigid. But what if you stack multiple layers, each adding non-linearity? That gives you a neural network.
Paper #1, Part 8: Backpropagation (How Blame Gets Passed Backward)
Gradient descent needs gradients. But computing them directly would be impossibly slow. Backpropagation is the trick that makes it fast.
Paper #1, Part 7: Learning by Trial and Error (Gradient Descent)
We have scores. We have loss. Now we need to adjust weights to reduce that loss. Gradient descent is the algorithm that does this, and it works by walking downhill in the dark.
Why Do Hinge Loss and Softmax Look the Way They Do?
Part 6 takes hinge loss and softmax as given. But why these formulas and not some other arbitrary ones? Why the max? Why the '+1'? Why raise e to a power instead of just taking absolute values? Here's where they come from.
Paper #1, Part 6: How Does the Computer Know It's Wrong? Loss Functions
A classifier can guess. But how do you measure how badly it guessed? Loss functions turn 'wrongness' into a number, and that number is what drives learning.
Paper #1, Part 5: Giving the Computer a Scorecard (Linear Classifiers)
We've tried memorizing examples (nearest neighbor). Now let's try something smarter: score each image and pick the highest score. It's simple, interpretable, and it teaches us how learning actually works.
Paper #1, Part 4: How to Test a Model Without Cheating
Before you can tell whether your classifier works, you have to measure it. The obvious way to measure has a trap in it, one the CS231n notes warn about. Here's the trap and the fix the course recommends.