Paper #2: Fighting Overfitting at Scale
With sixty million weights and only a million images, AlexNet could have memorized its way to a useless answer. Two defenses stopped it: more data, made for free, and dropout.
Tag
With sixty million weights and only a million images, AlexNet could have memorized its way to a useless answer. Two defenses stopped it: more data, made for free, and dropout.
A network can memorize training data perfectly. But memorized answers don't generalize. Overfitting is the enemy, and regularization is how you fight it.