Paper #3: What ResNet Won, and Where It Stopped
ResNet won the 2015 ImageNet contest with 3.57% top-5 error. On a small dataset, a 1,202-layer version did worse than a 110-layer one, and the paper says why.
Tag
overfitting
Paper #2: Fighting Overfitting at Scale
With sixty million weights and only a million images, AlexNet could have memorized its way to a useless answer. Two defenses stopped it: more data, made for free, and dropout.
Paper #1, Part 11: Memorizing vs. Learning (Overfitting and Regularization)
A network can memorize training data perfectly. But memorized answers don't generalize. Overfitting is the enemy, and regularization is how you fight it.