Tag
Papers #2 to #4 improved the process of identifying a single thing in an image. Paper #5 labels every pixel instead: road here, pedestrian there, sky above. That needs a network that keeps detail while still seeing a wide area.
To go past 34 layers without the cost growing too fast, ResNet changed the shape of its blocks: shrink the data, do the main work, then expand it again. ResNet-152 ended up cheaper to run than the 19-layer VGG.
ResNet's fix is one addition: let each block of layers learn only a change to its input, and add the input back at the end. Then doing nothing becomes easy.
After AlexNet, adding layers kept helping, up to a point. Past about twenty layers, plain networks started scoring worse, even on the images they were trained on. ResNet starts from that puzzle.
The big ideas are in place. This post gathers the smaller tricks AlexNet used, including one now considered obsolete, and the plain recipe for actually training it.
AlexNet didn't invent anything new in its shape. It's the convolutional network from Paper #1, made big: five convolutional layers, three fully-connected ones, and sixty million weights.
Fifteen concepts. One goal: from raw pixels to classification. This is how it all fits together, and why CS231n became foundational reading.
Training a large CNN from scratch requires massive amounts of data and compute. Transfer learning reuses a network trained on one task as a head start on another.
Neural networks learn patterns, but we can't see them directly. Visualization techniques let us peek inside and understand what features a network has learned.
Convolutions find local patterns. Pooling zooms out. Stack them together and you build a hierarchy: edges → textures → shapes → objects.
A fully-connected network treats every pixel equally. Convolutions exploit one fact: important patterns are local. So a convolution scans the image in small patches, looking for a pattern wherever it appears.