Category
Image models, computer vision, and how AI understands pictures.
The context module raised accuracy on everyday photos and on street scenes from car cameras. The bigger result was the argument behind it: a network that has to label every pixel can keep its maps large all the way through.
The authors removed VGG-16's last two shrinking steps and beat earlier segmentation networks by more than 5 points. Then they added a small stack of dilated layers, started so each one passes its input through unchanged, and gained more.
A dilated convolution spaces out the positions a filter reads, so the same few weights cover a wider area. Stack a few with growing gaps, and the area each output sees grows exponentially, while the map never shrinks.
Classification networks shrink their feature maps on purpose: it lets later layers see wide areas cheaply. For one label that costs nothing. For a label at every pixel, it throws away the detail the answer needs.
Papers #2 to #4 improved the process of identifying a single thing in an image. Paper #5 labels every pixel instead: road here, pedestrian there, sky above. That needs a network that keeps detail while still seeing a wide area.
With the original blocks, a 1,001-layer network did worse than a 110-layer one. With pre-activation blocks, it reached 4.92% error on CIFAR-10, the best in the paper. The authors give two reasons.
With the shortcut left alone, one step still sat on the path between blocks: the ReLU after the addition. Moving batch normalization and ReLU to before each weight layer removed it, and error went down.
The authors tried scaling, gating, a 1×1 convolution, and dropout on the shortcuts of a 110-layer ResNet. Every one did worse than a shortcut that passes its input through unchanged.
If the shortcut and the step after the addition both do nothing, any block's output is an earlier block's output plus every change in between. Going backward, the same fact gives the correction a route that does not fade.
ResNet showed that shortcuts work but offered only a hypothesis about why. A year later, the same authors followed the signal through the shortcuts, and found one step in the original block that got in the way.
ResNet won the 2015 ImageNet contest with 3.57% top-5 error. On a small dataset, a 1,202-layer version did worse than a 110-layer one, and the paper says why.
To go past 34 layers without the cost growing too fast, ResNet changed the shape of its blocks: shrink the data, do the main work, then expand it again. ResNet-152 ended up cheaper to run than the 19-layer VGG.
The same two networks from the first post, rebuilt with shortcuts. Without them, going from 18 to 34 layers made error worse. With them, it made error better by about three points.
ResNet's fix is one addition: let each block of layers learn only a change to its input, and add the input back at the end. Then doing nothing becomes easy.
The usual suspect when deep networks fail is a learning signal that fades as it travels back through the layers. By 2015 a technique called batch normalization had largely dealt with that, so ResNet's problem had to be something else.
After AlexNet, adding layers kept helping, up to a point. Past about twenty layers, plain networks started scoring worse, even on the images they were trained on. ResNet starts from that puzzle.
With AlexNet trained, this last post looks at what the network learned, and why this paper mattered.
The big ideas are in place. This post gathers the smaller tricks AlexNet used, including one now considered obsolete, and the plain recipe for actually training it.
With sixty million weights and only a million images, AlexNet could have memorized its way to a useless answer. Two defenses stopped it: more data, made for free, and dropout.
ReLU made training fast enough. But the network was too big to fit on a single graphics card, so the authors did something unusual: they cut it in half.
AlexNet's shape is familiar from Paper #1. What we skipped is how the activation function controls training speed. Picking a fast one, ReLU, is the first thing that made training a network as big as AlexNet possible.
AlexNet didn't invent anything new in its shape. It's the convolutional network from Paper #1, made big: five convolutional layers, three fully-connected ones, and sixty million weights.
Before 2012, teaching a computer to recognize objects meant designing the clues by hand. Then a deep neural network won the hardest contest in the field, and not by a little.
Fifteen concepts. One goal: from raw pixels to classification. This is how it all fits together, and why CS231n became foundational reading.
Training a large CNN from scratch requires massive amounts of data and compute. Transfer learning reuses a network trained on one task as a head start on another.
Neural networks learn patterns, but we can't see them directly. Visualization techniques let us peek inside and understand what features a network has learned.
Convolutions find local patterns. Pooling zooms out. Stack them together and you build a hierarchy: edges → textures → shapes → objects.
A fully-connected network treats every pixel equally. Convolutions exploit one fact: important patterns are local. So a convolution scans the image in small patches, looking for a pattern wherever it appears.
Before we build anything clever, let's try the dumbest possible approach to image classification. It's simpler than it sounds, and it teaches us something crucial about why this problem is hard.
Given a grid of numbers, name what the picture shows. It sounds simple. Seven specific things make it one of the hardest problems computers ever faced.
Before we build anything, we need one idea: to a computer, a picture is a grid of numbers. A small cartoon dog, saved as a text file, shows this directly. Every one of its numbers is readable.