Paper #5 · 2016
Multi-Scale Context Aggregation by Dilated Convolutions
Seeing wider context in an image without throwing away resolution.
Start readingPapers #2 to #4 improved the process of identifying a single thing in an image. Paper #5 labels every pixel instead: road here, pedestrian there, sky above. That needs a network that keeps detail while still seeing a wide area.
Classification networks shrink their feature maps on purpose: it lets later layers see wide areas cheaply. For one label that costs nothing. For a label at every pixel, it throws away the detail the answer needs.
A dilated convolution spaces out the positions a filter reads, so the same few weights cover a wider area. Stack a few with growing gaps, and the area each output sees grows exponentially, while the map never shrinks.
The authors removed VGG-16's last two shrinking steps and beat earlier segmentation networks by more than 5 points. Then they added a small stack of dilated layers, started so each one passes its input through unchanged, and gained more.
The context module raised accuracy on everyday photos and on street scenes from car cameras. The bigger result was the argument behind it: a network that has to label every pixel can keep its maps large all the way through.