Deep Residual Learning for Image Recognition
Plain networks got worse as they got deeper, not from overfitting but from optimisation difficulty. Residual connections let each block learn a correction to its input instead of a whole new mapping, and suddenly networks with over a hundred layers trained cleanly.
Key ideas
- Degradation with depth is an optimisation problem, not a capacity problem
- Identity shortcuts add no parameters and almost no compute
- Bottleneck blocks make very deep networks affordable
Why read it now
The skip connection is the single most reused architectural idea in the field. Transformers, diffusion U-Nets, and modern vision models all depend on it.
Question to keep in mind
What does a residual block learn when the best thing it can do is nothing?