Learning on the Edge: Explicit Boundary Handling in CNNs

Abstract

Convolutional neural networks (CNNs) handle the case where filters extend beyond the image boundary using several heuristics, such as zero, repeat or mean padding. These schemes are applied in an ad-hoc fashion and, being weakly related to the image content and oblivious of the target task, result in low output quality at the boundary. In this paper, we propose a simple and effective improvement that learns the boundary handling itself. At training-time, the network is provided with a separate set of explicit boundary filters. At testing-time, we use these filters which have learned to extrapolate features at the boundary in an optimal way for the specific task. Our extensive evaluation, over a wide range of architectural changes (variations of layers, feature channels, or both), shows how the explicit filters result in improved boundary handling. Consequently, we demonstrate an improvement of 5% to 20% across the board of typical CNN applications (colorization, de-Bayering, optical flow, and disparity estimation).

Citation Style:    Display Alpha ACM SIGGRAPH IEEEtran Publication

Learning on the Edge: Explicit Boundary Handling in CNNs. Carlo Innamorati, Tobias Ritschel, Tim Weyrich, Niloy J. Mitra. Proc. British Machine Vision Conference (BMVC), 11 pages, Sep 2018. Best Student Paper Honourable Mention Award.Carlo Innamorati, Tobias Ritschel, Tim Weyrich, and Niloy J. Mitra. Learning on the edge: Explicit boundary handling in CNNs. In Proceedings of the British Machine Vision Conference (BMVC). BMVA Press, September 2018. selected for oral presentation.Innamorati, C., Ritschel, T., Weyrich, T., and Mitra, N. J. 2018. Learning on the edge: Explicit boundary handling in CNNs. In Proceedings of the British Machine Vision Conference (BMVC), BMVA Press. selected for oral presentation.C. Innamorati, T. Ritschel, T. Weyrich, and N. J. Mitra, “Learning on the edge: Explicit boundary handling in CNNs,” in Proceedings of the British Machine Vision Conference (BMVC). BMVA Press, Sep. 2018, selected for oral presentation. PDF (8.1 MB) Suppl. Material (34 MB) External Project Page BibTeX

Related Publication

Learning on the Edge: Investigating Boundary Filters in CNNs. Carlo Innamorati, Tobias Ritschel, Tim Weyrich, Niloy J. Mitra. International Journal of Computer Vision (IJCV), 10 pages, Springer, Oct 2019.Carlo Innamorati, Tobias Ritschel, Tim Weyrich, and Niloy J. Mitra. Learning on the edge: Explicit boundary handling in CNNs. International Journal of Computer Vision (IJCV), October 2019.Innamorati, C., Ritschel, T., Weyrich, T., and Mitra, N. J. 2019. Learning on the edge: Explicit boundary handling in CNNs. International Journal of Computer Vision (IJCV) (Oct.).C. Innamorati, T. Ritschel, T. Weyrich, and N. J. Mitra, “Learning on the edge: Explicit boundary handling in CNNs,” International Journal of Computer Vision (IJCV), Oct. 2019. [Web Page][PDF (8.1 MB)][External Project Page][BibTeX][DOI]

Acknowledgments

We thank Paul Guerrero, Aron Monszpart and Tuanfeng Yang Wang for their technical help in setting up and fixing the machines used to carry out the experiments in this work. This work was partially funded by the European Union’s Horizon 2020 research and innovation programme under the Marie Skłodowska-Curie grant agreement No 642841 (DISTRO), by the ERC Starting Grant SmartGeometry (StG-2013-335373), and by the UK Engineering and Physical Sciences Research Council (grant EP/K023578/1).