Abstract
The paper describes usage of deep neural networks for flat roof defect segmentation on aerial images. Such architectures as U-Net, DeepLabV3+ and HRNet+ OCR are studied for recognition five categories of roof defects: “hollows”, “swelling”, “folds”, “patches” and “breaks”. Paper introduces RoofD dataset containing 6400 image pairs: aerial photos and corresponding ground truth masks. Based on this dataset different approaches to neural networks training are analyzed. New SDice coefficient with categorical cross-entropy is studied for precise training of U-Net and proposed light U-NetMCT architecture. Weighted categorical cross-entropy is studied for DeepLabV3+ and HRNet+ OCR training. It is shown that these training methods allow correctly recognize rare categories of defects. The state-of-the-art model multi-scale HRNet+ OCR achieves the best quality metric of 0.44 mean IoU. In sense of inference time the fastest model is U-NetMCT and DeeplabV3+ with worse quality of 0.33–0.37 mean IoU. The most difficult category for segmentation is “patches” because of small amount of images with this category in the dataset. Paper also demonstrates the possibility of implementation of the obtained models in the special software for automation of the roof state examination in industry, housing and communal services.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Kofler, C., Spöck, G., Muhr, R.: Classifying defects in topography images of silicon wafers. In: Winter Simulation Conference (WSC), pp. 3646–3657 (2017)
Soukup, D., Huber-Mörk, R.: Convolutional neural networks for steel surface defect detection from photometric stereo images. Lect. Notes Comput. Sci., vol. 8887, pp. 668 –677 (2014)
Faghih-Roohi, S., et al.: Deep convolutional neural networks for detection of rail surface defects. In: International Joint Conference Neural Networks (IJCNN), pp. 2584–2589 (2016)
Maestro-Watson, D., Balzategui, J., Eciolaza, L., Arana-Arexolaleiba, N.: Deflectometric data segmentation for surface inspection: a fully convolutional neural network approach. J. Electron. Imaging 29(4), 041007 (2020)
Li, S., Zhao, X., Zhou G.: Automatic pixel-level multiple damage detection of concrete structure using fully convolutional network. Comput. Aided Civil Infrastruct. Eng., 34(7), 616–634 (2019)
Yudin, D., Naumov, A., Dolzhenko, A., Patrakova, E.: Software for roof defects recognition on aerial photographs. J. Phys: Conf. Ser. 1015(3), 032152 (2018)
Computer Vision Annotation Tool (CVAT). https://github.com/opencv/cvat. Accessed 10 May 2020
Ronneberger, O., Fischer, P., Brox, T.: U-net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28
Sudre, C.H., Li, W., Vercauteren, T., Ourselin, S., Jorge Cardoso, M.: Generalised dice overlap as a deep learning loss function for highly unbalanced segmentations. In: Cardoso, M.J., Arbel, T., Carneiro, G., Syeda-Mahmood, T., Tavares, J.M.R.S., Moradi, M., Bradley, A., Greenspan, H., Papa, J.P., Madabhushi, A., Nascimento, J.C., Cardoso, J.S., Belagiannis, V., Lu, Z. (eds.) DLMIA/ML-CDS -2017. LNCS, vol. 10553, pp. 240–248. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-67558-9_28
Yudin, D.A., Skrynnik, A., Krishtopik, A., Belkin, I., Panov, A.I.: Object detection with deep neural networks for reinforcement learning in the task of autonomous vehicles path planning at the intersection. Opt. Memory Neural Networks 28(4), 283–295 (2019). https://doi.org/10.3103/S1060992X19040118
Chen, L.-C., Zhu, Y., Papandreou, G., Schroff, F., Adam, H.: Encoder-decoder with atrous separable convolution for semantic image segmentation. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) ECCV 2018. LNCS, vol. 11211, pp. 833–851. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01234-2_49
Yuan, Y., Chen, X., Wang, J.: Object-contextual representations for semantic segmentation. ArXiv, abs/1909.11065 (2019)
Chollet, F.: Xception: deep learning with depthwise separable convolutions. CVPR 2017, arXiv:1610.02357 (2017)
Wang, J., Sun, K., Cheng, T., Jiang, B., Deng, C., Zhao, Y., Liu, D., Mu, Y., Tan, M., Wang, X., Liu, W., Xiao, B.: Deep high-resolution representation learning for visual recognition. IEEE Trans. Pattern Anal. Mach. Intell. (2020)
Acknowledgment
Task formulation, RoofD dataset and training approaches of deep neural networks (with modified Dice coefficient and weighted cross-entropy) were developed during the project of Russian Fund of Basic Research No 18-47-310009. Experimental results were obtained during works supported by the Government of the Russian Federation (Agreement No. 075-02-2019-967).
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Editor(s) (if applicable) and The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Yudin, D.A., Adeshkin, V., Dolzhenko, A.V., Polyakov, A., Naumov, A.E. (2021). Roof Defect Segmentation on Aerial Images Using Neural Networks. In: Kryzhanovsky, B., Dunin-Barkowski, W., Redko, V., Tiumentsev, Y. (eds) Advances in Neural Computation, Machine Learning, and Cognitive Research IV. NEUROINFORMATICS 2020. Studies in Computational Intelligence, vol 925. Springer, Cham. https://doi.org/10.1007/978-3-030-60577-3_20
Download citation
DOI: https://doi.org/10.1007/978-3-030-60577-3_20
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60576-6
Online ISBN: 978-3-030-60577-3
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)