Abstract
The availability of images of events almost in real-time on social media has a prospect in many application developments. A humanitarian technology for disaster type and level assessment can be developed using the images and video available on social media. In this paper, we investigate the potential use of various available deep learning techniques to develop such an application. For our research, based on the use of publicly available image data, we have started collecting disaster images from various sources from South Asia. We created the South Asia Disaster (SAD) image dataset containing 493 images from various online news portals. Using the Keras as our framework to run our models: Visual Geometry Group (VGG-16 and VGG-19), Inception-V3 and Inception-ResNet-V2 (ResNet: Residual Network). However, to boost up the training speed, we dropped the fully connected layer and added a small, fully connected model. To identify the five different disasters: fire disaster, flood disaster, human disaster, infrastructure disaster, natural disaster; our proposed method with VGG-16 model’s recognition accuracy was 84.51%, which is the highest accuracy on the SAD dataset. After performing the testing, we calculate the VGG-16 classifier’s attention to visualize which part of the disaster images VGG-16 pays attention.
Similar content being viewed by others
References
Ahmad K, Conci N, Natale F (2018) A comparative study of global and deep features for the analysis of user-generated natural disaster related images (accepted for presentation in IVMSP 2018)
Al Jazeera ZHC, Media Network, aljazeera. In: viewed 20 Jun 2019. https://www.aljazeera.com/indepth/inpictures/pictures-bangladesh-front-lines-climate-crisis-191203102447517.html
Alam F, Imran M, Ofli F (2017) Image4act: online social media image processing for disaster response. In: Proceedings of the 2017 IEEE/ACM international conference on advances in social networks analysis and mining 2017, ASONAM ’17. Association for Computing Machinery, New York, NY, USA, pp 601–604. https://doi.org/10.1145/3110025.3110164
Amin MA, Mohammed MK (2015) Overview of the imageclef 2015 medical clustering task. In: CLEF
Anam M (1991) The daily star. In: viewed 20 Jun 2019. https://www.thedailystar.net/
Avgerinakis K, Moumtzidou A, Andreadis S, Michail E, Gialampoukidis I, Vrochidis S, Kompatsiaris I (2017) Visual and textual analysis of social media and satellite images for flood detection @ multimedia satellite task mediaeval 2017
Bose J, The Times of India. In: viewed 20 Jun 2019. https://timesofindia.indiatimes.com/
Giannakeris P, Avgerinakis K, Karakostas A, Vrochidis S, Kompatsiaris I (2018) People and vehicles in danger—a fire and flood detection system in social media. In: 2018 IEEE 13th image, video, and multidimensional signal processing workshop (IVMSP), pp 1–5
Jetley S, Lord NA, Lee N, Torr PHS (2018) Learn to pay attention. CoRR. arXiv:1804.02391
Mark Getty JK, gettyimages. In: viewed 20 Jun 2019, https://www.gettyimages.com/photos/india-fire?mediatype=photography&page=3&phrase=india
Mark Getty JK, gettyimages. In: viewed 20 Jun 2019
Mark Getty JK, gettyimages. In: viewed 20 Jun 2019. https://www.gettyimages.com/photos/bangladesh-flood?mediatype=photography&phrase=bangladesh
Mouzannar H, Rizk Y, Awad M (2018) Damage identification in social media posts using multimodal deep learning. In: ISCRAM
Muhammad K, Ahmad J, Baik SW (2018) Early fire detection using convolutional neural networks during surveillance for effective disaster management. Neurocomputing 288(C):30–42. https://doi.org/10.1016/j.neucom.2017.04.083
Network AJM, aljazeera. In: viewed 20 Jun 2019. https://www.aljazeera.com/
Nguyen DT, Ofli F, Imran M, Mitra P (2017) Damage assessment from social media imagery data during disasters. In: Proceedings of the 2017 IEEE/ACM international conference on advances in social networks analysis and mining 2017, ASONAM ’17, Association for Computing Machinery, New York, NY, USA, pp 569–576. https://doi.org/10.1145/3110025.3110109
Rizk Y, Jomaa HS, Awad M, Castillo C (2019) A computationally efficient multi-modal classification approach of disaster-related twitter images. In: Proceedings of the 34th ACM/SIGAPP symposium on applied computing, SAC ’19, Association for Computing Machinery, New York, NY, USA, pp 2050–2059. https://doi.org/10.1145/3297280.3297481
Szegedy C, Ioffe S, Vanhoucke V, Alemi AA (2017) Inception-v4, inception-resnet and the impact of residual connections on learning. In: Proceedings of the thirty-first AAAI conference on artificial intelligence, AAAI’17. AAAI Press, pp 4278–4284
Szegedy C, Vanhoucke V, Ioffe S, Shlens J, Wojna Z (2016) Rethinking the inception architecture for computer vision. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 2818–2826
Zafar Sobhan KAA (2013) Dhakatribune. In: viewed 20 Jun 2019. https://www.dhakatribune.com/
Zhang X, Zou J, He K, Sun J (2016) Accelerating very deep convolutional networks for classification and detection. IEEE Trans Pattern Anal Mach Intell 38(10):1943–1955. https://doi.org/10.1109/TPAMI.2015.2502579
Zhou B, Khosla A, Lapedriza A, Oliva A, Torralba A (2016) Learning deep features for discriminative localization. In: 2016 IEEE conference on computer vision and pattern recognition (CVPR), pp 2921–2929
Acknowledgements
This project is supported by a grant from the Bangladesh Information and Communication Technology Ministries ICT Division, and Independent University, Bangladesh (IUB). We would also like to thank Dr. Moinul Islam Zaber, Associate Professor, CSE, Dhaka University, Bangladesh, for his valuable suggestions during this project.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Arif, Amin, M.A., Ali, A.A. et al. Visual attention-based comparative study on disaster detection from social media images. Innovations Syst Softw Eng 16, 309–319 (2020). https://doi.org/10.1007/s11334-020-00368-1
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11334-020-00368-1