Abstract
Field data collection and geospatial map generation are critical aspects in different fields such as road asset management, urban planning, and geospatial applications. However, one of the primary impediments to data collection is the availability of spatial and attribute data. This issue is aggravated by the high cost of conventional data collection and data processing methods and by the lack of geospatial data collection policies. This study proposes an inexpensive approach that enables real-time field data observation and geospatial data generation from video streams connected to a laptop and positioning sensors using deep learning technology. This proposed method was evaluated via an application called “DeepAutoMapping”, which was built on top of Python, then underwent through two different evaluation scenarios. The results demonstrated that the proposed approach is quick, easy to use and that it provides a high detection accuracy and an acceptable positioning accuracy in the outdoor environment. The proposed solution may also be considered as a pipeline for efficient and economical method of geospatial data collection and auto-map generation in the future.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Al-Azizi JI, Shafri HZM (2017) Performance evaluation of pedestrian locations based on contemporary smartphones. International Journal of Navigation & Observation 2017:1–10. https://doi.org/10.1155/2017/6750346
Alexey AB (2019). Windows and Linux version of Darknet Yolo V3 and V2 neural networks for object detection. Github: https://github.com/AlexeyAB/darknet#when-should-i-stop-training
Bradski G, Kaehler A (2008). Learning OpenCV: computer vision with the OpenCV Library. O' Reilly Media, Inc.
Caffe (2018). 2018. http://caffe.berkeleyvision.org/
Cao YT, Wang JM, Sun YK, Duan XJ (2013) Circle marker-based distance measurement using a single camera. Lecture Notes on Software Engineering 1(4):376. https://doi.org/10.7763/LNSE.2013.V1.80
Chen C, Seff A, Kornhauser A, Xiao J (2015a) on via region-based fully convolutional networks.DeepDriving: learning affordance for direct perception in autonomous driving. In proceedings of the IEEE international conference on computer vision (pp. 2722-2730). https://doi.org/10.1109/ICCV.2015.312
Chen X, Fang H, Lin TY, Vedantam R, Gupta S, Dollár P, Zitnick CL (2015b). Microsoft COCO captions: Data collection and evaluation server. https://arxiv.org/abs/1504.00325
Dai J, Li Y, He K, Sun J (2016) R-FCN: object detection via region-based fully convolutional networks. In advances in neural information processing systems (pp. 379-387). https://arxiv.org/abs/1605.06409
Damodharan P, Aravind P, Gomathi K, Keerthana, R, ManishaSamrin K (2017) Controlling input device based on Iris movement detection using artificial neural network
Deng J, Dong W, Socher R, Li LJ, Li K, Fei-Fei L (2009, June) ImageNet: a large-scale hierarchical image database. In 2009 IEEE conference on computer vision and pattern recognition (pp. 248-255). IEEE. https://doi.org/10.1109/CVPR.2009.5206848
Ding S, Lin L, Wang G, Chao H (2015) Deep feature learning with relative distance comparison for person re-identification. Pattern Recogn 48(10):2993–3003. https://doi.org/10.1016/j.patcog.2015.04.005
Everingham M, Van Gool L, Williams CK, Winn J, Zisserman A (2010) The Pascal visual object classes (VOC) challenge. Int J Comput Vis 88(2):303–338. https://doi.org/10.1007/s11263-009-0275-4
Girshick R (2015) Fast R-CNN. In proceedings of the IEEE international conference on computer vision (pp. 1440-1448). https://doi.org/10.1109/ICCV.2015.169
Girshick R, Donahue J, Darrell T, Malik J (2014) Rich feature hierarchies for accurate object detection and semantic segmentation. In proceedings of the IEEE conference on computer vision and pattern recognition (pp. 580-587). https://doi.org/10.1109/CVPR.2014.81
Goodchild MF (2009) Geographic information systems and science: today and tomorrow. Ann GIS 15(1):3–9. https://doi.org/10.1080/19475680903250715
Goodfellow I, Bengio Y, Courville A (2016) Deep learning. MIT press https://www.deeplearningbook.org/
Guimaraes RG, Rosa RL, De Gaetano D, Rodriguez DZ, Bressan G (2017) Age groups classification in social network using deep learning. IEEE Access 5:10805–10816. https://doi.org/10.1109/access.2017.2706674
Harrington P (2012) Machine learning in action. Manning, Greenwich
Holzmann C, Hochgatterer M (2012, June) Measuring distance wgaryith Mobile phones using single-camera stereo vision. In 2012 32nd International conference on distributed computing systems workshops (pp. 88-93). IEEE. https://doi.org/10.1109/ICDCSW.2012.22
Huang J, Rathod V, Sun C, Zhu M, Korattikara A, Fathi A, Fischer I, Wojna Z, Song Y, Guadarrama S, Murphy K (2017) Speed/accuracy trade-offs for modern convolutional object detectors. In proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7310-7311). https://doi.org/10.1109/CVPR.2017.351
Kang MS, Lim YC (2017) High performance and fast object detection in road environments. In 2017 seventh international conference on image processing theory, tools and applications (IPTA) (pp. 1-6). IEEE. https://doi.org/10.1109/IPTA.2017.8310148
Kanjee R, Bachoo AK, Carroll J (2013, October) Vision-based adaptive cruise control using pattern matching. In 2013 6th robotics and mechatronics conference (RobMech) (pp. 93-98). IEEE. https://doi.org/10.1109/RoboMech.2013.6685498
Korpilo S, Virtanen T, Lehvävirta S (2017) Smartphone GPS tracking-inexpensive and efficient data collection on recreational movement. Landsc Urban Plan 157:608–617. https://doi.org/10.1016/j.landurbplan.2016.08.005
Lin GT, Santoso PS, Lin CT, Tsai CC, Guo JI (2017, December) Stop line detection and distance measurement for road intersection based on deep learning neural network. In 2017 Asia-Pacific signal and information processing association annual summit and conference (APSIPA ASC) (pp. 692-695). IEEE. https://doi.org/10.1109/APSIPA.2017.8282121
Liu W, Anguelov D, Erhan D, Szegedy C, Reed S, Fu CY, Berg AC (2016, October) SSD: single shot Multibox detector. In European conference on computer vision (pp. 21-37). Springer, Cham. https://doi.org/10.1007/978-3-319-46448-0_2
Lu X, Tsao Y, Matsuda S, Hori C (2013, August) Speech enhancement based on deep denoising autoencoder. In Interspeech (Vol. 2013, pp. 436-440). https://www.isca-speech.org/archive/archive_papers/interspeech_2013/i13_0436.pdf
Lwin KK, Murayama Y (2011) Web-based GIS system for real-time field data collection using a personal Mobile phone. Journal of Geographic Information System 3(4):382. https://doi.org/10.4236/jgis.2011.34037
Martinez A, Ramirez F, Estrada H, Torres LA (2017) Generic module for collecting data in smart cities. International archives of the photogrammetry, Remote Sensing & Spatial Information Sciences, 42. https://doi.org/10.5194/isprs-archives-XLII-4-W3-65-2017
Microsoft (2019) https://www.microsoft.com/accessories/en-us/products/webcams/lifecam-hd-3000/t3h-00011
Nahhas FH, Shafri HZ, Sameen MI, Pradhan B, Mansor S (2018) Deep learning approach for building detection using Lidar-Orthophoto fusion. Journal of Sensors 2018:1–12. https://doi.org/10.1155/2018/7212307
Puente I, González-Jorge H, Arias P, Armesto J (2011) Land-based Mobile laser scanning systems: a review. International archives of the photogrammetry, remote sensing and spatial information sciences, 38(5/W12). https://doi.org/10.5194/isprsarchives-XXXVIII-5-W12-163-2011
Pulli K, Baksheev A, Kornyakov K, Eruhimov V (2012) Real-time computer vision with OpenCV. Commun ACM 55(6):61–69. https://doi.org/10.1145/2184319.2184337
PyTorch (2019). https://pytorch.org/
Radovic M, Adarkwa O, Wang Q (2017) Object recognition in aerial images using convolutional neural networks. Journal of Imaging 3(2):21. https://doi.org/10.3390/jimaging3020021
Redmon J, Farhadi A (2017) YOLO9000: better, faster, stronger. In proceedings of the IEEE conference on computer vision and pattern recognition (pp. 7263-7271). https://doi.org/10.1109/CVPR.2017.690
Redmon J, Farhadi A (2018) Yolov3: An Incremental Improvement. https://arxiv.org/abs/1804.02767
Redmon J, Divvala S, Girshick R, Farhadi A (2016) You Only Look Once: unified, real-time object detection. In proceedings of the IEEE conference on computer vision and pattern recognition (pp. 779-788). https://doi.org/10.1109/CVPR.2016.91
Ren S, He K, Girshick R, Sun J (2015) Faster R-CNN: towards real-time object detection with region proposal networks. In advances in neural information processing systems (pp. 91-99). https://arxiv.org/abs/1506.01497
Russell S, Norvig P (2002) Artificial intelligence: a modern approach
Saipullah K, Ismail NA, Anuar A, Sarimin N (2013) Comparison of feature extractors for real-time object detection on android smartphone. Journal of Theoretical & Applied Information Technology 47(1)
Silva JF, Camargo PO, Oliveira RA, Gallis RBA, Guardia MC, Reiss MLL, Silva RAC (2000) A street map built by a Mobile Maping system. International archives of photogrammetry and remote sensing, 33(B2; PART 2), pp.510-517
Singh SP, Kumar A, Darbari H, Singh L, Rastogi A, Jain S (2017). Machine translation using deep learning: An overview. In 2017 International Conference on Computer, Communications and Electronics (Comptelix) IEEE 162–167. https://doi.org/10.1109/COMPTELIX.2017.8003957
Sotelo MA, Fernandez D, Naranjo JE, Gonzalez C, García R, de Pedro T, Reviejo J (2004, September) Vision-based adaptive cruise control for intelligent road vehicles. In 2004 IEEE/RSJ international conference on intelligent robots and systems (IROS)(IEEE cat. No. 04CH37566) (Vol. 1, pp. 64-69). IEEE. https://doi.org/10.1109/IROS.2004.1389330
Stein GP, Mano O, Shashua A (2003, June) Vision-based ACC with a single camera: bounds on range and range rate accuracy. In IEEE IV2003 intelligent vehicles symposium. Proceedings (Cat. No. 03TH8683) (pp. 120-125). IEEE. https://ieeexplore.ieee.org/document/1212895
Sun Y, Wang X, Tang X (2014) Deep learning face representation from predicting 10,000 classes. In proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1891-1898). https://doi.org/10.1109/CVPR.2014.244
Tang S, Yuan Y (2015) Object detection based on convolutional neural network. In international conference-IEEE-2016. https://ieeexplore.ieee.org/document/8029130
TensorFlow (2018) https://www.tensorflow.org/
Tzutalin (2015) LabelImg. 2015. https://github.com/tzutalin/labelImg
Vakalopoulou M, Karantzalos K, Komodakis N, Paragios N (2015) July. Building detection in very high resolution multispectral data with deep learning features. In 2015 IEEE international geoscience and remote sensing symposium (IGARSS) (pp. 1873-1876). IEEE. https://doi.org/10.1109/IGARSS.2015.7326158
Vasari P (2019) Overfitting vs Underfitting in machine learning. 2019. http://datajango.com/over-fitting-vs-under-fitting-in-machine-learning/
Wang B, Tang S, Xiao JB, Yan QF, Zhang YD (2019) Detection and tracking based Tubelet generation for video object detection. J Vis Commun Image Represent 58:102–111. https://doi.org/10.1016/j.jvcir.2018.11.014
Yang B, Fang L, Li J (2013) Semi-automated extraction and delineation of 3D roads of street scene from Mobile laser scanning point clouds. ISPRS J Photogramm Remote Sens 79:80–93. https://doi.org/10.1016/j.isprsjprs.2013.01.016
Acknowledgments
The authors would like to acknowledge the support and facilities provided by Universiti Putra Malaysia (UPM). The comments from the anonymous reviewers are highly appreciated and significantly improved this manuscript.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Declarations
Not applicable.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Al-Azizi, J.I., Shafri, H.Z.M., Hashim, S.J.B. et al. DeepAutoMapping: low-cost and real-time geospatial map generation method using deep learning and video streams. Earth Sci Inform 15, 1481–1494 (2022). https://doi.org/10.1007/s12145-020-00529-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12145-020-00529-7