Abstract
Vehicle accidents due to drowsiness in drivers take thousands of lives each year worldwide. This fact clearly exhibits a need for a drowsiness detection application that can help prevent such accidents and ultimately save lives. In this work, we propose a novel deep learning methodology based on Convolutional Neural Networks (CNN) to tackle this problem. The proposed methodology treats drowsiness detection as an object detection task, and from an incoming video stream of a driver, detects and localizes open and closed eyes. MobileNet CNN architecture with Single Shot Multibox Detector (SSD) is used for this task of object detection. A separate algorithm is then used to detect driver drowsiness based on the output from the MobileNet-SSD architecture. In order to train the MobileNet-SSD Network a custom dataset of about 6000 images was compiled and labeled with the objects face, eye open and eye closed. Out of these, 350 images were randomly separated and used to test the trained model. The trained model was evaluated on the test dataset using the PASCAL VOC metric and achieved a Mean Average Precision (mAP) of 0.84 on these categories. The proposed methodology, while maintaining reasonable accuracy, is also computationally efficient and cost effective, as it can process an incoming video stream in real time on a standalone mobile device without the need of expensive hardware support. It can easily be deployed on cheap embedded devices in vehicles, such as the Raspberry Pi 3 or a mobile smartphone.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Mohn, T.: Around 5,000 people were killed last year due to drowsy driving. https://www.forbes.com/sites/tanyamohn/2016/08/08/nearly-83-6-million-american-drivers-are-sleep-deprived-new-report-highlights-dangers-high-cost/#3bbc82664007
Rapaport, L.: Drowsy drivers often behind fatal crashes. https://www.reuters.com/article/us-health-driving-sleep/drowsy-drivers-often-behind-fatal-crashes-idUSKBN15P2PM
Facts and Stats: Drowsy driving – stay alert, arrive alive. http://drowsydriving.org/about/facts-and-stats/
Saini, V., Saini, R.: Driver drowsiness detection system and techniques: a review. Int. J. Comput. Sci. Inf. Technol. 5, 4245–4249 (2014)
Bhatt, P.P., Trivedi, J.A.: Various methods for driver drowsiness detection: an overview. Int. J. Comput. Sci. Eng. 9, 70–74 (2017)
Chieh, T.C., Mustafa, M.M., Hussain, A., Hendi, S.F., Majlis, B.Y.: Development of vehicle driver drowsiness detection system using Electrooculogram (EOG). In: 1st International Conference on Computers, Communications, and Signal Processing With Special Track on Biomedical Engineering (CCSP), Kuala Lumpur, Malaysia, pp. 165–168 (2005)
Takei, Y., Furukawa, Y.: Estimate of driver’s fatigue through steering motion. In: 2005 IEEE International Conference on Systems, Man and Cybernetics, vol. 2, pp. 1–6 (2005)
Lee, A.: Comparing deep neural networks and traditional vision algorithms in mobile robotics. Swart. Coll. (2015)
Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60(2), 91–110 (2004)
Lazebnik, S., Schmid, C., Ponce, J.: Beyond bags of features: spatial pyramid matching for recognizing natural scene categories, pp. 2169–2178. IEEE Computer Society (2006)
Dalal, N., Triggs, B.: Histograms of oriented gradients for human detection. In: Conference on Computer Vision and Pattern Recognition (2005)
Bittner, R., Hána, K., Poušek, L., Smrka, P., Schreib, P., Vysoký, P.: Detecting of fatigue states of a car driver. In: Brause, R.W., Hanisch, E. (eds.) ISMDA 2000. LNCS, vol. 1933, pp. 260–273. Springer, Heidelberg (2000). https://doi.org/10.1007/3-540-39949-6_32
Bergasa, L.M., Nuevo, J.: Real-time system for monitoring driver vigilance. In: Proceedings of the IEEE International Symposium on Industrial Electronics, vol. III, pp. 1303–1308 (2005)
Suzuki, M., Yamamoto, N., Yamamoto, O., Nakano, T., Yamamoto, S.: Measurement of driver’s consciousness by image processing - a method for presuming driver’s drowsiness by eye-blinks coping with individual differences. In: Conference Proceedings - International Conference on Systems, Man and Cybernetics, vol. 4, pp. 2891–2896 (2007)
Technology | Smart Eye. http://smarteye.se/technology/
Fan, C.: Driver fatigue detection based. In: Proceedings of the 2004 IEEE International Conference on Networking, Sensing and Control, pp. 7–12 (2004)
Abtahi, S., Hariri, B., Shirmohammadi, S.: Driver drowsiness monitoring based on yawning detection. In: IEEE International Instrumentation and Measurement Technology Conference, Binjiang, Hangzhou, China (2011)
Danisman, T., Bilasco, I.M., Djeraba, C., Ihaddadene, N.: Drowsy driver detection system using eye blink patterns. In: Proceedings of the 2010 International Conference on Machine and Web Intelligence, ICMWI 2010, pp. 230–233 (2010)
Bronte, S., Bergasa, L.M., Almaz, J., Yebes, J.: Vision-based drowsiness detector for real driving conditions (2012)
Krizhevsky, A., Sutskever, I., Hinton, G.E.: ImageNet classification with deep convolutional neural networks. In: Neural Information Processing Systems (2012)
He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. In: Conference on Computer Vision and Pattern Recognition (2016)
Ren, S., He, K., Girshick, R., Sun, J.: Faster R-CNN: towards real-time object detection with region proposal networks. In: Neural Information Processing Systems, pp. 1–14 (2015)
Shelhamer, E., Long, J., Darrell, T.: Fully convolutional networks for semantic segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 39, 640–651 (2017)
Dwivedi, K., Biswaranjan, K., Sethi, A.: Drowsy driver detection using representation learning. In: Souvenir 2014 IEEE International Advanced Computing Conference, IACC 2014, pp. 995–999 (2014)
Reddy, B., Kim, Y.-H., Yun, S., Seo, C., Jang, J.: Real-time driver drowsiness detection for embedded system using model compression of deep neural networks. In: Conference on Computer Vision and Pattern Recognition Workshops (CVPRW), pp. 438–445 (2017)
Amazon.com: NVIDIA Jetson TK1 Development Kit: Computers & Accessories. https://www.amazon.com/NVIDIA-Jetson-TK1-Development-Kit/dp/B00L7AWOEC
Jabbar, R., Al-Khalifa, K., Kharbeche, M., Alhajyaseen, W., Jafari, M., Jiang, S.: Real-time driver drowsiness detection for android application using deep neural networks techniques. Procedia Comput. Sci. 130, 400–407 (2018)
Lyu, J., Yuan, Z., Chen, D.: Long-term multi-granularity deep framework for driver drowsiness detection (2018)
Howard, A.G., et al.: MobileNets: efficient convolutional neural networks for mobile vision applications (2017)
Liu, W., et al.: SSD: single shot MultiBox detector. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9905, pp. 21–37. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46448-0_2
Huang, J., et al.: Speed/accuracy trade-offs for modern convolutional object detectors. In: Proceedings - 30th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2017, pp. 3296–3305, January 2017
Schiffman, H.R.: Sensation and Perception: An Integrated Approach. Wiley, New York (2000)
Jain, V., Learned-Miller, E.: FDDB: a benchmark for face detection in unconstrained settings. UM-CS-2010-009 (2010)
Abtahi, S., Omidyeganeh, M., Shirmohammadi, S., Hariri, B.: YawDD. In: Proceedings of the 5th ACM Multimedia Systems Conference on - MMSys 2014, pp. 24–28. ACM Press, New York (2014)
Song, F., Tan, X., Liu, S.: Eyes closeness detection from still images with multi-scale histograms of principal oriented gradients (2014). http://parnec.nuaa.edu.cn/xtan/data/ClosedEyeDatabases.html
Pan, S.J., Fellow, Q.Y.: A survey on transfer learning. IEEE Trans. Knowl. Data Eng. 22, 1345–1359 (2010)
Oquab, M., Bottou, L., Laptev, I., Sivic, J.: Learning and transferring mid-level image representations using convolutional neural networks. In: IEEE Conference on Computer Vision and Pattern Recognition, pp. 1717–1724 (2014)
Lin, T.-Y., et al.: Microsoft COCO: common objects in context. In: Fleet, D., Pajdla, T., Schiele, B., Tuytelaars, T. (eds.) ECCV 2014. LNCS, vol. 8693, pp. 740–755. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-10602-1_48
Everingham, M., et al.: The PASCAL Visual Object Classes (VOC) challenge. Int. J. Comput. Vis. 88, 303–338 (2010)
TensorFlow Lite | TensorFlow. https://www.tensorflow.org/lite/
Xperia TM Z (2013)
Krishnamoorthi, R.: Quantizing deep convolutional networks for efficient inference: a whitepaper (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Shakeel, M.F., Bajwa, N.A., Anwaar, A.M., Sohail, A., Khan, A., Haroon-ur-Rashid (2019). Detecting Driver Drowsiness in Real Time Through Deep Learning Based Object Detection. In: Rojas, I., Joya, G., Catala, A. (eds) Advances in Computational Intelligence. IWANN 2019. Lecture Notes in Computer Science(), vol 11506. Springer, Cham. https://doi.org/10.1007/978-3-030-20521-8_24
Download citation
DOI: https://doi.org/10.1007/978-3-030-20521-8_24
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-20520-1
Online ISBN: 978-3-030-20521-8
eBook Packages: Computer ScienceComputer Science (R0)