Skip to main content
SpringerLink
Log in

Robust multimodal biometric authentication on IoT device through ear shape and arm gesture

  • Published:
Multimedia Tools and Applications Aims and scope Submit manuscript

Abstract

Nowadays, authentication is required for both physical access to buildings and internal access to computers and systems. Biometrics are one of the emerging technologies used to protect these highly sensitive structures. However, biometric systems based on a single trait enclose several problems such as noise sensitivity and vulnerability to spoof attacks. In this regard, we present in this paper a fully unobtrusive and robust multimodal authentication system that automatically authenticates a user by the way he/she answers his/her phone, after extracting ear and arm gesture biometric modalities from this single action. To deal the challenges facing ear and arm gesture authentication systems in real-world applications, we propose a new method based on image fragmentation that makes the ear recognition more robust in relation to occlusion. The ear feature extraction process has been made locally using Local Phase Quantization (LPQ) in order to get robustness with respect to pose and illumination variation. We also propose a set of four statistical metrics to extract features from arm gesture signals. The two modalities are combined on score-level using a weighted sum. In order to evaluate our contribution, we conducted a set of experiments to demonstrate the contribution of each of the two biometrics and the advantage of their fusion on the overall performance of the system. The multimodal biometric system achieves an equal error rate (EER) of 5.15%.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

References

  1. Abate AF, Nappi M, Ricciardi S (2019) I-Am: Implicitly authenticate me person authentication on mobile devices through ear shape and arm gesture. IEEE Trans Syst Man Cybern Syst 49(3):469–481. https://doi.org/10.1109/TSMC.2017.2698258

    Article  Google Scholar 

  2. Abozaid A, Haggag A, Kasban H, Eltokhy M (2018) Multimodal biometric scheme for human authentication technique based on voice and face recognition fusion. Multimed Tools Appl. https://doi.org/10.1007/s11042-018-7012-3

  3. Ahonen T, Hadid A, Pietikainen M (2006) Face description with local binary patterns: Application to face recognition. IEEE Trans Pattern Anal Mach Intell 28(12):2037–2041. https://doi.org/10.1109/TPAMI.2006.244

    Article  Google Scholar 

  4. Akhtar Z, Sandeep K, Nasir A (2011) Spoof attacks on multimodal biometric systems. Int Conf Inf Netw Technol 4:46–51

    Google Scholar 

  5. Akhtar Z, Buriro A, Crispo B, Falk T. H. (2017) Multimodal smartphone user authentication using touchstroke, phone-movement and face patterns. In: Global conference on signal and information processing (GlobalSIP), pp 1368–1372. https://doi.org/10.1109/GlobalSIP.2017.8309185

  6. Aronowitz H, Li M, Toledo-Ronen O, Harary S, Geva A, Ben-David S, Rendel A, Hoory R, Ratha N, Pankanti S, Nahamoo D (2014) Multi-modal biometrics for mobile authentication. IEEE Int Jt Conf Biom, pp 1–8. https://doi.org/10.1109/BTAS.2014.6996269

  7. Arteaga-Falconi JS, Al Osman H, El Saddik A (2018) ECG and fingerprint bimodal authentication. Sustain Cities Soc 40:274–283. https://doi.org/10.1016/j.scs.2017.12.023

    Article  Google Scholar 

  8. Barra S, Fenu G, De Marsico M, Castiglione A, Nappi M (2018) Have you permission to answer this phone?. In: 2018 International workshop on biometrics and forensics (IWBF), pp 1–7. https://doi.org/10.1109/IWBF.2018.8401563

  9. Buriro A, Crispo B, Zhauniarovich Y (2017) Please hold on: Unobtrusive user authentication using smartphone’s built-in sensors. In: IEEE International conference on identity, security and behavior analysis (ISBA). https://doi.org/10.1109/ISBA.2017.7947684. IEEE, New Delhi India, pp 1–8

  10. Chhabria SA, Dharaskar RV, Thakare VM (2013) Survey of fusion techniques for design of efficient multimodal systems. In: International conference on machine intelligence research and advancement, pp 486–492. https://doi.org/10.1109/ICMIRA.2013.103

  11. Dhvani S, Vinayak H (2016) IoT based biometrics implementation on raspberry Pi. Procedia Comput Sci 79:328–336. https://doi.org/10.1016/j.procs.2016.03.043

    Article  Google Scholar 

  12. Dornaika F, Moujahid A, El Merabet Y, Ruichek Y (2017) A comparative study of image segmentation algorithms and descriptors for building detection. In: Handbook of neural computation, pp 591–606. https://doi.org/10.1016/B978-0-12-811318-9.00032-6

  13. Emersic Z, Struc V, Peer P (2017) Ear recognition: More than a survey. Neurocomputing 255:26–39. https://doi.org/10.1016/j.neucom.2016.08.139

    Article  Google Scholar 

  14. Gofman M, Mitra S (2016) Multimodal biometrics for enhanced mobile device security. Commun ACM 59(4):58–65. https://doi.org/10.1145/2818990

    Article  Google Scholar 

  15. Gofman M, Mitra S, Cheng K, Smith N (2015) Quality-based score-level fusion for secure and robust multimodal biometrics-based authentication on consumer mobile devices. In: ICSEA 2015 : The tenth international conference on software engineering advances, pp 274–276

  16. Gofman M, Mitra S, Smith N (2016) Hidden Markov models for feature-level fusion of biometrics on mobile devices. In: IEEE/ACS 13th International Conference of Computer Systems and Applications (AICCSA), pp 1–2. https://doi.org/10.1109/AICCSA.2016.7945755

  17. Gofman M, Sandico N, Mitra S, Suo E, Muhi S, Vu T (2018) multimodal biometrics via discriminant correlation analysis on mobile devices. In: Proceedings of the international conference on security and management (SAM). The steering committee of The World congress in computer science, Computer engineering and applied computing (WorldComp)

  18. Habib K, Arild T, Leister W (2014) A novel authentication framework based on biometric and radio fingerprinting for the IoT in eHealth. In: Proceedings of international conference on smart systems devices and technologies (SMART), pp 32–37. https://doi.org/10.13140/2.1.3944.1286

  19. Introduction to USTB ear image databases (2002) University of Science and Technology of Beijing, http://www1.ustb.edu.cn/resb/en/doc/Imagedb123introen.pdf

  20. Kandgaonkar TV, Mente RS, Shinde AR, Raut SD (2015) Ear biometrics: A survey on ear image databases and techniques for ear detection and recognition. IBMRD’s J Manag Res 4(1):88–103. https://doi.org/10.17697/ibmrd/2015/v4i1/60357

    Google Scholar 

  21. Kannala J, Rahtu E (2012) BSIf: binarized statistical image features. In: Proceedings of the International Conference on Pattern Recognition (ICPR), pp 1363–1366

  22. Kim DJ, Kil-Ram H, Kwang-Seok H (2008) An implementation of multimodal speaker verification system using teeth image and voice on mobile environment. J Inst Electron Eng Korea CI 45(5):162–172

    Google Scholar 

  23. Krizaj J, Struc V, Pavesic N (2010) Adaptation of SIFT features for robust face recognition. Int Conf Image Anal Recog, pp 394–404. https://doi.org/10.1007/978-3-642-13772-3∖_40

  24. Kumar A, Wu C (2012) Automated human identification using ear imaging. Pattern Recognit 45(3):956–968. https://doi.org/10.1016/j.patcog.2011.06.005

    Article  Google Scholar 

  25. Macek N, Franc I, Bogdanoski M, Mirkovic A (2016) Multimodal biometric authentication in IoT: Single camera case study. In: 8th International conference on business information security (BISEC’2016), pp 33–37

  26. Mahmoud RO, Selim MM, Muhi OA (2020) Fusion time reduction of a feature level based multimodal biometric authentication system. Int J Sociotechnol Knowl Dev 12(1):67–83. https://doi.org/10.4018/IJSKD.2020010104

    Article  Google Scholar 

  27. Malhotra A, Sankaran A, Mittal A, Vatsa M, Singh R (2017) Chapter 6 - Fingerphoto authentication using smartphone camera captured under varying environmental conditions. In: Human recognition in unconstrained environments, pp 119–144. https://doi.org/10.1016/B978-0-08-100705-1.00006-3

  28. Manjunath BS, Ma WY (1996) Texture features for browsing and retrieval of image data. IEEE Trans Pattern Anal Mach Intell 18(8):837–848. https://doi.org/10.1109/34.531803

    Article  Google Scholar 

  29. Mohanta TK, Mohapatra S (2014) Development of multimodal biometric framework for smartphone authentication system. Int J Comput Appl 102 (7):6–11. https://doi.org/10.5120/17825-8597

    Google Scholar 

  30. Ojala T, Pietikainen M, Harwood D (1996) A comparative study of texture measures with classification based on feature distributions. Pattern Recogn 29(1):51–59. https://doi.org/10.1016/0031-3203(95)00067-4

    Article  Google Scholar 

  31. Ojansivu V, Heikkila J (2008) Blur insensitive texture classification using local phase quantization. In: Image and Signal Processing: 3rd International Conference, ICISP 2008. https://doi.org/10.1007/978-3-540-69905-7_27. Springer, Berlin, pp 236–243

  32. Ojansivu V, Rahtu E, Heikkila J (2008) Rotation invariant local phase quantization for blur insensitive texture analysis. In: 19th International conference on pattern recognition, pp 236–243. https://doi.org/10.1109/ICPR.2008.4761377

  33. Olazabal O, Gofman M, Bai Y, Choi Y, Sandico N, Mitra S, Pham K (2019) Multimodal Biometrics for Enhanced IoT Security. In: 2019 IEEE 9th annual computing and communication workshop and conference (CCWC), pp 886–893. https://doi.org/10.1109/CCWC.2019.8666599

  34. Paul PP, Gavrilova ML, Alhajj R (2014) Decision fusion for multimodal biometrics using social network analysis. IEEE Trans Syst Man Cybern Syst 44(11):1522–1533. https://doi.org/10.1109/TSMC.2014.2331920

    Article  Google Scholar 

  35. Pflug A, Paul PN, Busch C (2014) A comparative study on texture and surface descriptors for ear biometrics. In: 2014 International carnahan conference on security technology (ICCST), pp 1–8. https://doi.org/10.1109/BTAS.2014.6996239

  36. Pflug A, Busch C, Ross A (2014) 2D ear classification based on unsupervised clustering. In: Proceedings of the international joint conference on biometrics, pp 1–8. https://doi.org/10.1109/BTAS.2014.6996239

  37. Rodrigues RN, Lee LL, Venu G (2009) Robustness of multimodal biometric fusion methods against spoof attacks. J Vis Lang Comput 20(3):169–179. https://doi.org/10.1016/j.jvlc.2009.01.010

    Article  Google Scholar 

  38. Rodrigues RN, Niranjan K, Venu G (2010) Evaluation of biometric spoofing in a multimodal system. In: 2010 Fourth IEEE international conference on biometrics: Theory applications and systems (BTAS). https://doi.org/10.1109/BTAS.2010.5634531

  39. Sequeira AF, Monteiro JC, Rebelo A, Oliveira HP (2014) MobBIO: A multimodal database captured with a portable handheld device. In: 2014 International conference on computer vision theory and applications (VISAPP), vol 3

  40. Sitova Z, Sedenka J, Yang Q, Peng G, Zhou G, Gasti P, Balagani KS (2016) HMOG: New behavioral biometric features for continuous authentication of smartphone users. IEEE Trans Inf Forensic Secur 11 (5):877–892. https://doi.org/10.1109/TIFS.2015.2506542

    Article  Google Scholar 

  41. Vu NS, Caplier A (2010) Face recognition with patterns of oriented edge magnitudes. Comp Vision pp 313–326. https://doi.org/10.1007/978-3-642-15549-9∖_23

  42. Wu L, Yang J, Zhou M, Chen Y, Wang Q (2020) LVID: A multimodal biometrics authenticationsystem on smartphones. IEEE Trans Inf Forensic Secur 15:1–6. https://doi.org/10.1109/TIFS.2019.2944058

    Google Scholar 

  43. Zhang Q, Li H, Sun Z, Tan T (2018) Deep feature fusion for iris and periocular biometrics on mobile devices. IEEE Trans Inf Forensic Secur 13(11):2897–2912. https://doi.org/10.1109/TIFS.2018.2833033

    Article  Google Scholar 

  44. Zhang Y, Mu Z, Yuan L, Yu C (2018) Ear verification under uncontrolled conditions with convolutional neural networks. IET Biometrics 7(3):185–198. https://doi.org/10.1049/iet-bmt.2017.0176

    Article  Google Scholar 

Download references

Acknowledgements

This work was carried out in the framework of research activities of the laboratory LIMED, which is affiliated to the Faculty of Exact Sciences of the University of Bejaia, with collaboration with LIGM, University of Gustave-Eiffel.

Funding

This research did not receive any specific grant from funding agencies in the public, commercial, or not-for-profit sectors.

Author information

Authors and Affiliations

  1. Laboratoire d’Informatique Médicale (LIMED), Faculté des Sciences Exactes Université de Bejaia, 06000, Bejaia, Algeria

    Feriel Cherifi & Kamal Amroun

  2. LIGM, ESIEE Paris, Université Gustave-Eiffel, Noisy-le-Grand, France

    Mawloud Omar

  3. Laboratoire de Modélisation et Optimisation des Systèmes, Faculté des Sciences Exactes Université de Bejaia, 06000, Bejaia, Algeria

    Mawloud Omar

Authors
  1. Feriel Cherifi
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kamal Amroun
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mawloud Omar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Feriel Cherifi.

Ethics declarations

Conflict of interests

The authors declare that they have no conflict of interest.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cherifi, F., Amroun, K. & Omar, M. Robust multimodal biometric authentication on IoT device through ear shape and arm gesture. Multimed Tools Appl 80, 14807–14827 (2021). https://doi.org/10.1007/s11042-021-10524-9

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11042-021-10524-9

Keywords

Search

Navigation