Skip to main content
SpringerLink
Log in

Evaluation of accurate iris center and eye corner localization method in a facial image for gaze estimation

  • Regular Paper
  • Published:
Multimedia Systems Aims and scope Submit manuscript

Abstract

Accurate estimation of eye-related information is important for many applications such as gaze estimation, face alignment, driver drowsiness detection, etc. Earlier works fail to estimate eye information in low-resolution images captured by a regular camera or webcam. This paper is aimed at developing an Iris Center (IC) and Eye Corner (EC) localization method in low-resolution facial images with an application of gaze estimation. A three-stage method is proposed for IC and EC localization. In the first stage, a circular gradient-intensity-based operator is proposed for rough ICs estimation and a CNN model is used in the second stage to find true ICs. In the third stage, Explicit Shape Regression (ESR) method is used for EC localization where initialization is done taking the ICs as a reference point to the mean eye contour shape model. The proposed IC localization method is evaluated on BioID and Gi4E database and it shows better accuracy compare to some of the state-of-the-art methods. This method further evaluated for gaze estimation based on IC and EC which does not require any prior calibrations unlike earlier infrared illumination-based gaze trackers. Here, the experiment for gaze estimation is performed in our proposed NITSGoP database that prepared under indoor conditions with complex background and uneven illuminations. The experimental results suggest that the proposed method can be used for gaze estimation with better accuracy both in still images and videos.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig.1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15
Fig. 16
Fig. 17
Fig. 18

Similar content being viewed by others

References

  1. Drewes, H. (2014). Eye Gaze Tracking. Interactive Displays: Natural Human Interface Technologies, 251–283. https://doi.org/10.1002/9781118706237.ch8

  2. Oliveira, L. S., Borges, D. L., Vidal, F. B., and Chang, L (2012). A fast eye localization and verification method to improve face matching in surveillance videos. IEEE International Conference on Systems, Man, and Cybernetics (SMC); pp. 840–845

  3. Majaranta, P., Bulling, A.: Eye tracking and eye-based human–computer interaction. In: Advances in physiological computing, pp. 39–65. Springer, London (2014)

    Chapter  Google Scholar 

  4. Duchowski, A.T.: Eye tracking methodology. Theory Practice 328(614), 2–3 (2007)

    Google Scholar 

  5. Timm, F., Barth, E.: Accurate eye centre localisation by means of gradients. Visapp 11, 125–130 (2011)

    Google Scholar 

  6. Kim, B. S., Lee, H., and Kim, W. Y.: Rapid eye detection method for non-glasses type 3D display on portable devices. IEEE Transactions on consumer electronics, vol. 56, no. 4, pp. 2498-2505 (2010). https://doi.org/10.1109/TCE.2010.5681133

  7. Kim, H., Jo, J., Toh, K.A., Kim, J.: Eye detection in a facial image under pose variation based on multi-scale iris shape feature. Image Vis. Comput. 57, 147–164 (2017)

    Article  Google Scholar 

  8. Cootes, T.F., Edwards, G.J., Taylor, C.J.: Active appearance models. IEEE Trans. Pattern Anal. Mach. Intell. 23(6), 681–685 (2001)

    Article  Google Scholar 

  9. Cristinacce, D., Cootes, T.F.: Feature detection and tracking with constrained local models. Bmvc 1(2), 3 (2006)

    MATH  Google Scholar 

  10. Xiong, X., and De la Torre, F. (2013). Supervised descent method and its applications to face alignment. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 532–539).

  11. Cao, X., Wei, Y., Wen, F., Sun, J.: Face alignment by explicit shape regression. Int. J. Comput. Vis. 107(2), 177–190 (2014)

    Article  MathSciNet  Google Scholar 

  12. Song, F., Tan, X., Chen, S., Zhou, Z.H.: A literature survey on robust and efficient eye localization in real-life scenarios. Pattern Recogn. 46(12), 3157–3173 (2013)

    Article  Google Scholar 

  13. Alonso-Fernandez, F., Bigun, J.: A survey on periocular biometrics research. Pattern Recogn. Lett. 82, 92–105 (2016)

    Article  Google Scholar 

  14. Jing, M.Q., Chen, L.H.: A novel method for horizontal eye line detection under various environments. Int. J. Pattern Recogn. Artif. Intell. 24(03), 475–498 (2010)

    Article  Google Scholar 

  15. San, N.N., Aye, N.: Performance evaluation of eye detection system using WSAPG. IJCCER 2(2), 52–56 (2014)

    Google Scholar 

  16. Skodras, E., Fakotakis, N.: Precise localization of eye centers in low-resolution color images. Image Vis. Comput. 36, 51–60 (2015)

    Article  Google Scholar 

  17. Hassaballah, M., Kanazawa, T., Ido, S.: Efficient eye detection method based on grey intensity variance and independent components analysis. IET Comput. Vision 4(4), 261–271 (2010)

    Article  Google Scholar 

  18. Valenti, R., Gevers, T.: Accurate eye center location through invariant isocentric patterns. IEEE Trans. Pattern Anal. Mach. Intell. 34(9), 1785–1798 (2012)

    Article  Google Scholar 

  19. Viola, P., and Jones, M. (2001). Rapid object detection using a boosted cascade of simple features, Computer Vision and Pattern Recognition, Proceedings of the 2001 IEEE Computer Society Conference, Vol. 1, pp. I-I.

  20. Chen, S., Liu, C.: Eye detection using discriminatory Haar features and a new efficient SVM. Image Vis. Comput. 33, 68–77 (2015)

    Article  Google Scholar 

  21. Kroon, B., Maas, S., Boughorbel, S., Hanjalic, A.: Eye localization in low and standard definition content with application to face matching. Comput. Vis. Image Underst. 113(8), 921–933 (2009)

    Article  Google Scholar 

  22. Savakis, A., Sharma, R., and Kumar, M. (2014). Efficient eye detection using HOG-PCA descriptor. In: Proceedings SPIE 9027, imaging and multimedia analytics in a web and mobile world 2014, 90270J. https://doi.org/10.1117/12.2036824

  23. Tan, X., Song, F., Zhou, Z. H., and Chen, S. (2009). Enhanced pictorial structures for precise eye localization under uncontrolled conditions. Computer Vision and Pattern Recognition, IEEE Conference, pp. 1621–1628.

  24. Monzo, D., Albiol, A., Sastre, J., Albiol, A.: Precise eye localization using HOG descriptors. Mach. Vis. Appl. 22(3), 471–480 (2011)

    Google Scholar 

  25. Ito, Y., Ohyama, W., Wakabayashi, T., and Kimura, F. (2012). Detection of eyes by circular Hough transform and histogram of gradient, In Pattern Recognition (ICPR), 21st IEEE International Conference, pp. 1795–1798.

  26. Zhu, J., and Yang, J. (2002). Subpixel eye gaze tracking. In Proceedings of Fifth IEEE International Conference on Automatic Face Gesture Recognition (pp. 131–136).

  27. Zhou, R., He, Q., Wu, J., Hu, C., Meng, Q.H.: Inner and outer eye corners detection for facial features extraction based on ctgf algorithm. Appl Mech Mater 58, 1966–1971 (2011). (Trans Tech Publications)

    Article  Google Scholar 

  28. Xia, H., Yan, G.: A novel method for eye corner detection based on weighted variance projection function. 2nd International congress on image and signal processing, Tianjin, pp. 1–4. (2009). https://doi.org/10.1109/CISP.2009.5304434

  29. Bengoechea, J.J., Cerrolaza, J.J., Villanueva, A., Cabeza, R.: Evaluation of accurate eye corner detection methods for gaze estimation. J. Eye Mov Res. 7(3), 1–8 (2014)

    Google Scholar 

  30. Jain, A.K.: Fundamentals of digital image processing. Prentice Hall, Englewood Cliffs, NJ (1989)

    MATH  Google Scholar 

  31. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)

    Article  Google Scholar 

  32. Krizhevsky, A., Sutskever, I., Hinton, G. E.: ImageNet classification with deep convolutional neural networks. Communications of the ACM 60(6), 84–90 (2017). https://doi.org/10.1145/3065386

  33. Jariwala, K.N., Nandi, A., Dalal, U.D.: A real-time robust eye center localization using geometric eye model and edge gradients in unconstrained visual environment. Int. J. Comput. Appl. 128(1), 22–27 (2015)

    Google Scholar 

  34. Ross, A., and Govindarajan, R. (2004). Feature level fusion in biometric systems, In Proceedings of Biometric Consortium Conference (BCC); pp. 1–2.

  35. Hao, Y., Zhu, H., Wu, K., Lin, X., Ma, L.: Salient-points-guided face alignment. In: Multimedia systems 25, 475–485. https://doi.org/10.1007/s00530-017-0555-8.

  36. Kazemi, V., and Sullivan, J. (2014). One millisecond face alignment with an ensemble of regression trees. In Proceedings of the IEEE conference on computer vision and pattern recognition (pp. 1867–1874).

  37. Jesorsky, O., Kirchberg, K.J., Frischholz, R.W.: Robust face detection using the Hausdorff distance. In: International Conference on Audio-and Video-based Biometric Person Authentication, pp. 90–95. Springer, Berlin, Heidelberg (2001)

    Chapter  Google Scholar 

  38. BioID Technology Research, the BioID Face Database, https://ftp.uni-erlangen.de/pub/facedb/, 2001.

  39. Ponz, V., Villanueva, A., and Cabeza, R. (2012). Dataset for the evaluation of eye detector for gaze estimation, In Proceedings of the 2012 ACM Conference on Ubiquitous Computing, pp. 681–684.

  40. Asadifard, M., Shanbezadeh, J.: Automatic adaptive center of pupil detection using face detection and CDF analysis. Proc. Int. Multiconf. Eng. Comput. Sci. 1, 3 (2010)

    Google Scholar 

  41. Hamouz, M., Kittler, J., Kamarainen, J.K., Paalanen, P., Kalviainen, H., Matas, J.: Feature-based affine-invariant localization of faces. IEEE Trans. Pattern Anal. Mach. Intell. 27(9), 1490–1495 (2005)

    Article  Google Scholar 

  42. Ren, Y., Wang, S., Hou, B., Ma, J.: A novel eye localization method with rotation invariance. IEEE Trans. Image Process. 23(1), 226–239 (2014)

    Article  MathSciNet  Google Scholar 

  43. George, A., Routray, A.: Fast and accurate algorithm for eye localization for gaze tracking in low-resolution images. IET Comput. Vis. 10(7), 660–669 (2016)

    Article  Google Scholar 

  44. Leo, M., Cazzato, D., De Marco, T., Distante, C.: Unsupervised eye pupil localization through differential geometry and local self-similarity matching. PLoS ONE 9(8), e102829 (2014)

    Article  Google Scholar 

  45. Baek, S.J., Choi, K.A., Ma, C., Kim, Y.H., Ko, S.J.: Eyeball model-based iris center localization for visible image-based eye-gaze tracking systems. IEEE Trans. Consum. Electron. 59(2), 415–421 (2013)

    Article  Google Scholar 

  46. Daugman, J.: How iris recognition works. The Essential Guide to Image Processing (Second Edition), 715–739 (2009). https://doi.org/10.1016/B978-0-12-374457-9.00025-1

  47. Wang, J., Sung, A., and Venkateswarlu, R. (2003). Eye gaze estimation from a single image of one eye, In Computer Vision, Proceedings, Ninth IEEE International Conference; pp. 136–143.

  48. Martinez, A. M. (1998). The AR face database. CVC Technical Report24.

  49. Gross, R., Matthews, I., Cohn, J., Kanade, T., Baker, S.: Multi-pie. Image Vis. Comput. 28(5), 807–813 (2010)

    Article  Google Scholar 

  50. Laddi, A., Prakash, N.R.: An augmented image gradients based supervised regression technique for iris center localization. Multimed. Tools Appl. 76(5), 7129–7139 (2017)

    Article  Google Scholar 

  51. Ahmed, M., Laskar, R.H.: Eye center localization in a facial image based on geometric shapes of iris and eyelid under natural Variability. Image Vis. Comput. (2019). https://doi.org/10.1016/j.imavis.2019.05.002

    Article  Google Scholar 

  52. Laddi, A., Prakash, N.R.: Eye gaze tracking based directional control interface for interactive applications. Multimed. Tools Appl. 78(22), 31215–31230 (2019)

    Article  Google Scholar 

  53. Kim, H.I., Kim, J.B., Park, R.H.: Efficient and fast iris localization using binary radial gradient features for human–computer interaction. Int. J. Pattern Recogn. Artif. Intell. 31(11), 1756015 (2017)

    Article  Google Scholar 

  54. Ahmed, M., Ahmed, R., Thakuria, A. J., and Laskar, R. H. (2019). Eye center guided constrained local model for landmark localization in facial image," 9th Annual Information Technology, Electromechanical Engineering and Microelectronics Conference (IEMECON), Jaipur, India, 2019, pp. 168-173, https://doi.org/10.1109/IEMECONX.2019.8877093

  55. Ahmed, M., Laskar, R. H., and Laskar, R. H. ( 2019). Analyzing the effect of eye center localization on accurate landmark localization in a facial image," International Conference on Automation, Computational and Technology Management (ICACTM), London, United Kingdom, 2019, pp. 544–549, https://doi.org/10.1109/ICACTM.2019.8776719

  56. Larrazabal, A.J., Cena, C.G., Martínez, C.E.: Video-oculography eye tracking towards clinical applications: a review. Comput. Biol. Med. 108, 57–66 (2019)

    Article  Google Scholar 

  57. Xia, Y., Lou, J., Dong, J., Qi, L., Li, G., Yu, H.: Hybrid regression and isophote curvature for accurate eye center localization. Multimed. Tools Appl. 79(1), 805–824 (2020)

    Article  Google Scholar 

  58. Dai, L., Liu, J., Ju, Z., Gao, Y.: Iris center localization using energy map with image inpaint technology and post-processing correction. IEEE Access 8, 16965–16978 (2020)

    Article  Google Scholar 

  59. Ahmed, N. Y.: Real-time accurate eye center localization for low-resolution grayscale images. J Real-Time Image Proc 1–28. https://doi.org/10.1007/s11554-020-00955-2

  60. Choi, J.H., Lee, K.I., Song, B.C.: Eye pupil localization algorithm using convolutional neural networks. Multimed. Tools Appl. 79(43), 32563–32574 (2020)

    Article  Google Scholar 

  61. Levinshtein, A., Phung, E., Aarabi, P.: Hybrid eye center localization using cascaded regression and hand-crafted model fitting. Image Vis. Comput. 71, 17–24 (2018)

    Article  Google Scholar 

  62. Abbasi, M., Khosravi, M.R.: A robust and accurate particle filter-based pupil detection method for big datasets of eye video. J. Grid Comput. 18(2), 305–325 (2020)

    Article  Google Scholar 

Download references

Acknowledgments

This research work has been carried out in the Speech and Image Processing Lab, NIT Silchar, India, and is supported by Visvesvaraya Ph.D. Scheme of MeitY, Government of India (Ref No. PhD-MLA/4(74)/2015-16). Moreover, the authors would like to thank all subjects for participation in collecting the NITSGoP database.

Author information

Authors and Affiliations

  1. Department of Electronics and Communication Engineering, National Institute of Technology Silchar, Assam, 788010, India

    Manir Ahmed & Rabul Hussain Laskar

Authors
  1. Manir Ahmed
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rabul Hussain Laskar
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Manir Ahmed.

Additional information

Communicated by Manir Ahmed.

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Ahmed, M., Laskar, R.H. Evaluation of accurate iris center and eye corner localization method in a facial image for gaze estimation. Multimedia Systems 27, 429–448 (2021). https://doi.org/10.1007/s00530-020-00744-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00530-020-00744-8

Keywords

Search

Navigation