Skip to main content
SpringerLink
Log in

Face Localization and Enhancement

  • Chapter
  • First Online:
Human Centric Visual Analysis with Deep Learning

Abstract

Facial landmark localization plays a critical role in facial recognition and analysis. In this chapter, we first propose a novel cascaded backbone-branches fully convolutional neural network (BB-FCN) for rapidly and accurately localizing facial landmarks in unconstrained and cluttered settings. The proposed BB-FCN generates facial landmark response maps directly from raw images without any preprocessing. It follows a coarse-to-fine cascaded pipeline, which consists of a backbone network for roughly detecting the locations of all facial landmarks and one branch network for each type of detected landmark to further refine their locations (©[2019] IEEE. Reprinted, with permission, from [1].). At the end of this chapter, we also introduce the progress in face hallucination, a fundamental problem in the face analysis field that refers to generating a high-resolution facial image from a low-resolution input image (©[2019] IEEE. Reprinted, with permission, from [2].).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 119.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Hardcover Book
USD 159.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. L. Liu, G. Li, Y. Xie, Y. Yu, Q. Wang, L. Lin, Facial landmark machines: a backbone-branches architecture with progressive representation learning. IEEE Trans. Multimedia. https://doi.org/10.1109/TMM.2019.2902096

    Article  Google Scholar 

  2. Q. Cao, L. Lin, Y. Shi, X. Liang, G. Li, Attention-aware face hallucination via deep reinforcement learning, in 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu, HI, pp. 1656–1664 (2017). https://doi.org/10.1109/CVPR.2017.180

  3. P. Luo, X. Wang, X. Tang, A deep sum-product architecture for robust facial attributes analysis, in ICCV, pp. 2864–2871 (2013)

    Google Scholar 

  4. C. Lu, X. Tang, Surpassing human-level face verification performance on lfw with gaussianface, in AAAI (2015)

    Google Scholar 

  5. L. Liu, C. Xiong, H. Zhang, Z. Niu, M. Wang, S. Yan, Deep aging face verification with large gaps. TMM 18(1), 64–75 (2016)

    Google Scholar 

  6. Z. Zhu, P. Luo, X. Wang, X. Tang, Deep learning identity-preserving face space, in ICCV, pp. 113–120 (2013)

    Google Scholar 

  7. C. Ding, D. Tao, Robust face recognition via multimodal deep face representation. TMM 17(11), 2049–2058 (2015)

    Google Scholar 

  8. Y. Li, L. Liu, L. Lin, Q. Wang, Face recognition by coarse-to-fine landmark regression with application to atm surveillance, in CCCV (Springer, 2017), pp. 62–73

    Google Scholar 

  9. P. Viola, M. Jones, Rapid object detection using a boosted cascade of simple features, in CVPR, vol. 1. IEEE, pp. I–511 (2001)

    Google Scholar 

  10. X. Zhu, D. Ramanan, Face detection, pose estimation, and landmark localization in the wild, in CVPR (IEEE, 2012), pp. 2879–2886

    Google Scholar 

  11. Z. Yan, H. Zhang, R. Piramuthu, V. Jagadeesh, D. DeCoste, W. Di, Y. Yu, Hd-cnn: hierarchical deep convolutional neural networks for large scale visual recognition, in ICCV, pp. 2740–2748 (2015)

    Google Scholar 

  12. M. Köstinger, P. Wohlhart, P.M. Roth, H. Bischof, Annotated facial landmarks in the wild: a large-scale, real-world database for facial landmark localization, in ICCV Workshops (IEEE, 2011), pp. 2144–2151

    Google Scholar 

  13. Z. Zhang, P. Luo, C.C. Loy, X. Tang, Facial landmark detection by deep multi-task learning, in ECCV (Springer, 2014), pp. 94–108

    Google Scholar 

  14. X. Burgos-Artizzu, P. Perona, P. Dollár, Robust face landmark estimation under occlusion, in ICCV, pp. 1513–1520 (2013)

    Google Scholar 

  15. X. Cao, Y. Wei, F. Wen, J. Sun, Face alignment by explicit shape regression. IJCV 107(2), 177–190 (2014)

    Article  MathSciNet  Google Scholar 

  16. X. Yu, J. Huang, S. Zhang, W. Yan, D. Metaxas, Pose-free facial landmark fitting via optimized part mixtures and cascaded deformable shape model, in ICCV, pp. 1944–1951 (2013)

    Google Scholar 

  17. X. Xiong, F. Torre, Supervised descent method and its applications to face alignment, in CVPR, pp. 532–539 (2013)

    Google Scholar 

  18. K. Zhang, Z. Zhang, Z. Li, Y. Qiao, Joint face detection and alignment using multitask cascaded convolutional networks. IEEE Signal Process. Lett. 23(10), 1499–1503 (2016)

    Article  Google Scholar 

  19. S. Xiao, J. Feng, J. Xing, H. Lai, S. Yan, A. Kassim, Robust facial landmark detection via recurrent attentive-refinement networks, in ECCV (Springer, 2016), pp. 57–72

    Google Scholar 

  20. Z. Liu, P. Luo, X. Wang, X. Tang, Deep learning face attributes in the wild, in ICCV, pp. 3730–3738 (2015)

    Google Scholar 

  21. Z. Zhang, P. Luo, C.C. Loy, X. Tang, Learning deep representation for face alignment with auxiliary attributes. IEEE Trans. Pattern Anal. Mach. Intell. 38(5), 918–930 (2016)

    Article  Google Scholar 

  22. E. Zhou, Z. Cao, Q. Yin, Naive-deep face recognition: touching the limit of lfw benchmark or not? arXiv preprint arXiv:1501.04690 (2015)

  23. E. Zhou, H. Fan, Z. Cao, Y. Jiang, Q. Yin, Learning face hallucination in the wild, in AAAI, pp. 3871–3877 (2015)

    Google Scholar 

  24. S. Zhu, S. Liu, C.C. Loy, X. Tang, Deep cascaded bi-network for face hallucination. arXiv preprint arXiv:1607.05046 (2016)

  25. C. Liu, H.-Y. Shum, W.T. Freeman, Face hallucination: theory and practice. Int. J. Comput. Vis. 75(1), 115–134 (2007)

    Article  Google Scholar 

  26. C. Dong, C.C. Loy, K. He, X. Tang, Learning a deep convolutional network for image super-resolution, in ECCV, pp. 184–199 (2014)

    Chapter  Google Scholar 

  27. J. Najemnik, W.S. Geisler, Optimal eye movement strategies in visual search. Nature 434(7031), 387–391 (2005)

    Article  Google Scholar 

  28. Y. Sun, D. Liang, X. Wang, X. Tang, Deepid3: face recognition with very deep neural networks. arXiv preprint arXiv:1502.00873 (2015)

  29. J.C. Caicedo, S. Lazebnik, Active object localization with deep reinforcement learning, in ICCV, pp. 2488–2496 (2015)

    Google Scholar 

  30. K. Gregor, I. Danihelka, A. Graves, D.J. Rezende, D. Wierstra, DRAW: a recurrent neural network for image generation, in ICLR, pp. 1462–1471 (2015)

    Google Scholar 

  31. D. Silver, A. Huang, C.J. Maddison, A. Guez, L. Sifre et al., Mastering the game of go with deep neural networks and tree search. Nature 529, 484–503 (2016)

    Article  Google Scholar 

  32. J. Kim, J.K. Lee, K.M. Lee, Accurate image super-resolution using very deep convolutional networks (2016)

    Google Scholar 

  33. O. Tuzel, Y. Taguchi, J.R. Hershey, Global-local face upsampling network. arXiv preprint arXiv:1603.07235 (2016)

  34. S. Gu, W. Zuo, Q. Xie, D. Meng, X. Feng, L. Zhang, Convolutional sparse coding for image super-resolution, in ICCV, pp. 1823–1831 (2015)

    Google Scholar 

  35. R.J. Williams, Simple statistical gradient-following algorithms for connectionist reinforcement learning. Mach. Learn. 8(3), 229–256 (1992)

    MathSciNet  MATH  Google Scholar 

  36. V. Mnih, N. Heess, A. Graves, K. kavukcuoglu, Recurrent models of visual attention, in NIPS, pp. 2204–2212 (2014)

    Google Scholar 

  37. O. Jesorsky, K.J. Kirchberg, R. Frischholz, Robust face detection using the hausdorff distance, in AVBPA, pp. 90–95 (2001)

    Google Scholar 

  38. G.B. Huang, M. Ramesh, T. Berg, E. Learned-Miller, Labeled faces in the wild: A database for studying face recognition in unconstrained environments. Technical Report 07-49, University of Massachusetts, Amherst, October 2007

    Google Scholar 

  39. G.B. Huang, V. Jain, E. Learned-Miller, Unsupervised joint alignment of complex images, in ICCV (2007)

    Google Scholar 

  40. C.-Y. Yang, S. Liu, M.-H. Yang, Structured face hallucination, in CVPR, pp. 1099–1106 (2013)

    Google Scholar 

  41. X. Ma, J. Zhang, C. Qi, Hallucinating face by position-patch. Pattern Recogn. 43(6), 2224–2236 (2010)

    Article  Google Scholar 

  42. D.P. Kingma, J. Ba. Adam: amethod for stochastic optimization, in ICLR (2015)

    Google Scholar 

  43. T. Chen, L. Lin, L. Liu, X. Luo, X. Li, Disc: deep image saliency computing via progressive representation learning. TNNLS 27(6), 1135–1149 (2016)

    MathSciNet  Google Scholar 

  44. L. Liu, H. Wang, G. Li, W. Ouyang, L. Lin, Crowd counting using deep recurrent spatial-aware network, in IJCAI (2018)

    Google Scholar 

  45. L. Liu, R. Zhang, J. Peng, G. Li, B. Du, L. Lin, Attentive crowd flow machines, in ACM MM (ACM, 2018), pp. 1553–1561

    Google Scholar 

  46. Y. Sun, X. Wang, X. Tang, Deep convolutional network cascade for facial point detection, in CVPR, pp. 3476–3483 (2013)

    Google Scholar 

  47. R. Weng, J. Lu, Y.-P. Tan, J. Zhou, Learning cascaded deep auto-encoder networks for face alignment. TMM 18(10), 2066–2078 (2016)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. School of Data and Computer Science, Sun Yat-sen University, Guangzhou, Guangdong, China

    Liang Lin & Dongyu Zhang

  2. School of Information Engineering, The Chinese University of Hong Kong, Hong Kong, Hong Kong

    Ping Luo

  3. School of Computer Science, Harbin Institute of Technology, Harbin, China

    Wangmeng Zuo

Authors
  1. Liang Lin
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dongyu Zhang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Ping Luo
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Wangmeng Zuo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Liang Lin .

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Singapore Pte Ltd.

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Lin, L., Zhang, D., Luo, P., Zuo, W. (2020). Face Localization and Enhancement. In: Human Centric Visual Analysis with Deep Learning. Springer, Singapore. https://doi.org/10.1007/978-981-13-2387-4_3

Download citation

  • DOI: https://doi.org/10.1007/978-981-13-2387-4_3

  • Published:

  • Publisher Name: Springer, Singapore

  • Print ISBN: 978-981-13-2386-7

  • Online ISBN: 978-981-13-2387-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation