Abstract
Many computer vision researches aim at building automatic systems that can analyze a broad range of visual concepts. In this chapter we are interested in those concepts that are related to human social attributes. The chapter reports an intelligent systems that can first identify face from social media, then recognize the age, gender, and beauty score attributes. Most importantly, the system can automatically edit the facial image to get it younger and more attractive, and hence making the social media sharing experience more pleasant. We applied deep learning and multitask learning techniques to train a convolutional neural network model for the above recognition task, and applied a gradient-based method to generate the editing information that can beautify the facial image. Our work goes beyond previous works in several aspects: (1) The fully automatic system does not require costly manual annotation of landmark facial features but simply takes the raw pixels as inputs for recognition; (2) The constructed neural network model can be used to both beautify and beastify the facial image, with the amount of modification controllable by a single regularization parameter; (3) we imposed no restrictions in terms of pose, lighting, background, expression, age, and ethnicity on the images used for training and testing; and (4) we further developed a visualization method to interpret the learned model and revealed the existence of several features related to these human attributes.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
References
Shewaye, T.N.: Age group and gender recognition from human facial images. In: IEEE CVPR (2012)
Yang, M.-H., Kriegman, D., Ahuja, N.: Detecting faces in images: a survey. In: IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 34–58 (2002)
Zhang, C., Zhang, Z.: Survey of recent advances in face detection. Microsoft Technical Report MSR-TR-2010-66 (2010)
Chen, Y., Han, M., Song, K., Ho, Y.: Image-based age-group classification design using facial features. In: System Science and Engineering (ICSSE), pp. 548–522 (2010)
Asuman, G., Nabiyev, V.: Automatic age classification with LBP. In: Computer and Information Sciences (ISCIS), pp. 1–4 (2008)
Wang, J., Yau, W., Wang, H.: Age categorization via ECOC with fused gabor and LBP features. In: Workshop Applications of Computer Vision (WACV), pp. 1–6 (2009)
Kwon, Y., Lobo, N.: Age classification from facial images. J. Comput. Vis. Image Underst. 74(1), 1–21 (1999)
Iga, R., Izumi, K., Hayashi, H., Fukano, G., Ohtani, T.: A gender and age estimation system from face images. In: Proceeding of SICE 2003 Annual Conference, pp. 332–339 (2003)
Guo, D., Sim, T.: Digital face makeup by example. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition (2009)
Bui, L., Tran, D., Huang, X., Chetty, G.: Face gender recognition based on 2D principal component analysis and support vector machine. In: 4th International Conference on Network and System Security (NSS), pp. 579–582 (2010)
Lu, H., Lin, H.: Gender recognition using adaboosted feature. In: Third International Conference on Natural Computation, vol. 2, pp. 646–650 (2007)
Brunelli, R., Poggio, T.: HyperBF networks for gender classification. In: Proceedings of the DARPA Image Understanding Workshop, pp. 311–314 (1992)
Lian, H., Lu, B.: Multi-view gender classification using local binary patterns and support vector machines. In: Advances in Neural Networks (2006)
Sun, Z., Bebis, G., Yuan, X., Louis, S.: Genetic feature extraction for gender classification: a comparison study. In: IEEE Workshop on Applications of Computer Vision (2002)
Ku, C., Ernst, A.: Face detection and tracking in video sequences using the modified census transformation. Image Vis. Comput. 24, 564–572 (2006)
Cross, J., Cross, J.: Age, sex, race, and the perception of facial beauty. Dev. Psychol. 5, 433–439 (1971)
Aarabi, P., Hughes, D., Mohajer, K., Emami, M.: The automatic measurement of facial beauty. In: IEEE International Conference on Systems, Man, and Cybernetics, vol. 4 (2001)
Alley, T., Cunningham, M.: Averaged faces are attractive, but very attractive faces are not average. Psychol. Sci. 2, 123–125 (1991)
Grammer, K., Thornhill, R.: Human (Homo sapiens) facial attractiveness and sexual selection: the role of symmetry and averageness. J. Comp. Psychol. 108, 233–242 (1994)
Ou, Y., Wu, X., Qian, H., Xu, Y.: A real time race classification system. In: IEEE International Conference on Information Acquisition, pp. 378–383 (2005)
Lin, H., Lu, H., Zhang, L.: A new automatic recognition system of gender, age and ethnicity. In: Sixth World Congress on Intelligent Control and Automation, vol. 2, pp. 9988–9991 (2002)
Hosoi, S., Takikawa, E., Kawade, M.: Ethnicity estimation with facial images. In: Sixth IEEE International Conference on Automatic Face and Gesture Recognition, pp. 195–200 (2004)
Ng, C., Tay, Y., Goi, B.: Vision-based human gender recognition: a survey. CoRR (2012), abs/1204.1611
Gray, D., Yu, K., Xu, W., Gong, Y.: Predicting facial beauty without landmarks. In: European Conference on Computer Vision (ECCV) (2010)
LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(11), 2278–2324 (1998)
Fu, Y., Guo, G., Huang, T.S.: Age synthesis and estimation via faces: a survey. In: IEEE Transaction on PAMI (2010)
Gutta, S., Huang, J., Phillips, P.J., Wechsler, H.: Mixture of experts for classification of gender, ethnic origin, and pose of human faces. In: IEEE Transaction on Neural Networks. Learning System (2000)
Lin, C., Wang, S.: Fuzzy support vector machines. IEEE Trans. Neural Netw. 13(2), 464–471 (2002)
Moghaddam, B., Yang, M.: Learning gender with support faces. IEEE Trans. Pattern Anal. Mach. Intell. 24(5), 707–711 (2002)
Zhou, Y., Gu, L., Zhang, H.: Bayesian tangent shape model: estimating shape and pose parameters via Bayesian inference. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1 (2003)
Eisenthal, Y., Dror, G., Ruppin, E.: Facial attractiveness: beauty and the machine. Neural Comput. 18(1), 119–142 (2006)
Kagian, A., Dror, G., Leyvand, T., Cohen-Or, D., Ruppin, E.: A humanlike predictor of facial attractiveness. In: Advances in Neural Information Processing Systems, pp. 649–656 (2005)
Likert, R.: Technique for the measurement of attitudes. Arch. Psychol. 22, 55 (1932)
Oliva, A., Torralba, A.: Modeling the shape of the scene: a holistic representation of the spatial envelope. Int. J. Comput. Vis. 42, 145–175 (2001)
Chopra, S., Hadsell, R., LeCun, Y.: Learning a similarity metric discriminatively, with application to face verification. In: IEEE Computer Society Conference on Computer Vision and Pattern Recognition, vol. 1 (2005)
Fukushima, K.: Neocognitron: a hierarchical neural network capable of visual pattern recognition. Neural Netw. 1, 119–130 (1988)
Hubel, D., Wiesel, T.: Receptive fields, binocular interaction and functional architecture in the cats visual cortex. J. Physiol. 160, 106–154 (1962)
Huang, G., Jain, V., Amherst, M., Learned-Miller, E.: Unsupervised joint alignment of complex images. In: IEEE International Conference on Computer Vision (2007)
Gunes, H., Piccardi, M., Jan, T.: Comparative beauty classification for pre-surgery planning. In: IEEE International Conference on Systems, Man and Cybernetics, vol. 3 (2004)
Joy, K., Primeaux, D.: A comparison of two contributive analysis methods applied to an ANN modeling facial attractiveness. In: International Conference on Software Engineering Research, Management and Applications, pp. 82–86 (2006)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Wang, J., Gong, Y., Gray, D. (2014). Female Facial Beauty Attribute Recognition and Editing. In: Fu, Y. (eds) Human-Centered Social Media Analytics. Springer, Cham. https://doi.org/10.1007/978-3-319-05491-9_7
Download citation
DOI: https://doi.org/10.1007/978-3-319-05491-9_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-05490-2
Online ISBN: 978-3-319-05491-9
eBook Packages: Computer ScienceComputer Science (R0)