Abstract
Facial expression recognition is a process of identifying the different facial expressions of the individuals to categorize the mental health of the individual. This system is used in most of the fields but is vastly used in the medical field to identify the mental health issues. In this paper, a novel approach has been proposed to identify the facial expressions of the individuals and categorizing it into seven different emotions. Initially, the images collected from the dataset are subjected to pre-processing for de-noising. Then, the major geometric and appearance-based features are extracted from the images. The most relevant features are selected from the extracted feature set. Finally, based on the selected features, the classification is performed where the input images get labelled into seven different emotions. The classification is carried out with the use of the hybrid strategy called the Deep Belief Rain Optimization (DBRO) technique. The efficiency of the proposed model is proved through the simulations and it is identified to outperform the other existing approaches.
Similar content being viewed by others
Data availability
Data sharing not applicable to this article as no datasets were generated or analysed during the current study.
References
Alreshidi A, Ullah M (2020) Facial emotion recognition using hybrid features. In: Informatics Multidisciplinary Digital Publishing Institute 7(1):1–13
Anguraju K, Kumar NS, Kumar SJ, Anandhan K, Preethi P (2020) Adaptive feature selection based learning model for emotion recognition. J Critic Rev
Arora M, Kumar M (2021) AutoFER: PCA and PSO based automatic facial emotion recognition. Multimed Tools Appl 80(2):3039–3049
Chen Y, He F, Li H, Zhang D, Wu Y (2020) A full migration BBO algorithm with enhanced population quality bounds for multimodal biomedical image registration. Appl Soft Comput 93:106335
Choudhary D, Shukla J (2020) Feature extraction and feature selection for emotion recognition using facial expression. In: 2020 IEEE sixth international conference on multimedia big data (BigMM), pp 125–133
Dhiman G, Singh KK, Soni M, Nagar A, Dehghani M, Slowik A, Kaur A, Sharma A, Houssein EH, Cengiz K (2021 Apr 1) MOSOA: a new multi-objective seagull optimization algorithm. Expert Syst Appl 167:114150
Garg D, Goel P, Pandya S, Ganatra A, Kotecha K (2018) A deep learning approach for face detection using YOLO. In: 2018 IEEE Punecon, pp 1–4
Graumann L, Duesenberg M, Metz S, Schulze L, Wolf OT, Roepke S, Otte C, Wingenfeld K (2021 Jan 1) Facial emotion recognition in borderline patients is unaffected by acute psychosocial stress. J Psychiatr Res 132:131–135
Hassan AK, Mohammed SN (2020 Oct 1) A novel facial emotion recognition scheme based on graph mining. Defence Technology 16(5):1062–1072
Hosny KM, Kassem MA, Fouad MM (2020) Classification of skin lesions into seven classes using transfer learning with AlexNet. J Digit Imaging 33(5):1325–1334
Hou N, He F, Zhou Y, Chen Y (2020) An efficient GPU-based parallel tabu search algorithm for hardware/software co-design. Front Comput Sci 14(5):1–18
Jahanjoo A, Naderan M, Rashti MJ (2020) Detection and multi-class classification of falling in elderly people by deep belief network algorithms. J Ambient Intell Humaniz Comput 11(10):4145–4165
Jain DK, Shamsolmoali P, Sehdev P (2019) Extended deep neural network for facial emotion recognition. Pattern Recogn Lett 120:69–74
Li B, Lima D (2021) Facial expression recognition via ResNet-50. Int J Cogn Comput Eng 2:57–64
Li H, He F, Chen Y, Luo J (2020) Multi-objective self-organizing optimization for constrained sparse array synthesis. Swarm Evol Comput 58:100743
Liang Y, He F, Zeng X (2020) 3D mesh simplification with feature preservation based on whale optimization algorithm and differential evolution. Integrated computer-aided engineering preprint, pp 1–19
Luo J, He F, Yong J (2020) An efficient and robust bat algorithm with fusion of opposition-based learning and whale optimization algorithm. Intell Data Anal 24(3):581–606
Ma T, Benon K, Arnold B, Yu K, Yang Y, Hua Q, Wen Z, Paul AK (2020) Bottleneck feature extraction-based deep neural network model for facial emotion recognition. In: International Conference on Mobile Networks and Management, Springer, Cham, pp 30–46
Mehendale N (2020 Mar) Facial emotion recognition using convolutional neural networks (FERC). SN Appl Sci 2(3):1–8
Mistry K, Rizvi B, Rook C, Iqbal S, Zhang L, Joy CP (2020) A multi-population FA for automatic facial emotion recognition. In: 2020 international joint conference on neural networks (IJCNN), IEEE, pp 1–8
Moazzeni AR, Khamehchi E (2020 Dec 1) Rain optimization algorithm (ROA): a new metaheuristic method for drilling optimization solutions. J Pet Sci Eng 195:107512
Nawaz R, Cheah KH, Nisar H, Yap VV (2020 Jul 1) Comparison of different feature extraction methods for EEG-based emotion recognition. Biocybern Biomed Eng 40(3):910–926
Nguyen TD (n.d.) Multimodal emotion recognition using deep learning techniques (Doctoral dissertation, Queensland University of Technology), pp 1–138
Patwari M, Gutjahr R, Raupach R, Maier A (2020) Low dose CT Denoising via joint bilateral filtering and intelligent parameter optimization, pp 1–4. arXiv preprint arXiv:2007.04768
Rahul M, Shukla R, Goyal PK, Siddiqui ZA, Yadav V (2021) Gabor filter and ICA-based facial expression recognition using two-layered hidden Markov model. In: Advances in computational intelligence and communication technology. Springer, Singapore, pp 511–518
Saha S, Ghosh M, Ghosh S, Sen S, Singh PK, Geem ZW, Sarkar R (2020 Jan) Feature selection for facial emotion recognition using cosine similarity-based harmony search algorithm. Appl Sci 10(8):2816
Sepas-Moghaddam A, Etemad A, Pereira F, Correia PL (2020) Facial emotion recognition using light field images with deep attention-based bidirectional LSTM. In: ICASSP 2020-2020 IEEE international conference on acoustics, speech and signal processing (ICASSP), IEEE, pp 3367–3371
Siddiqui MF, Javaid AY (2020 Sep) A multimodal facial emotion recognition framework through the fusion of speech with visible and infrared images. Multimodal Technol Interact 4(3):46
Simcock G, McLoughlin LT, De Regt T, Broadhouse KM, Beaudequin D, Lagopoulos J, Hermens DF (2020 Jan) Associations between facial emotion recognition and mental health in early adolescence. Int J Environ Res Public Health 17(1):330
Slimani K, Kas M, El Merabet Y, Ruichek Y, Messoussi R (2020 Aug 1) Local feature extraction based facial emotion recognition: a survey. Int J Electr Comput Eng 10(4):4080
Staff AI, Luman M, Van der Oord S, Bergwerff CE, van den Hoofdakker BJ, Oosterlaan J (2021 Jan 7) Facial emotion recognition impairment predicts social and emotional problems in children with (subthreshold) ADHD. Eur Child Adolesc Psychiatry 31:1–3
Ulusoy SI, Gülseren ŞA, Özkan N, Bilen C (2020 Jul 1) Facial emotion recognition deficits in patients with bipolar disorder and their healthy parents. Gen Hosp Psychiatry 65:9–14
Wang S-H, Phillips P, Dong Z-C, Zhang Y-D (2018) Intelligent facial emotion recognition based on stationary wavelet entropy and Jaya algorithm. Neurocomputing 272:668–676
Wang K, Su G, Liu L, Wang S (2020 Jul 20) Wavelet packet analysis for speaker-independent emotion recognition. Neurocomputing 398:257–264
Wieckowski AT, White SW (2020 Jan) Attention modification to attenuate facial emotion recognition deficits in children with autism: a pilot study. J Autism Dev Disord 50(1):30–41
Yang M, Xiao X, Liu Z, Sun L, Guo W, Cui L, Sun D, Zhang P, Yang G (2020) Deep RetinaNet for dynamic left ventricle detection in multiview echocardiography classification. Sci Program 2020:1–6
Yarasca FA, Henríquez SD (2020) Intelligent system based on wavelets for automatic facial emotion recognition. In: 2020 IEEE engineering international research conference (EIRCON), pp 1–4
Yildirim S, Kaya Y, Kılıç F (2021 Feb) A modified feature selection method based on metaheuristic algorithms for speech emotion recognition. Appl Acoust 173:107721
Yin Z, Liu L, Chen J, Zhao B, Wang Y (2020) Locally robust EEG feature selection for individual-independent emotion recognition. Exp Sys Appl 162:113768
Zhou W, Gao S, Zhang L, Lou X (2020 Mar 13) Histogram of oriented gradients feature extraction from raw Bayer pattern images. IEEE Trans Circuits Syst II Express Briefs 67(5):946–950
Funding
No funding is provided for the preparation of manuscript.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethics approval
This article does not contain any studies with human participants or animals performed by any of the authors.
Conflict of interest
Authors Fakir Mashuque Alamgir, Md. Shafiul Alam declares that they have no conflict of interest.
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Alamgir, F.M., Alam, M.S. An artificial intelligence driven facial emotion recognition system using hybrid deep belief rain optimization. Multimed Tools Appl 82, 2437–2464 (2023). https://doi.org/10.1007/s11042-022-13378-x
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-022-13378-x