Abstract
The representation learning is the key to deep learning. As a special deep learning algorithm, the generalization performance of the multilayer extreme learning machine (ML-ELM) is influenced by the feature extraction capability of the extreme learning machine autoencoder (ELM-AE). But the ELM-AE does not consider class labels, and it is difficult for the ELM-AE to build the discriminative feature space in the classification tasks. Thus, to improve the class separability of abstract feature, the discriminant regularization is introduced into the ELM-AE and a new regularized ELM-AE called discriminative extreme learning machine autoencoder (DELM-AE) is proposed. And by stacking the ELM-AE for feature extraction and adopting the ELM for classifying, the multilayer discriminative extreme learning machine (ML-DELM) is proposed for classification tasks. The DELM-AE adds the discriminant regularization term to the loss function to reduce the distance between samples and their class centers in feature space. Compared with the ELM-AE and ML-ELM, the DELM-AE and ML-DELM indirectly utilizes the category information contained in the discriminant regularization term to guide the feature extraction. Empirical evaluations and experiments on various benchmark datasets show that feature representation learned by the DELM-AE is discriminative and sparse, and the generalization performance and structural sparsity of ML-DELM are comparable with the other state-of-the-art ML-ELM algorithms.
Similar content being viewed by others
Data availability statement
The data used to support the findings of this study are available from the corresponding author upon request.
References
Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1–3):489–501
Zhu S, Wang H, Lv H, Zhang HS (2021) Augmented online sequential quaternion extreme learning machine. Neural Process Lett 53(2):1161–1186
Zabala-Blanco D, Mora M, Barrientos RJ, Hernandez-Garcia R, Naranjo-Torres J (2020) Fingerprint classification through standard and weighted extreme learning machines. Appl Sci Basel 10(12):4125
Akyol K (2020) Comparing of deep neural networks and extreme learning machines based on growing and pruning approach. Expert Syst Appl 140:112875
Yan DQ, Chu YH, Zhang HY, Liu DS (2018) Information discriminative extreme learning machine. Soft Comput 22(2):677–689
Li R, Wang XD, Lei L, Song YF (2019) L21-norm based loss function and regularization extreme learning machine. IEEE Access 7:6575–6586
Huang GB, Chen L, Siew CK (2006) Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Trans Neural Netw 17(4):879–892
Huang GB, Zhou HM, Ding XJ, Zhang R (2011) Extreme learning machine for regression and multiclass classification. IEEE Trans Syst Man Cybern Part B Cybern 42(2):513–529
Chen Y, Tong CD, Ge YH, Lan T (2021) Fault detection based on auto-regressive extreme learning machine for nonlinear dynamic processes. Appl Soft Comput 106:107319
Sikder N, Mohammad Arif AS, Islam MM, Nahid AA (2021) Induction motor bearing fault classification using extreme learning machine based on power features. Arab J Sci Eng 46(9):8475–8491
Bhatti YK, Jamil A, Nida N, Yousaf MH, Viriri S, Velastin SA (2021) Facial expression recognition of instructor using deep features and extreme learning machine. Comput Intell Neurosci 2021:5570870
Goel T, Murugan R (2020) Classifier for face recognition based on deep convolutional-optimized kernel extreme learning machine. Comput Electr Eng 85:106640
Lahoura V, Singh H, Aggarwal A, Mohammed MA, Damasevicius R (2021) Cloud computing-based framework for breast cancer diagnosis using extreme learning machine. Diagnostics 11(2):241
Zhao LL, Qian JH, Tian FC, Liu R, Liu B (2021) A weighted discriminative extreme learning machine design for lung cancer detection by an electronic nose system. IEEE Trans Instrum Meas 70:1–9
Hinton GE, Salakhutdinov RR (2006) Reducing the dimensionality of data with neural networks. Science 313(5786):504–507
Alom MZ, Taha TM, Yakopcic C, Westberg S, Sidike P (2019) A state-of-the-art survey on deep learning theory and architectures. Electronics 8(3):292
Kasun L, Zhou H, Huang GB, Vong CM (2013) Representational learning with extreme learning machine for big data. IEEE Intell Syst 28(6):31–34
Zhang J, Xiao WD, Li YJ, Zhang S, Zhang ZQ (2020) Multilayer probability extreme learning machine for device-free localization. Neurocomputing 396:383–393
Khan MA, Abbas S, Khan KM, Al Ghamdi MA, Rehman A (2020) Intelligent forecasting model of COVID-19 novel coronavirus outbreak empowered with deep extreme learning machine. CMC Comput Mat Contin 64(3):1329–1342
Li Q, Zhang XY, Ma TJ, Jiao CL, Wang H, Hu W (2021) A multi-step ahead photovoltaic power prediction model based on similar day, enhanced colliding bodies optimization, variational mode decomposition, and deep extreme learning machine. Energy 224:120094
Cao F, Yang Z, Ren J, Chen WZ, Han GJ, Shen YS (2019) Local block multilayer sparse extreme learning machine for effective feature extraction and classification of hyperspectral images. IEEE Trans Geosci Remote Sens 57(8):5580–5594
Li H, Zhao H, Li H (2018) Neural-response-based extreme learning machine for image classification. IEEE Trans Neural Netw Learn Syst 30(2):539–552
Zhang N, Ding S, Shi Z (2016) Denoising Laplacian multi-layer extreme learning machine. Neurocomputing 171:1066–1074
Wong CM, Vong CM, Wong PK, Cao JW (2016) Kernel-based multilayer extreme learning machines for representation learning. IEEE Trans Neural Netw Learn Syst 29(3):757–762
Sun K, Zhang JS, Zhang CX, Hu JY (2017) Generalized extreme learning machine autoencoder and a new deep neural network. Neurocomputing 230:374–381
Li R, Wang XD, Song Y, Lei L (2021) Hierarchical extreme learning machine with L21-norm loss and regularization. Int J Mach Learn Cybern 12(5):1297–1310
Liu Y, Wang J (2022) Transfer learning based multi-layer extreme learning machine for probabilistic wind power forecasting. Appl Energy 312:118729
Luo X, Li Y, Wang WP, Ban XJ, Wang JH, Zhao WB (2020) A robust multilayer extreme learning machine using kernel risk-sensitive loss criterion. Int J Mach Learn Cybern 11(1):197–216
Wu D, Li T, Wan Q (2021) A hybrid deep kernel incremental extreme learning machine based on improved coyote and beetle swarm optimization methods. Complex Intell Syst 7(6):3015–3032
Blake CL, Merz CJ (1998) UCI repository of machine learning databases. Department of Information Computer Science, University of California, Irvine, CA. [Online]. http://archive.ics.uci.edu/m. Accessed 21 Jan 2022
LeCun Y, Bottou L, Bengio Y, Haffner P (1998) Gradient-based learning applied to document recognition. Proc IEEE 86(11):2278–2324
Xiao H, Rasul K, Vollgraf R (2017) Fashion-mnist: a novel image dataset for benchmarking machine learning algorithms. arXiv preprint arXiv:1708.07747
Hull JJ (1994) A database for handwritten text recognition research. IEEE Trans Pattern Anal Mach Intell 16(5):550–554
Du H, Song DQ, Chen Z, Shu HP, Guo ZZ (2020) Prediction model oriented for landslide displacement with step-like curve by applying ensemble empirical mode decomposition and the PSO-ELM method. J Clean Prod 270:122248
Krishnan GS, Kamath S (2019) A novel GA-ELM model for patient-specific mortality prediction over large-scale lab event data. Appl Soft Comput 80:525–533
Acknowledgements
The research is supported by the National Natural Science Foundation of China (61876189, 61273275, 61806219, and 61703426) and the Natural Science Basic Research Plan in Shaanxi Province (no. 2021JM—226).
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
The authors declare no conflict of interest.
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
Lai, J., Wang, X., Xiang, Q. et al. Multilayer discriminative extreme learning machine for classification. Int. J. Mach. Learn. & Cyber. 14, 2111–2125 (2023). https://doi.org/10.1007/s13042-022-01749-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s13042-022-01749-7