Abstract
This paper presents an extension of the well-known Extreme Learning Machines (ELMs). The main goal is to provide probabilities as outputs for Multiclass Classification problems. Such information is more useful in practice than traditional crisp classification outputs. In summary, Gaussian Mixture Models are used as post-processing of ELMs. In that context, the proposed global methodology is keeping the advantages of ELMs (low computational time and state of the art performances) and the ability of Gaussian Mixture Models to deal with probabilities. The methodology is tested on 3 toy examples and 3 real datasets. As a result, the global performances of ELMs are slightly improved and the probability outputs are seen to be accurate and useful in practice.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Qi, X., Davison, B.D.: Web page classification: Features and algorithms. ACM Comput. Surv. 41(2), 12:1–12:31 (2009)
Patil, A.S., Pawar, B.: Automated classification of web sites using naive bayesian algorithm. In: Proceedings of the International MultiConference of Engineers and Computer Scientists. vol. 1 (2012)
Dahl, G., Stokes, J.W., Deng, L., Yu, D.: Large-scale malware classification using random projections and neural networks. In: Proceedings IEEE Conference on Acoustics, Speech, and Signal Processing, IEEE SPS, May 2013
Rieck, K., Trinius, P., Willems, C., Holz, T.: Automatic analysis of malware behavior using machine learning. J. Comput. Secur. 19(4), 639–668 (2011)
Miche, Y., Akusok, A., Hegedus, J., Nian, R.: A Two-Stage Methodology using K-NN and False Positive Minimizing ELM for Nominal Data Classification. Cognitive Computation, pp. 1–26 (2014)
Akusok, A., Veganzones, D., Björk, K.M., Séverin, E., du Jardin, P., Lendasse, A., Miche, Y.: ELM clustering-application to bankruptcy prediction-. In: International Work Conference on TIme SEries, pp. 711–723 (2014)
Sirola, M., Talonen, J., Lampi, G.: SOM based methods in early fault detection of nuclear industry. In: ESANN (2009)
Luo, J., Vong, C.M., Wong, P.K.: Sparse bayesian extreme learning machine for multi-classification. IEEE Transactions on Neural Networks and Learning Systems 25(4), 836–843 (2014)
Huang, G.B., Zhu, Q.Y., Siew, C.K.: Extreme learning machine: Theory and applications. Neurocomputing 70(1–3), 489–501 (2006)
Huang, G.B., Chen, L., Siew, C.K.: Universal approximation using incremental constructive feedforward networks with random hidden nodes. IEEE Transactions on Neural Networks 17(4), 879–892 (2006)
Miche, Y., van Heeswijk, M., Bas, P., Simula, O., Lendasse, A.: TROP-ELM: A double-regularized ELM using LARS and Tikhonov regularization. Neurocomputing 74(16), 2413–2421 (2011)
Lendasse, A., Akusok, A., Simula, O., Corona, F., van Heeswijk, M., Eirola, E., Miche, Y.: Extreme learning machine: a robust modeling technique? yes!. In: Rojas, I., Joya, G., Gabestany, J. (eds.) IWANN 2013, Part I. LNCS, vol. 7902, pp. 17–35. Springer, Heidelberg (2013)
Yu, Q., van Heeswijk, M., Miche, Y., Nian, R., He, B., Séverin, E., Lendasse, A.: Ensemble delta test-extreme learning machine (dt-elm) for regression. Neurocomputing 129, 153–158 (2014). cited By 2
Miche, Y., Sorjamaa, A., Bas, P., Simula, O., Jutten, C., Lendasse, A.: Op-elm: Optimally pruned extreme learning machine. IEEE Transactions on Neural Networks 21(1), 158–162 (2010)
Miche, Y., Sorjamaa, A., Lendasse, A.: OP-ELM: theory, experiments and a toolbox. In: Kůrková, V., Neruda, R., Koutník, J. (eds.) ICANN 2008, Part I. LNCS, vol. 5163, pp. 145–154. Springer, Heidelberg (2008)
Cambria, E., Huang, G.B., Kasun, L.L.C., Zhou, H., Vong, C.M., Lin, J., Yin, J., Cai, Z., Liu, Q., Li, K., Leung, V.C., Feng, L., Ong, Y.S., Lim, M.H., Akusok, A., Lendasse, A., Corona, F., Nian, R., Miche, Y., Gastaldo, P., Zunino, R., Decherchi, S., Yang, X., Mao, K., Oh, B.S., Jeon, J., Toh, K.A., Teoh, A.B.J., Kim, J., Yu, H., Chen, Y., Liu, J.: Extreme Learning Machines. IEEE Intelligent Systems 28(6), 30–59 (2013)
Akusok, A., Veganzones, D., Miche, Y., Severin, E., Lendasse, A.: Finding originally mislabels with MD-ELM. In: Proc. of the 22th European Symposium on Artificial Neural Networks, Computational Intelligence and Machine Learning (ESANN 2014), pp. 689–694 (2014)
van Heeswijk, M., Miche, Y., Lindh-Knuutila, T., Hilbers, P.A.J., Honkela, T., Oja, E., Lendasse, A.: Adaptive ensemble models of extreme learning machines for time series prediction. In: Alippi, C., Polycarpou, M., Panayiotou, C., Ellinas, G. (eds.) ICANN 2009, Part II. LNCS, vol. 5769, pp. 305–314. Springer, Heidelberg (2009)
Grigorievskiy, A., Miche, Y., Ventelä, A.M., Séverin, E., Lendasse, A.: Long-term time series prediction using op-elm. Neural Networks 51, 50–56 (2014). cited By 4
Pouzols, F., Lendasse, A.: Evolving fuzzy optimally pruned extreme learning machine for regression problems. Evolving Systems 1(1), 43–58 (2010)
van Heeswijk, M., Miche, Y., Oja, E., Lendasse, A.: Gpu-accelerated and parallelized ELM ensembles for large-scale regression. Neurocomputing 74(16), 2430–2437 (2011). Advances in Extreme Learning Machine: Theory and Applications Biological Inspired Systems. Computational and Ambient Intelligence Selected papers of the 10th International Work-Conference on Artificial Neural Networks (IWANN2009)
Yu, Q., Miche, Y., Eirola, E., van Heeswijk, M., Séverin, E., Lendasse, A.: Regularized extreme learning machine for regression with missing data. Neurocomputing 102, 45–51 (2013). cited By 9
Benoît, F., van Heeswijk, M., Miche, Y., Verleysen, M., Lendasse, A.: Feature selection for nonlinear models with extreme learning machines. Neurocomputing 102, 111–124 (2013). cited By 8
Akusok, A., Miche, Y., Hegedus, J., Nian, R., Lendasse, A.: A two-stage methodology using k-nn and false-positive minimizing elm for nominal data classification. Cognitive Computation 6(3), 432–445 (2014). cited By 0
Allen, D.M.: The relationship between variable selection and data agumentation and a method for prediction. Technometrics 16(1), 125–127 (1974)
Rao, C.R., Mitra, S.K.: Generalized Inverse of Matrices and Its Applications. John Wiley & Sons Inc (1971)
Bishop, C.M.: Pattern Recognition and Machine Learning. Springer (2006)
Eirola, E., Lendasse, A., Vandewalle, V., Biernacki, C.: Mixture of gaussians for distance estimation with missing data. Neurocomputing 131, 32–42 (2014)
Dempster, A.P., Laird, N.M., Rubin, D.B.: Maximum likelihood from incomplete data via the EM algorithm. Journal of the Royal Statistical Society 39(1), 1–38 (1977). Series B (Methodological)
McLachlan, G., Krishnan, T.: The EM Algorithm and Extensions. Wiley Series in Probability and Statistics. John Wiley & Sons, New York (1997)
Schwarz, G.: Estimating the dimension of a model. The annals of statistics 6(2), 461–464 (1978)
McLachlan, G.J., Peel, D.: Finite Mixture Models. Wiley Series in Probability and Statistics. John Wiley & Sons, New York (2000)
Lichman, M.: UCI Machine Learning Repository (2013). http://archive.ics.uci.edu/ml
Myers, R.: Classical and Modern Regression with Applications. Bookware Companion Series, PWS-KENT (1990)
Akusok, A., Grigorievskiy, A., Lendasse, A., Miche, Y.: Image-based classification of websites. In: Villmann, T., Schleif, F.M. (eds.) Machine Learning Reports 02/2013. Volume ISSN: 1865–3960 of Machine Learning Reports., Saarbrücken, Germany, Workshop of the GI-Fachgruppe Neuronale Netze and the German Neural Networks Society in connection to GCPR 2013, Proceedings of the Workshop - New Challenges in Neural Computation 2013, pp. 25–34 September 2013
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Eirola, E. et al. (2015). Extreme Learning Machines for Multiclass Classification: Refining Predictions with Gaussian Mixture Models. In: Rojas, I., Joya, G., Catala, A. (eds) Advances in Computational Intelligence. IWANN 2015. Lecture Notes in Computer Science(), vol 9095. Springer, Cham. https://doi.org/10.1007/978-3-319-19222-2_13
Download citation
DOI: https://doi.org/10.1007/978-3-319-19222-2_13
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-19221-5
Online ISBN: 978-3-319-19222-2
eBook Packages: Computer ScienceComputer Science (R0)