Abstract
When the training data in a two-class classification problem is overwhelmed by one class, most classification techniques fail to correctly identify the data points belonging to the underrepresented class. This paper proposes Similarity-based Imbalanced Classification (SBIC) that simultaneously optimizes the weights of the empirical similarity function and identifies the locations of absent data points, i.e. unobserved data points from the minority class. Similar to cost-sensitive approaches, SBIC operates on an algorithmic level to handle imbalanced structures and similar to synthetic data generation approaches, it utilizes the properties of unobserved data points. The main contribution of the paper is to show that a similarity function can be used to tackle imbalanced datasets. The results of applying the proposed method to imbalanced datasets suggests that SBIC is comparable to, and in some cases outperforms, other commonly used classification techniques for imbalanced datasets.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
He, H., Garcia, E.A.: Learning from imbalanced data. IEEE Trans. Knowl. Data Eng. 21(9), 1263–1284 (2009)
Byon, E., Shrivastava, A.K., Ding, Y.: A classification procedure for highly imbalanced class sizes. IIE Trans. 42(4), 288–303 (2010)
Pourhabib, A., Mallick, B.K., Ding, Y.: Absent data generating classifier for imbalanced class sizes. J. Mach. Learn. Res. 16, 2695–2724 (2015)
Mika, S., Rätsch, G., Weston, J., Schölkopf, B., Müllers, K.R.: Fisher discriminant analysis with kernels, in neural networks for signal processing IX. In: Proceedings of the 1999 IEEE Signal Processing Society Workshop, pp. 41–48, August 1999
Gilboa, I., Lieberman, O., Schmeidler, D.: A similarity-based approach to prediction. J. Econom. 162(1), 124–131 (2011)
Gilboa, I., Lieberman, O., Schmeidler, D.: Empirical similarity. Rev. Econ. Stat. 88(3), 433–444 (2006)
Park, C., Huang, J.Z., Ding, Y.: A computable plug-in estimator of minimum volume sets for novelty detection. Oper. Res. 58(5), 1469–1480 (2010)
Efron, B.: The Jackknife, the Bootstrap and Other Resampling Plans. CBMS-NSF Regional Conference Series in Applied Mathematics, vol. 38. SIAM, Philadelphia (1982)
Galar, M., Fernandez, A., Barrenechea, E., Bustince, H., Herrera, F.: A review on ensembles for the class imbalance problem: bagging-, boosting-, and hybrid-based approaches. IEEE Trans. Syst. Man Cybern. C Appl. Rev. 42(4), 463–484 (2012)
Chen, J.J., Tsai, C.A., Young, J.F., Kodell, R.L.: Classification ensembles for unbalanced class sizes in predictive toxicology. SAR QSAR Environ. Res. 16(6), 517–529 (2005)
Chawla, N.V., Bowyer, K.W., Hall, L.O., Kegelmeyer, W.P.: SMOTE: synthetic minority over-sampling technique. J. Artif. Intell. Res. 16, 321–357 (2002)
Han, H., Wang, W.Y., Mao, B.H.: Borderline-SMOTE: a new over-sampling method in imbalanced data sets learning. Advances in Intelligent Computing. Lecture Notes in Computer Science, vol. 3644, pp. 878–887. Springer, Heidelberg (2005)
Chen, S., He, H., Garcia, E.A.: RAMOBoost: ranked minority oversampling in boosting. IEEE Trans. Neural Netw. 21(10), 1624–1642 (2010)
Barua, S., Islam, M.M., Yao, X., Murase, K.: MWMOTE-majority weighted minority oversampling technique for imbalanced data set learning. IEEE Trans. Knowl. Data Eng. 26(2), 405–425 (2014)
Ramentol, E., Caballero, Y., Bello, R., Herrera, F.: SMOTE-RSB*: a hybrid preprocessing approach based on oversampling and undersampling for high imbalanced data-sets using smote and rough sets theory. Knowl. Inf. Syst. 33(2), 245–265 (2012)
Elkan, C.: The foundations of cost-sensitive learning. In: Proceedings of the Seventeenth International Joint Conference on Artificial Intelligence, pp. 973–978 (2001)
Masnadi-Shirazi, H., Vasconcelos, N.: Risk minimization, probability elicitation, and cost-sensitive SVMs. In: Proceedings of the 27th International Conference on Machine Learning (ICML 2010), pp. 759–766 (2010)
Zhou, Z.-H., Liu, X.-Y.: Training cost-sensitive neural networks with methods addressing the class imbalance problem. IEEE Trans. Knowl. Data Eng. 18(1), 63–77 (2006)
Sun, Y., Kamel, M.S., Wong, A.K., Wang, Y.: Cost-sensitive boosting for classification of imbalanced data. Pattern Recognit. 40(12), 3358–3378 (2007)
Li, Q.P.: Speaker Authentication. Springer, Heidelberg (2012)
Xie, S., Imani, M., Dougherty, E.R., Braga-Neto, U.M.: Nonstationary linear discriminant analysis. In: 2017 51st Asilomar Conference on Signals, Systems, and Computers, pp. 161–165, October 2017
de Mantaras, R.L., Armengol, E.: Machine learning from examples: inductive and lazy methods. Data Knowl. Eng. 25(1), 99–123 (1998)
Billot, A., Gilboa, I., Schmeidler, D.: Axiomatization of an exponential similarity function. Math. Soc. Sci. 55(2), 107–115 (2008)
Byrd, R.H., Gilbert, J.C., Nocedal, J.: A trust region method based on interior point techniques for nonlinear programming. Math. Program. 89(1), 149–185 (2000)
Hastie, T., Tibshirani, R., Friedman, J.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction. Springer Series in Statistics, 2nd edn. Springer, New York (2009)
Liu, X.-Y., Wu, J., Zhou, Z.-H.: Exploratory undersampling for class-imbalance learning. IEEE Trans. Syst. Man Cybern. B Cybern. 39(2), 539–550 (2009)
Kanungo, T., Mount, D.M., Netanyahu, N.S., Piatko, C.D., Silverman, R., Wu, A.Y.: An efficient k-means clustering algorithm: analysis and implementation. IEEE Trans. Pattern Anal. Mach. Intell. 24(7), 881–892 (2002)
Bradley, A.P.: The use of the area under the ROC curve in the evaluation of machine learning algorithms. Pattern Recognit. 30(7), 1145–1159 (1997)
Veropoulos, K., Campbell, C., Cristianini, N., et al.: Controlling the sensitivity of support vector machines. In: Proceedings of the International Joint Conference on AI, pp. 55–60 (1999)
Lichman, M.: UCI machine learning repository (2013). http://archive.ics.uci.edu/mllastaccessed07/2014
Center for evidence-based medicine. http://www.cebm.brown.edu/static/imbalanced-datasets.zip (2014). Accessed July 2014
Dems̆ar, J.: Statistical comparisons of classifiers over multiple data sets. J. Mach. Learn. Res. 7, 1–30 (2006)
Acknowledgements
The research was partly supported by OSU Foundation for the National Energy Solutions Institute - Smart Energy Source, grant 20-96680. This work was completed utilizing the High Performance Computing Center facilities of Oklahoma State University at Stillwater.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer Nature Switzerland AG
About this paper
Cite this paper
Pourhabib, A. (2020). Empirical Similarity for Absent Data Generation in Imbalanced Classification. In: Arai, K., Bhatia, R. (eds) Advances in Information and Communication. FICC 2019. Lecture Notes in Networks and Systems, vol 69. Springer, Cham. https://doi.org/10.1007/978-3-030-12388-8_70
Download citation
DOI: https://doi.org/10.1007/978-3-030-12388-8_70
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-12387-1
Online ISBN: 978-3-030-12388-8
eBook Packages: Intelligent Technologies and RoboticsIntelligent Technologies and Robotics (R0)