Intuitionistic fuzzy least square twin support vector machines for pattern classification

Abstract

Twin support vector machine (TSVM) is an effective machine learning tool for classification problems. However, TSVM classifier works on empirical risk principle only and also while training, each sample contributes equally, even if it is a noise or an outlier. It does not incorporate the uncertainties associated with data into modeling and hence its generalization ability declines. To address these issues, intuitionistic fuzzy regularized least square twin support vector machine having intuitionistic fuzzy network has been proposed in this paper. The non-parallel classifiers are obtained by solving two systems of linear equations only rather than the solution of two quadratic programming problems as in TSVM, which leads to speed up the training process. Moreover, the method follows both structural risk and empirical risk minimization principles. In order to de-escalate the effect of pollutant patterns, their contribution of the patterns into learning the decision function has been made according to their importance in the classification. The significance of the training patterns is measured in terms of intuitionistic fuzzy numbers based on their geometrical locations and surroundings. The method is further extended to find non-parallel decision planes in the feature space using nonlinear kernel function, which also gives rise to the solution of two systems of linear equations. To show the efficacy of the proposed method, computer simulations on fourteen standard and six big UCI datasets using linear and Gaussian kernels are performed and their results have been compared with well-established methods in the literature. The experimental results are represented in terms of accuracy, computational time, F-measure, sensitivity and specificity rates. The outcomes demonstrate that the proposed method outperforms the existing methods and is also feasible for big datasets. The comparison and statistical inferences using two non-parametric: Friedman and Nemenyi tests, conclude that the proposed approach is fast and yields better generalization.

Appendix A

For any $C_1,C_2>0,$ the matrix $C_1\{H^TH+C_2I+C_1(S_2G)^T(S_2G)\}$ is always invertible.

Proof

Let x be a non-zero column vector of $(n+1)\times 1$ order.

Now,

$$\begin{aligned} x^T(H^TH)x=(Hx)^T(Hx). \\ ~~~~~~~~=||Hx||^2 \\ \ge 0. \end{aligned}$$

Hence, $H^TH$ is a positive semi-definite matrix.

Similarly, $C_1(S_2G)^T(S_2G),$ for any $C_1>0$ is a positive semi-definite matrix.

Since, the sum of two positive semi-definite matrices is also a positive semi-definite matrix, therefore, $H^TH+C_1(S_2G)^T(S_2G)$ is a positive semi-definite matrix.

Now, as I (identity matrix) is a positive definite matrix, hence, for all $C_1,C_2>0,$ $C_1\{H^TH+C_2I+C_1(S_2G)^T(S_2G)\}$ is a positive definite matrix which in term yields the result. $\square$