Optimal “Anti-Bayesian” Parametric Pattern Classification for the Exponential Family Using Order Statistics Criteria

Abstract

This paper reports some pioneering results in which optimal parametric classification is achieved in a counter-intuitive manner, quite opposed to the Bayesian paradigm. The paper, which builds on the results of [1], demonstrates (with both theoretical and experimental results) how this can be done for some distributions within the exponential family. To be more specific, within a Bayesian paradigm, if we are allowed to compare the testing sample with only a single point in the feature space from each class, the optimal Bayesian strategy would be to achieve this based on the (Mahalanobis) distance from the corresponding means, which in one sense, is the most central point in the respective distribution. In this paper, we shall show that we can obtain optimal results by operating in a diametrically opposite way, i.e., a so-called “anti-Bayesian” manner. Indeed, we shall show that by working with a very few (sometimes as small as two) points distant from the mean, one can obtain remarkable classification accuracies. These points, in turn, are determined by the Order Statistics of the distributions, and the accuracy of our method, referred to as Classification by Moments of Order Statistics (CMOS), attains the optimal Bayes’ bound! In this paper, we shall show the claim for two uni-dimensional members of the exponential family. The theoretical results, which have been verified by rigorous experimental testing, also present a theoretical foundation for the families of Border Identification (BI) reported algorithms.