Finding Outpoints in Noisy Binary Sequences — A Revised Empirical Evaluation

Abstract

Kearns et al. (1997) in an earlier paper presented an empirical evaluation of model selection methods on a specialized version of the segmentation problem. The inference task was the estimation of a predefined Boolean function on the real interval [0,1] from a noisy random sample. Three model selection methods based on the Guaranteed Risk Minimization, Minimum Description Length (MDL) Principle and Cross Validation were evaluated on samples with varying noise levels. The authors concluded that, in general, none of the methods was superior to the others in terms of predictive accuracy. In this paper we identify an inefficiency in the MDL approach as implemented by Kearns et al. and present an extended empirical evaluation by including a revised version of the MDL method and another approach based on the Minimum Message Length (MML) principle.