Normalized Maximum Likelihood Models for Boolean Regression with Application to Prediction and Classification in Genomics

Conclusions

Boolean regression classes of models are powerful modeling tools having associated NML models which can be easily computed and used in MDL inference, in particular for factor selection.

Comparing the MDL methods based on the two-part codes with those based on the NML models, we note that the former is faster to evaluate, but the latter provides a significantly shorter codelength and hence a better description of the data. When analyzing the gene expression data, speed may be a major concern, since one has to test ({sk_k/ⁿ}) possible groupings of k genes, with n in the order of thousands and usually less than 10. The two-partcodes may then be used for pre-screening of the gene groupings, to remove the obviously poor performers, and then the NML model could be applied to obtain the final selection from a smaller pool of candidates. The running time for all our experiments reported here is in the order of tens of minutes.

The use of the MDL principle for classification with the class of Boolean models provides an effective classification method as demonstrated with the important cancer classification example based on gene expression data. The NML model for the class M(θ, k, f) was used for the selection of informative feature genes. When using the sets of feature genes, selected by NML model, we achieved classification error rates significantly lower than those reported recently for the same data set.