Abstract
Model diversity is essential for ensemble classifiers, which make predictions by combining predictions from multiple simpler models. While ensemble classifiers often outperform single-model classifiers, their success crucially depends on the ensemble’s construction. Genetic programming (GP) is a powerful evolutionary algorithm that can evolve populations of simple classifiers; however, standard GP algorithms produce populations of models with correlated predictions. Recent work in the broader evolutionary computing community has begun focusing on methods for evolving diverse populations, such as MAP-Elites [24], which can evolve populations that are diverse in a low dimensional behavior space. In this work, we demonstrate a novel technique for using MAP-Elites to create diverse GP populations, which can be used as ensemble classifiers. We demonstrate the utility of our framework, which we call Neuro-MAP-Elites, by comparing it with other classification algorithms across a diverse set of classification datasets.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
See github.com/BigTuna08/nme for the code to tune parameters of all models.
References
Beadle, L., Johnson, C.: Semantically driven crossover in genetic programming, pp. 111–116, July 2008
Beadle, L., Johnson, C.: Semantically driven mutation in genetic programming, pp. 1336–1342, May 2009
Berkson, J.: Application of the logistic function to bio-assay. J. Am. Statist. Assoc. 39(227), 357–365 (1944)
Boisvert, S., Sheppard, J.W.: Quality diversity genetic programming for learning decision tree ensembles. In: Hu, T., Lourenço, N., Medvet, E. (eds.) EuroGP 2021. LNCS, vol. 12691, pp. 3–18. Springer, Cham (2021). https://doi.org/10.1007/978-3-030-72812-0_1
Brameier, M.F., Banzhaf, W.: Linear Genetic Programming, 1st edn. Springer Publishing Company Inc., Cham (2007)
Breiman, L.: Random forests. Mach. Learn. 45(1), 5–32 (2001)
Brown, G., Wyatt, J., Harris, R., Yao, X.: Diversity creation methods: a survey and categorisation. Inf. Fusion 6(1), 5–20 (2005)
Canaan, R., Togelius, J., Nealen, A., Menzel, S.: Diverse agents for ad-hoc cooperation in Hanabi. In: 2019 IEEE Conference on Games (CoG), pp. 1–8 (2019)
Cardoso, R.P., Hart, E., Kurka, D.B., Pitt, J.V.: Using novelty search to explicitly create diversity in ensembles of classifiers. In: GECCO 2021, pp. 849–857. ACM, New York, NY, USA (2021)
Cully, A., Clune, J., Tarapore, D., Mouret, J.B.: Robots that can adapt like animals. Nature 521, 503–507 (2015)
Dolson, E., Lalejini, A., Ofria, C.: Exploring genetic programming systems with map-elites, August 2018
Fontaine, M.C., Lee, S., Soros, L.B., De Mesentier Silva, F., Togelius, J., Hoover, A.K.: Mapping hearthstone deck spaces through map-elites with sliding boundaries. In: GECCO 2019, pp. 161–169. ACM, New York, NY, USA (2019)
Gagné, C., Sebag, M., Schoenauer, M., Tomassini, M.: Ensemble learning for free with evolutionary algorithms? In: GECCO 2007, pp. 1782–1789. ACM, New York, NY, USA (2007)
Gravina, D., Khalifa, A., Liapis, A., Togelius, J., Yannakakis, G.N.: Procedural content generation through quality diversity (2019)
Hastie, T., Tibshirani, R., Friedman, J.H.: The Elements of Statistical Learning: Data Mining, Inference, and Prediction, 2nd edn. Springer, Cham (2009). https://doi.org/10.1007/978-0-387-84858-7
Higgins, I., t al.: beta-VAE: learning basic visual concepts with a constrained variational framework. In: ICLR (2017)
Hinton, G.E., Salakhutdinov, R.R.: Reducing the dimensionality of data with neural networks. Science 313(5786), 504–507 (2006)
Kingma, D.P., Welling, M.: Auto-encoding variational bayes (2014)
Koza, J.R.: Genetic programming: automatic programming of computers. EvoNews 1(3), 4–7 (1997)
Krawiec, K., Lichocki, P.: Approximating geometric crossover in semantic space. In: GECCO 2009, pp. 987–994. ACM, New York, NY, USA (2009)
Lehman, J., Stanley, K.O.: Abandoning objectives: evolution through the search for novelty alone. Evol. Comput. 19(2), 189–223 (2011)
Miller, J.: Cartesian Genetic Programming, vol. 43, June 2003
Moraglio, A., Krawiec, K., Johnson, C.G.: Geometric semantic genetic programming. In: Coello, C.A.C., Cutello, V., Deb, K., Forrest, S., Nicosia, G., Pavone, M. (eds.) PPSN 2012. LNCS, vol. 7491, pp. 21–31. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-32937-1_3
Mouret, J., Clune, J.: Illuminating search spaces by mapping elites (2015)
Nguyen, Q.U., Hoai, N., O’Neill, M., McKay, R., Galván-López, E.: Semantically-based crossover in genetic programming: application to real-valued symbolic regression. Genetic Program. Evol. Mach. 12, 91–119 (2011)
Olson, R.S., La Cava, W., Orzechowski, P., Urbanowicz, R.J., Moore, J.H.: PMLB: a large benchmark suite for machine learning evaluation and comparison. BioData Mining 10(1), 36 (2017)
Pedregosa, F., et al.: Scikit-learn: machine learning in Python. J. Mach. Learn. Res. 12, 2825–2830 (2011)
Platt, J.C.: Probabilistic outputs for support vector machines and comparisons to regularized likelihood methods. In: Advances in Large Margin Classifiers, pp. 61–74. MIT Press (1999)
Rodrigues, N.M., Batista, J.E., Silva, S.: Ensemble genetic programming. In: Hu, T., Lourenço, N., Medvet, E., Divina, F. (eds.) EuroGP 2020. LNCS, vol. 12101, pp. 151–166. Springer, Cham (2020). https://doi.org/10.1007/978-3-030-44094-7_10
Sha, C., Cuperlovic-Culf, M., Hu, T.: Smile: systems metabolomics using interpretable learning and evolution. BMC Bioinform. 22(1), 284 (2021)
Stanley, K.O., Miikkulainen, R.: Evolving neural networks through augmenting topologies. Evol. Comput. 10(2), 99–127 (2002)
Vassiliades, V., Chatzilygeroudis, K., Mouret, J.: Using centroidal voronoi tessellations to scale up the multidimensional archive of phenotypic elites algorithm. IEEE Trans. Evol. Comput. 22(4), 623–630 (2018)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2022 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Nickerson, K., Kolokolova, A., Hu, T. (2022). Creating Diverse Ensembles for Classification with Genetic Programming and Neuro-MAP-Elites. In: Medvet, E., Pappa, G., Xue, B. (eds) Genetic Programming. EuroGP 2022. Lecture Notes in Computer Science, vol 13223. Springer, Cham. https://doi.org/10.1007/978-3-031-02056-8_14
Download citation
DOI: https://doi.org/10.1007/978-3-031-02056-8_14
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-031-02055-1
Online ISBN: 978-3-031-02056-8
eBook Packages: Computer ScienceComputer Science (R0)