Skip to main content
SpringerLink
Log in

Flex-GP: Genetic Programming on the Cloud

  • Conference paper
Applications of Evolutionary Computation (EvoApplications 2012)

Part of the book series: Lecture Notes in Computer Science ((LNTCS,volume 7248))

Included in the following conference series:

Abstract

We describe Flex-GP, which we believe to be the first largescale genetic programming cloud computing system. We took advantage of existing software and selected a socket-based, client-server architecture and an island-based distribution model. We developed core components required for deployment on Amazon’s EC2. Scaling the system to hundreds of nodes presented several unexpected challenges and required the development of software for automatically managing deployment, reporting, and error handling. The system’s performance was evaluated on two metrics, performance and speed, on a difficult symbolic regression problem. Our largest successful Flex-GP runs reached 350 nodes and taught us valuable lessons for the next phase of scaling.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Armbrust, M., Fox, A., Griffith, R., Joseph, A., Katz, R., Konwinski, A., Lee, G., Patterson, D., Rabkin, A., Stoica, I., Zaharia, M.: Above the clouds: A berkeley view of cloud computing. EECS Department, University of California, Berkeley, Tech. Rep. UCB/EECS-2009-28 (2009)

    Google Scholar 

  2. Cantú-Paz, E.: A survey of parallel genetic algorithms. Calculateurs Paralleles 10(2) (1998)

    Google Scholar 

  3. Dean, J., Ghemawat, S.: MapReduce: Simplified data processing on large clusters. Communications of ACM 51(1), 107–113 (2008)

    Article  Google Scholar 

  4. Laredo, J.L.J., Castillo, P.A., Paechter, B., Mora, A.M., Alfaro-Cid, E., Esparcia-Alcázar, A.I., Merelo, J.J.: Empirical Validation of a Gossiping Communication Mechanism for Parallel EAs. In: Giacobini, M. (ed.) EvoWorkshops 2007. LNCS, vol. 4448, pp. 129–136. Springer, Heidelberg (2007)

    Google Scholar 

  5. Luke, S., Panait, L., Balan, G., Paus, S., Skolicki, Z., Bassett, J., Hubley, R., Chircop, A.: ECJ: A Java-based evolutionary computation research system (2007), http://cs.gmu.edu/~eclab/projects/ecj/

  6. Ograph, B., Morgens, Y.: Cloud computing. Communications of the ACM 51(7) (2008)

    Google Scholar 

  7. Poli, R., Langdon, W., McPhee, N.: A field guide to genetic programming. Lulu Enterprises UK Ltd. (2008)

    Google Scholar 

  8. Tomassini, M.: Spatially structured evolutionary algorithms. Springer, Heidelberg (2005)

    MATH  Google Scholar 

  9. Vanneschi, L.: Theory and Practice for Efficient Genetic Programming. Ph.D. thesis, Université de Lausanne (2004)

    Google Scholar 

  10. Verma, A., Llora, X., Goldberg, D.E., Campbell, R.H.: Scaling genetic algorithms using MapReduce. In: Proceedings of Intelligent Systems Design and Applications, pp. 13–18 (2009)

    Google Scholar 

  11. Vladislavleva, E., Smits, G., Den Hertog, D.: Order of nonlinearity as a complexity measure for models generated by symbolic regression via Pareto genetic programming. IEEE Transactions on Evolutionary Computation 13(2), 333–349 (2009)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Massachusetts Institute of Technology, USA

    Dylan Sherry, Kalyan Veeramachaneni, James McDermott & Una-May O’Reilly

Authors
  1. Dylan Sherry
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Kalyan Veeramachaneni
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. James McDermott
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Una-May O’Reilly
    View author publications

    You can also search for this author in PubMed Google Scholar

Editor information

Editors and Affiliations

  1. Department of Mathematics and Statistics, University of Strathclyde, 16 Richmond Street, G1 1XQ, Glasgow, UK

    Cecilia Di Chio

  2. Financial Mathematics and Computation Cluster Natural Computing Research and Applications Group Complex and Adaptive Systems Laboratory, University College, Dublin, Ireland

    Alexandros Agapitos

  3. Department of Computer Engineering, University of Parma, Viale Usberti 181/a, 43100, Parma, Italy

    Stefano Cagnoni

  4. ETSI Informática, Universidad de Málaga, Campus de Teatinos, 29071, Málaga, Spain

    Carlos Cotta

  5. Centro Universitario de Merida, Universidad de Extremadura, Spain

    Francisco Fernández de Vega

  6. Dalle Molle Institute for Artificial Intelligence (IDSIA), Lugano, Switzerland

    Gianni A. Di Caro

  7. DFKI Bremen, Cyber-Physical Systems, Bremen, Germany

    Rolf Drechsler

  8. Knowledge Engineering Research Group, Aston University, B4 7ET, Aston Triangle, Birmingham, UK

    Anikó Ekárt

  9. Instituto Tecnológico de Informática, Edif. 8G Acceso B, Universidad Politécnica de Valencia, 46022, Valencia, Spain

    Anna I. Esparcia-Alcázar

  10. Next Generation Intelligent Networks Research Center (nexGIN RC), National University of Computer & Emerging Sciences (FAST-NU), Islamabad, Pakistan

    Muddassar Farooq

  11. Department of Computer Science,, University College London, Gower Street, WC1E 6BT, London, UK

    William B. Langdon

  12. Depto. Arquitectura y Tecnologa de Computadores, ETS Ingeiera Informtica, C/Daniel Saucedo Aranda, s/n, 18071, Granada, Spain

    Juan J. Merelo-Guervós

  13. Chair of Algorithm Engineering, Computational Intelligence Group, Dept. of Computer Science, Technische Universität Dortmund, Germany

    Mike Preuss

  14. HTWK Leipzig, Fakultät Elektrotechnik und Informationstechnik, Institut Mess–, Steuerungs– und Regelungstechnik, Postfach 30 11 66, D–04251, Leipzig, Germany

    Hendrik Richter

  15. INESC-ID Lisboa, Rua Alves Redol 9,, 1000-029, Lisboa, Portugal

    Sara Silva

  16. Centre for Informatics and Systems, University of Coimbra, Portugal

    Anabela Simões

  17. Dipartimento di Automatica e Informatica, Politecnico di Torino, Italy

    Giovanni Squillero

  18. Institute of High Performance Computing and Networking, ICAR-CNR, Naples, Italy

    Ernesto Tarantino

  19. Dipartimento di Tecnologie dell’Informazione, Universita’ degli Studi di Milano, Italy

    Andrea G. B. Tettamanzi

  20. IT University of Copenhagen, Rued Langaards Vej 7, 2300, Copenhagen, Denmark

    Julian Togelius

  21. Centre for Emergent Computing, Edinburgh Napier University, Edinburgh,, UK

    Neil Urquhart

  22. Department of Computer Engineering, Istanbul Technical University, 34490, Maslak, Turkey

    A. Şima Uyar

  23. Center For Computer Games Research, IT University Of Copenhagen, Rued Langgaards Vej 7, 2300, Copenhagen, Denmark

    Georgios N. Yannakakis

Rights and permissions

Reprints and permissions

Copyright information

© 2012 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Sherry, D., Veeramachaneni, K., McDermott, J., O’Reilly, UM. (2012). Flex-GP: Genetic Programming on the Cloud. In: Di Chio, C., et al. Applications of Evolutionary Computation. EvoApplications 2012. Lecture Notes in Computer Science, vol 7248. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-642-29178-4_48

Download citation

  • DOI: https://doi.org/10.1007/978-3-642-29178-4_48

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-642-29177-7

  • Online ISBN: 978-3-642-29178-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation