Skip to main content
SpringerLink
Log in

A survey on formal active and passive testing with applications to the cloud

  • Published:
annals of telecommunications - annales des télécommunications Aims and scope Submit manuscript

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

References

  1. Alcalde B, Cavalli A, Chen D, Khuu D, Lee D (2004) Network protocol system passive testing for fault management: A backward checking approach. In: Proceedings of 24th IFIP Int. Conf. on Formal Techniques for Networked and Distributed Systems (FORTE’04), Lecture Notes in Computer Science, vol. 3235. Springer, Berlin, pp 150–166

  2. Alur R, Dill D (1994) A theory of timed automata. Theor Comput Sci 126:183–235

    Article  MATH  MathSciNet  Google Scholar 

  3. Andrés C, Cambronero M, Nuñez M (2010) Formal passive testing of service-oriented systems. In: Proceedings of 2010 IEEE Int. Conf. on Services Computing (SCC 2010), pp 610-613

  4. Andrés C, Merayo M, Núñez M (2008) Passive testing of timed systems. In: Proceedings of 6th Int. Symp. on Automated Technology for Verification and Analysis (ATVA 2008), Lecture Notes in Computer Science, vol.5311. Springer, Berlin, pp 418–427

  5. Andrés C, Merayo MG, Núñez M (2012) Formal passive testing of timed systems: Theory and tools. Softw Test Verification Reliab 22(6):365–405

    Article  Google Scholar 

  6. Arnedo JA, Cavalli A, Núñez M (2003) Fast testing of critical properties through passive testing. In: Proceedings of 15th IFIP Int. Conf. on Testing of Communicating Systems (TestCom 2003), Lecture Notes in Computer Science, vol. 2644. Springer, Berlin, pp 295–310

  7. Banzai T, Koizumi H, Kanbayashi R, Imada T, Hanawa T, Sato M (2010) D-cloud: Design of a software testing environment for reliable distributed systems using cloud computing technology. In: Proceedings of 10th IEEE/ACM Int. Conf.on Cluster,Cloud and Grid Computing (CCGrid’10), pp 631–636

  8. Bayse E, Cavalli A, Nez MN, Zaïdi F (2005) A passive testing approach based on invariants: Application to the wap. Comput Netw 48(2):247–266

    Article  MATH  Google Scholar 

  9. Bellard F (2005) Qemu, a fast and portable dynamic translator, a fast and portable dynamic translator. In: Proceedings of 2005 USENIX Annual Technical Conference (ATEC ’05), pp 41 41. USENIX Association

  10. Benharref A, Dssouli R, Serhani M, En-Nouaary A, Glitho R (2007) New approach for efsm-based passive testing of web services. In: Proceedings of 19th IFIP Int. Conf. on Testing of Software and Communicating Systems (TestCom 2007),Lecture Notes in Computer Science, vol. 4581. Springer, Berlin, pp 13–27

  11. http://bro-ids.org/ 2014

  12. Buyya R, Ranjan R, Calheiros RN (2009) Modeling and simulation of scalable cloud computing environments and the CloudSim toolkit: Challenges and opportunities. In: Proceedings of 7th IEEE Int. Conf. High Performance Computing and Simulation (HPCS’09), pp 1–11

  13. Cacciari L, Rafiq O (1999) Controllability and observability in distributed testing. Inf Softw Technol 41(11–12):767–780

    Article  Google Scholar 

  14. Casanova H, Legrand A, Quinson M (2008) SimGrid: a generic framework for large-scale distributed experiments. In: Proceedings of 10th Int. Conf. on Computer Modeling and Simulation (UKSIM’08), pp 126–131

  15. Cavalli A, Gervy C, Prokopenko S (2003) New approaches for passive testing using an extended finite state machine specification. Inf Softw Technol 45(12):837–852

    Article  Google Scholar 

  16. Cavalli A, Grepet C, Maag S, Tortajada V (2004) A validation model for the dsr protocol. In: Proceedings of IEEE Int. Workshop on Wireless Ad Hoc Networking (WWAN’04), pp 768–773

  17. Chan W, Mei L, Zhang Z (2009) Modeling and testing of cloud applications. In: Proceedings of 4th IEEE Int. Conf. on Asia-Pacific Services Computing (APSCC’09), pp 111–118

  18. Chow TS (1978) Test design modeled by finite-state machines. IEEE Trans Softw Eng 4(3):178–187

    Article  MATH  Google Scholar 

  19. Corwin EH (2011) Deep packet inspection: Shaping the internet and the implications on privacy and security. Inf Secur J A Glob Perspect 20(6):311–316

    Article  Google Scholar 

  20. Dorofeeva M, El-Fakih K, Yevtushenko N (2005) An improved conformance testing method. In: Proceedings of 25th IFIP Int. Conf. on Formal Techniques for Networked and Distributed Systems (FORTE’05). Springer, Berlin, pp 363–378

  21. Dorofeeva R, El-Fakih K, Maag S, Cavalli AR, Yevtushenko N (2010) Fsm-based conformance testing methods: A survey annotated with experimental evaluation. Inf Softw Technol 52:1286–1297

    Article  Google Scholar 

  22. Duale AY, Uyar MU (2004) A method enabling feasible conformance test sequence generation for efsm models. IEEE Trans Comput 53(5):614–627

    Article  Google Scholar 

  23. El-Fakih K, Yevtushenko Nv, Bochmann G (2004) Fsm-based incremental conformance testing methods. IEEE Trans Softw Eng 30(7):425–436

    Article  Google Scholar 

  24. En-Nouaary A, Dssoul R, Khendek F, Elqortobi A (1998) Timed test cases generation based on state characterization technique. In: Proceedings of 19th IEEE Real-Time System Sympsium (RTSS’98), pp 220–229, IEEE

  25. Fujiwara S v, Bochmann G, Khendek F, Amalou M, Ghedamsi A (1991) Test selection based on finite state models. IEEE Trans Softw Eng 17(6):591–603

    Article  Google Scholar 

  26. García-Guirado A, Fernández-Pascual R, García JM (2009) Virtual-GEMS: an infrastructure to simulate virtual machines. In: Proceedings of 5th annual workshop on modeling, benchmarking and simulation (MoBS’09), pp 1–10

  27. http://www.gnu.org/software/gdb/ 2014

  28. Gonenc G (1970) A method for the design of fault detection experiments. IEEE Trans Comput C-19(6):551–558

    Article  Google Scholar 

  29. Hallé S, Villemaire R (2009) Runtime monitoring of web service choreographies using streaming xml. In: 2009 ACM Symposium on Applied Computing,SAC’09, pp 2118–2125

  30. Hierons R, Merayo M, Núñez M (2013) Passive testing with asynchronous communications. In: Proceedings of 33rd IFIP Int. Conf. on Formal Techniques for Distributed Systems (FORTE’13), Lecture Notes in Computer Science, vol. 7892. Springer, Berlin, pp 99–113

  31. Hierons RM (2012) Oracles for distributed testing. IEEE Trans Softw Eng 38(3):629–641

    Article  Google Scholar 

  32. Hierons RM, Merayo MG, Núñez M (2012) Implementation relations and test generation for systems with distributed interfaces. Distrib Comput 25(1):35–62

    Article  MATH  Google Scholar 

  33. Higashino Tv, Bochmann G (1994) Automatic analysis and test case derivation for a restricted class of lotos expressions with data parameters. IEEE Trans Softw Eng 20(1):29–42

    Article  Google Scholar 

  34. Higashino T, Nakata A, Taniguchi K, Cavalli AR (1999) Generating test cases for a timed i/o automaton model. In: Proceedings of 12th IFIP Int. Workshop on Testing of Communicating Systems (IWTCS’99). Springer, Berlin, pp 197–214

  35. Hiromori A, Umedu T, Yamaguchi H, Higashino T (2012) Protocol testing and performance evaluation for manets with non-uniform node density distribution. In: Proceedings of 24th IFIP Int. Conf. on Testing Software and Systems (ICTSS 2012). Springer, Berlin, pp 231–246

  36. Jard C, Jéron T, Kahlouche H, Viho C (1998) Towards automatic distribution of testers for distributed conformance testing. In: Proceedings of Int. Conf. on Formal Description Techniques and Protocol Specification, Testing and Verification (FORTE’98/PSTV’98). Kluwer Academic Publishers, Norwell, pp 353–368

  37. Joshi P, Gunawi H, Sen K (2011) PREFAIL: a programmable tool for multiple-failure injection. ACM SIGPLAN Not 46(10):171–188

    Article  Google Scholar 

  38. Kliazovich D, amd SU, Khan PB (2012) GreenCloud: a packet-level simulator of energy-aware cloud computing data centers. J Supercomput 62(3):1263–1283

    Article  Google Scholar 

  39. Ladani B, Alcalde B, Cavalli A (2005) Passive testing: A constrained invariant checking approach. In: Proceedings of 17th IFIP Int. Conf. on Testing of Communicating Systems (TestCom 2005), Lecture Notes in Computer Science, vol.3502. Springer, Berlin, pp 9–22

  40. Lai R (2002) A survey of communication protocol testing. J Syst Softw 62:21–46

    Article  Google Scholar 

  41. Lalanne F, Maag S (2013) A formal data-centric approach for passive testing of communication protocols. IEEE/ACM Trans Netw 21(3):788–801

    Article  Google Scholar 

  42. Lee D, Chen D, Hao R, Miller R, Wu J, Yin X (2002) A formal approach for passive testing of protocol data portions. In: Proceedings of 10th IEEE Int. Conf. on Network Protocols (ICNP 2002), pp 122–131

  43. Lee D, Chen D, Hao R, Miller R, Wu J, Yin X (2006) Network protocol system monitoring-a formal approach with passive testing. IEEE/ACM Trans Netw 14(2):424–437

    Article  Google Scholar 

  44. Lee D, Netravali A, Sabnani K, Sugla B, John A (1997) Passive testing and applications to network management. In: Proceedings of 5th IEEE Int. Conf. on Network Protocols (ICNP’97), pp 113–122

  45. Lee D, Yannakakis M (1996) Principles and methods of testing finite state machines-a survey. Proc IEEE 84(8):1090–1123

    Article  Google Scholar 

  46. Leucker M, Schallhart C (2009) A brief account of runtime verification. J Logic Algebraic Program 78(5):293–303

    Article  MATH  Google Scholar 

  47. Li X, Higashino T, Higuchi M, Taniguchi K (1994) Automatic generation of extended uio sequences for communication protocols in an efsm model. In: Proceedings of 7th IFIP Int. Workshop on Protocol Test Systems (IWPTS’94). Springer, Berlin, pp 225–240

  48. Luo G, Dssouli R, Bochmann Gv (1993) Generating synchronizable test sequences based on finite state machine with distributed ports. In: Proceedings of 6th IFIP Int. Workshop on Protocol Test Systems (IWPTS’93), pp 139–153. North-Holland

  49. Maag S, Zaidi F (2006) Testing methodology for an ad hoc routing protocol. In: Proceedings of 2006 ACM Int. Workshop on Performance Monitoring,Measurement, and Evaluation of Heterogeneous Wireless and Wired Networks, pp 48–55

  50. Mammar A, Cavalli A, Jimenez W, Mallouli W, Montes E (2011) Using testing techniques for vulnerability detection in c programs. In: Proceedings of 23rd IFIP Int. Conf. on Testing Software and Systems (ICTSS’11), Lecture Notes in Computer Science, vol. 7019. Springer, Berlin, pp 80–96

  51. Mehrotra N (2010) Cloud-testing vs. testing a cloud. In: Proceedings 10th Int. Software Testing Conf., pp 1–7

  52. Mell P, Scarfone K (2007) Guide to intrusion detection and prevention systems (IDPS). http://csrc.nist.gov/publications/nistpubs/800-94/SP800-94.pdf

  53. Miller R (1998) Passive testing of networks using a cfsm specification. In: Proceedings of 1998 IEEE Int. Conf. on Performance Computing and Communications (IPCCC ’98), pp 111–116

  54. Miller R, Arisha K (2000) On fault location in networks by passive testing. In: Proceedings of 2000 IEEE Int. Conf. on Performance, Computing, Communications (IPCCC ’00), pp 281–287

  55. Miller R, Arisha K (2001) Fault identification in networks by passive testing. In: Proceedings of 34th Simulation Symp., pp 277–284

  56. Morales G, Maag S, Cavalli A, Mallouli W, Montes E, Wehbi B (2010) Timed extended invariants for the passive testing of web services. In: Proceedings of 2010 IEEE Int. Conf. on Web Services (ICWS 2010), pp 592–599

  57. Mouttappa P, Maag S, Cavalli A (2012) An iosts based passive testing approach for the validation of data-centric protocols. In: Proceedings of 12th Int. Conf. on Quality Software (QSIC 2012), pp 49–58

  58. Mouttappa P, Maag S, Cavalli A (2013) Monitoring based on iosts for testing functional and security properties: Application to an automotive case study. In: Proceedings of 37th IEEE Int. Conf. on Computer Software and Applications (COMPSAC ’13), pp 1–10

  59. Mouttappa P, Maag S, Cavalli A (2013) Using passive testing based on symbolic execution and slicing techniques: Application to the validation of communication protocols. Comput Netw 57(15):2992–3008

    Article  Google Scholar 

  60. Nguyen H, Poizat P, Zaidi F (2012) Online verification of value-passing choreographies through property-oriented passive testing. In: Proceedings of 14th IEEE Int. Symp. on High-Assurance Systems Engineering (HASE 2012), pp 106–113

  61. Núñez A, Vazquez-Polettí JL, Caminero AC, Castañé GG, Carretero J, Llorente IM (2012) iCanCloud: a flexible and scalable cloud infrastructure simulator. J Grid Comput 10(1):185–209

    Article  Google Scholar 

  62. Petrenko A, Yevtushenko N, Lebedev A, Das A (1993) Nondeterministic state machines in protocol conformance testing. In: Proceedings of 6th IFIP Int. Workshop on Protocol Test Systems (IWPTS’93), pp 363–378.North-Holland

  63. Rafiq O, Cacciari L (2003) Coordination algorithm for distributed testing. J Supercomput 24(2):203–211

    Article  MATH  Google Scholar 

  64. Richters M, Gogolla M (2003) Aspect-oriented monitoring of UML and OCL constraints. In: 4th Int. Workshop on Aspect-Oriented Modeling with UML on 6th Int. Conf. on the Unified Modeling Language (UML’03)

  65. Rumbaugh J, Blaha M, Premerlani W, Eddy F, Lorensen WE, et al. (1991) Object-oriented modeling and design. Prentice-Hall Englewood Cliffs, NJ

    Google Scholar 

  66. Cheung SC, STC, Xu Z (2001) Toward generic timing tests for distributed multimedia software systems. In: Proceedings of 12th, I E E E Int. Symp. on Software Reliability Engineering (ISSRE’01), pp 210–220

  67. Sabnani K, Dahbura A (1988) A protocol test generation procedure. Comput Netw ISDN Syst 15(4):285–297

    Article  Google Scholar 

  68. Shin KS, Jung JH, Cheon JY, Choi SB (2007) Real-time network monitoring scheme based on SNMP for dynamic information. J Netw Comput Appl 30(1):331–353

    Article  Google Scholar 

  69. Sidhu DP, Leung TK (1989) Formal methods for protocol testing: A detailed study. IEEE Trans Softw Eng 15(4):413–426

    Article  Google Scholar 

  70. Simmonds J (2011) Dynamic analysis of web services.Ph.D. thesis, University of Toronto

  71. http://www.snort.org/ 2015

  72. Springintveld J, Vaandrager F, D’Argenio PR (2001) Testing timed automata. Theor Comput Sci 254(1-2):225–257

    Article  MATH  MathSciNet  Google Scholar 

  73. Sun T, Yasumoto K, Mori M, Higashino T (2003) Qos functional testing for multi-media systems. In: Proceedings of 23rd IFIP Int. Conf. on Formal Techniques for Networked and Distributed Systems (FORTE’03). Springer, Berlin, pp 319–334

  74. Tabourier M, Cavalli A (1999) Passive testing and application to the gsm-map protocol. Inf Softw Technol 41(11-12):813–821

    Article  Google Scholar 

  75. Ural H, Williams C (2006) Constructing checking sequences for distributed testing. Form Asp Comput 18(1):84–101

    Article  MATH  Google Scholar 

  76. Ural H, Xu Z (2007) An efsm-based passive fault detection approach. In: Proceedings of 19th IFIP Int. Conf. on Testing of Software and Communicating Systems (TestCom2007),Lecture Notes in Computer Science, vol 4581. Springer, Berlin, pp 335–350

  77. Ural H, Xu Z, Zhang F (2007) An improved approach to passive testing of fsm-based systems. In: Proceedings of 2nd IEEE Int. Workshop on Automation of Software Test (AST ’07), p 6

  78. Misic V, STC, Cheung SC (1998) Towards a framework for testing distributed multimedia software systems. In: Proceedings of 1998 IEEE Int. Symp. on Software Engineering for Parallel and Distributed Systems (PDSE’98), pp 72 81

  79. http://valgrind.org/ 2014

  80. Vuong ST, Chan W, Ito M (1989) The uiov-method for protocol test sequence generation. In: Proceedings of 2nd IFIP Int. Workshop on Protocol Test Systems (IWPTS’89), pp 161–175. North-Holland

  81. Wang CJ, Liu MT (1992) Axiomatic test sequence generation for extended finite state machines. In: Proceedings of 12th IEEE Int. Conf. on Distributed Computing Systems (ICDCS-12), pp 252–259

  82. Wehbi B, Montes E, Bourdelles M (2012) Events-based security monitoring using mmt tool. In: Proceedings of 5th IEEE Int. Conf. on Software Testing,Verification and Validation (ICST2012), pp 860–863

  83. Wu J, Zhao Y, Yin X (2002) 22nd IFIP Int. Conf. on Formal Techniques for Networked and Distributed Systems (FORTE 2002). Springer, Berlin, pp 101–116

    Google Scholar 

  84. Xiaoping C, Lalanne F, Maag S (2012) A logic-based passive testing approach for the validation of communicating protocols. In: Proceedings of 7th Int. Conf. on Evaluation of Novel Approaches to Software Engineering (ENASE’12), pp 53–64

  85. Zeng W, Wang Y (2009) Design and implementation of server monitoring system based on snmp. In: Proceedings of 2009 Conf. on Artificial Intelligence (JCAI ’09), pp 680–682

  86. Zhao Y, Yin X, Wu J (2003) Problems in the information dissemination of the internet routing. J Comput Sci Technol 18(2):139–152

    Article  MATH  MathSciNet  Google Scholar 

Download references

Acknowledgements

This research has been partially supported by the Spanish MEC project ESTuDIo (TIN2012-36812-C02), the Comunidad de Madrid project SICOMORo-CM (S2013/ICE-3006), the Japanese Grants-in-Aid for Scientific Research(S) (26220001), the UCM-Santander program to fund research groups (group 910606), and the INTER-TRUST european project (FP7 program).

Author information

Authors and Affiliations

  1. Télécom SudParis, 9 Rue Charles Fourier, 91000, Evry, France

    Ana R. Cavalli

  2. Graduate School of Information Science and Technology, Osaka University, Osaka, Japan

    Teruo Higashino

  3. Departamento de Sistemas Informáticos y Computación, Universidad Complutense de Madrid, Madrid, Spain

    Manuel Núñez

Authors
  1. Ana R. Cavalli
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Teruo Higashino
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Manuel Núñez
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ana R. Cavalli.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cavalli, A.R., Higashino, T. & Núñez, M. A survey on formal active and passive testing with applications to the cloud. Ann. Telecommun. 70, 85–93 (2015). https://doi.org/10.1007/s12243-015-0457-8

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12243-015-0457-8

Keywords

Search

Navigation