Abstract
Combinatorial interaction testing (CIT) is an emerging testing technique that has proved to be effective in finding faults due to the interaction among inputs. Efficient test generation for CIT is still an open problem especially when applied to real models having meaningful size and containing many constraints among inputs. In this paper we present a novel technique for the automatic generation of compact test suites starting from models containing constraints given in general form. It is based on the use of Multivalued Decision Diagrams (MDDs) which prove to be suitable to efficiently support CIT. We devise and experiment several optimizations including a novel variation of the classical greedy policy normally used in similar algorithms. The results of a thorough comparison with other similar techniques are presented and show that our approach can provide several advantages in terms of applicability, test suite size, generation time, and cost.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Advanced Combinatorial Testing System (ACTS), http://csrc.nist.gov/groups/SNS/acts/
Arcaini, P., Gargantini, A., Vavassori, P.: Validation of models and tests for constrained combinatorial interaction testing. In: The 3rd International Workshop on Combinatorial Testing (IWCT 2014) In conjunction with International Conference on Software Testing ICSTW, pp. 98–107. IEEE (2014)
Babar, J., Miner, A.: Meddly: Multi-terminal and edge-valued decision diagram library. In: 7th International Conference on the Quantitative Evaluation of Systems. IEEE (2010)
Brownlie, R., Prowse, J., Phadke, M.: Robust testing of AT&T PMX/starMAIL using OATS. AT&T Technical Journal 71(3), 41–47 (1992)
Bryce, R.C., Colbourn, C.J.: Prioritized interaction testing for pair-wise coverage with seeding and constraints. Information & Software Technology 48(10), 960–970 (2006)
Bryce, R.C., Colbourn, C.J.: One-test-at-a-time heuristic search for interaction test suites. In: Proceedings of the 9th Annual Conference on Genetic and Evolutionary Computation, GECCO 2007, pp. 1082–1089. ACM, New York (2007)
Bryce, R.C., Colbourn, C.J., Cohen, M.B.: A framework of greedy methods for constructing interaction test suites. In: ICSE 2005: Proc. of the 27th Int. Conf. on Software Engineering, pp. 146–155. ACM, New York (2005)
Calvagna, A., Gargantini, A.: A formal logic approach to constrained combinatorial testing. Journal of Automated Reasoning 45(4), 331–358 (2010)
Calvagna, A., Gargantini, A.: T-wise combinatorial interaction test suites construction based on coverage inheritance. Software Testing, Verification and Reliability 22(7), 507–526 (2012)
Calvagna, A., Gargantini, A., Vavassori, P.: Combinatorial interaction testing with CitLab. In: Sixth IEEE International Conference on Software Testing, Verification and Validation - Testing Tool Track (2013)
Cohen, D.M., Dalal, S.R., Fredman, M.L., Patton, G.C.: The AETG system: An approach to testing based on combinatorial design. IEEE Transactions On Software Engineering 23(7), 437–444 (1997)
Cohen, M., Dwyer, M., Shi, J.: Constructing interaction test suites for highly-configurable systems in the presence of constraints: A greedy approach. IEEE Trans. on Software Engineering 34(5), 633–650 (2008)
Covering Arrays by Simulated Annealing, http://cse.unl.edu/citportal/tools/casa/
Garvin, B.J., Cohen, M.B., Dwyer, M.B.: An improved meta-heuristic search for constrained interaction testing. In: Proceedings of the 2009 1st International Symposium on Search Based Software Engineering, SSBSE 2009, pp. 13–22. IEEE Computer Society, Washington, DC (2009)
Garvin, B.J., Cohen, M.B., Dwyer, M.B.: Evaluating improvements to a meta-heuristic search for constrained interaction testing. Empirical Software Engineering 16(1), 61–102 (2011)
Grindal, M., Offutt, J., Andler, S.F.: Combination testing strategies: a survey. Softw. Test, Verif. Reliab. 15(3), 167–199 (2005)
Hadzic, T., Hansen, E.R.: On automata, MDDs and BDDs in constraint satisfaction. In: Proceedings of the ECAI 2008 Workshop on Inference Methods based on Graphical Structures of Knowledge (2008)
Kuhn, D.R., Reilly, M.J.: An investigation of the applicability of design of experiments to software testing. In: Society, I. (ed.) 27th NASA/IEEE Software Engineering Workshop, pp. 91–95 (2002)
Kuhn, D.R., Wallace, D.R., Gallo, A.M.: Software fault interactions and implications for software testing. IEEE Trans. Software Eng. 30(6), 418–421 (2004)
Kuhn, R., Kacker, R., Lei, Y., Hunter, J.: Combinatorial software testing. Computer 42(8), 94–96 (2009)
Lei, Y., Kacker, R., Kuhn, D.R., Okun, V., Lawrence, J.: IPOG/IPOG-D: efficient test generation for multi-way combinatorial testing. Software Testing, Verification and Reliability 18(3), 125–148 (2008)
Nagayama, S., Sasao, T.: Compact representations of logic functions using heterogeneous MDDs. In: Proceedings of 33rd International Symposium on Multiple-Valued Logic, pp. 247–252 (2003)
Nie, C., Leung, H.: A survey of combinatorial testing. ACM Comput. Surv. 43(2), 11 (2011)
Pairwise web site, http://www.pairwise.org/
Salecker, E., Reicherdt, R., Glesner, S.: Calculating prioritized interaction test sets with constraints using binary decision diagrams. In: Proceedings of IEEE Fourth International Conference on Software Testing, Verification and Validation Workshops, pp. 278–285. IEEE Computer Society (2011)
Segall, I., Tzoref-Brill, R., Farchi, E.: Using binary decision diagrams for combinatorial test design. In: Proceedings of the 2011 International Symposium on Software Testing and Analysis, ISSTA 2011, pp. 254–264. ACM, New York (2011)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this paper
Cite this paper
Gargantini, A., Vavassori, P. (2014). Efficient Combinatorial Test Generation Based on Multivalued Decision Diagrams. In: Yahav, E. (eds) Hardware and Software: Verification and Testing. HVC 2014. Lecture Notes in Computer Science, vol 8855. Springer, Cham. https://doi.org/10.1007/978-3-319-13338-6_17
Download citation
DOI: https://doi.org/10.1007/978-3-319-13338-6_17
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-13337-9
Online ISBN: 978-3-319-13338-6
eBook Packages: Computer ScienceComputer Science (R0)