Skip to main content
SpringerLink
Log in

An exploration of combinatorial testing-based approaches to fault localization for explainable AI

  • Published:
Annals of Mathematics and Artificial Intelligence Aims and scope Submit manuscript

Abstract

We briefly review properties of explainable AI proposed by various researchers. We take a structural approach to the problem of explainable AI, examine the feasibility of these aspects and extend them where appropriate. Afterwards, we review combinatorial methods for explainable AI which are based on combinatorial testing-based approaches to fault localization. Last, we view the combinatorial methods for explainable AI through the lens provided by the properties of explainable AI that are elaborated in this work. We pose resulting research questions that need to be answered and point towards possible solutions, which involve a hypothesis about a potential parallel between software testing, human cognition and brain capacity.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Adadi, A., Berrada, M.: Peeking inside the black-box: A survey on explainable artificial intelligence (XAI). IEEE Access 6, 52138–52160 (2018)

    Article  Google Scholar 

  2. von Ahn, L., Blum, M., Hopper, N. J., Langford, J.: CAPTCHA: Using hard AI problems for security. In: Biham, E. (ed.) Advances in Cryptology — EUROCRYPT 2003, pp 294–311. Springer, Berlin, Heidelberg (2003)

  3. Ammann, P., Offutt, J.: Introduction to Software Testing. Cambridge University Press, Cambridge (2016)

    Book  Google Scholar 

  4. Artelt, A., Hammer, B.: On the computation of counterfactual explanations–a survey. arXiv:1911.07749 (2019)

  5. Biran, O., Cotton, C.: Explanation and justification in machine learning: A survey. In: IJCAI-17 workshop on explainable AI (XAI), vol. 8, pp 8–13 (2017)

  6. Colbourn, C. J., McClary, D. W.: Locating and detecting arrays for interaction faults. J. Comb. Optim. 15(1), 17–48 (2008)

    Article  MathSciNet  Google Scholar 

  7. Došilović, F. K., Brčić, M., Hlupić, N.: Explainable artificial intelligence: A survey. In: 2018 41st International convention on information and communication technology, electronics and microelectronics (MIPRO), pp 0210–0215. IEEE (2018)

  8. Dubois, T.: No AI in cockpit anytime soon, onera, thales say. Aviation Week and Space Technology (Nov. 26) (2018)

  9. Ehsan, U., Tambwekar, P., Chan, L., Harrison, B., Riedl, M. O.: Automated rationale generation: a technique for explainable AI and its effects on human perceptions. In: Proceedings of the 24th International Conference on Intelligent User Interfaces, pp. 263–274 (2019)

  10. Ghandehari, L. S., Chandrasekaran, J., Lei, Y., Kacker, R., Kuhn, D. R.: BEN: A combinatorial testing-based fault localization tool. In: 2015 IEEE Eighth International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 1–4 (2015)

  11. Grindal, M., Offutt, J.: Input parameter modeling for combination strategies. In: Proceedings of the 25th Conference on IASTED International Multi-Conference: Software Engineering, SE’07, pp 255–260. ACTA Press, Anaheim (2007)

  12. Grochtmann, M., Grimm, K.: Classification trees for partition testing. Software Testing, Verification and Reliability 3(2), 63–82 (1993)

    Article  Google Scholar 

  13. Gunning, D.: Explainable artificial intelligence (XAI). Defense Advanced Research Projects Agency (DARPA). http://www.cc.gatech.edu/alanwags/DLAI2016/(Gunning)%20IJCAI-16%20DLAI%20WS.pdf (2017)

  14. Gunning, D., Stefik, M., Choi, J., Miller, T., Stumpf, S., Yang, G.Z.: XAI—explainable artificial intelligence. Science Robotics 4(37). https://doi.org/10.1126/scirobotics.aay7120. https://robotics.sciencemag.org/content/4/37/eaay7120 (2019)

  15. Hayhurst, K. J.: A practical tutorial on modified condition/decision coverage. National Aeronautics and Space Administration (2001)

  16. Hilton, D. J.: Conversational processes and causal explanation. Psychol. Bull. 107(1), 65 (1990)

    Article  Google Scholar 

  17. Jayaram, R., Krishnan, R.: Approaches to fault localization in combinatorial testing: A survey. In: Satapathy, S. C., Bhateja, V., Das, S. (eds.) Smart Computing and Informatics, pp 533–540. Springer Singapore, Singapore (2018)

  18. Jin, H., Tsuchiya, T.: Constrained locating arrays for combinatorial interaction testing. J. Syst. Softw. 170, 110771 (2020)

    Article  Google Scholar 

  19. Kasparov, G.: Deep thinking: where machine intelligence ends and human creativity begins. Hachette UK (2017)

  20. Kuhn, D., Kacker, R., Lei, Y.: Practical combinatorial testing. NIST Special Publication 800–142 (2010)

  21. Kuhn, D. R., Kacker, R. N., Lei, Y., Simos, D. E.: Combinatorial methods for explainable AI. In: 2020 IEEE International Conference on Software Testing, Verification and Validation Workshops (ICSTW), pp. 167–170 (2020)

  22. Li, J., Nie, C., Lei, Y.: Improved delta debugging based on combinatorial testing. In: 2012 12th International Conference on Quality Software, pp. 102–105 (2012)

  23. Lugano, G.: Virtual assistants and self-driving cars. In: 2017 15th International Conference on ITS Telecommunications (ITST), pp. 1–5 (2017)

  24. Lundberg, S. M., Lee, S. I.: A unified approach to interpreting model predictions. In: Guyon, I., Luxburg, U. V., Bengio, S., Wallach, H., Fergus, R., Vishwanathan, S., Garnett, R. (eds.) Advances in Neural Information Processing Systems, vol. 30, pp 4765–4774. Curran Associates Inc (2017)

  25. Mandel, D. R., Hilton, D. J., Catellani, P. E.: The psychology of counterfactual thinking. Routledge (2005)

  26. Miller, G. A.: The magical number seven, plus or minus two: Some limits on our capacity for processing information. Psychol. Rev. 63(2), 81 (1956)

    Article  Google Scholar 

  27. Miller, T., Howe, P., Sonenberg, L.: Explainable AI: Beware of inmates running the asylum or: How I learnt to stop worrying and love the social and behavioural sciences arXiv:1712.00547 (2017)

  28. Nie, C., Leung, H.: The minimal failure-causing schema of combinatorial testing. ACM Trans. Softw. Eng. Methodol 20(4) (2011)

  29. Niu, X., Nie, C., Leung, H., Lei, Y., Wang, X., Xu, J., Wang, Y.: An interleaving approach to combinatorial testing and failure-inducing interaction identification. IEEE Trans. Softw. Eng. 46(6), 584–615 (2020)

    Article  Google Scholar 

  30. Papadimitriou, C. H.: The euclidean travelling salesman problem is NP-complete. Theor. Comput. Sci. 4(3), 237–244 (1977)

    Article  Google Scholar 

  31. Phillips, P.J., Hahn, C.A., Fontana, P.C., Broniatowski, D.A., Przybocki, M.A.: Four principles of explainable artificial intelligence (draft). https://doi.org/10.6028/NIST.IR.8312-draft (2020)

  32. Ribeiro, M. T., Singh, S., Guestrin, C.: ”why should i trust you?”: Explaining the predictions of any classifier. In: Proceedings of the 22nd ACM SIGKDD International Conference on Knowledge Discovery and Data Mining, KDD ’16, pp 1135–1144. Association for Computing Machinery, New York (2016)

  33. Russel, S., Norvig, P.: Artificial intelligence: a modern approach. Pearson Education Limited (2013)

  34. Shahaf, D., Amir, E.: Towards a theory of AI completeness. In: AAAI Spring Symposium: Logical Formalizations of Commonsense Reasoning, pp. 150–155 (2007)

  35. Stockmeyer, L.: Planar 3-colorability is polynomial complete. ACM Sigact News 5(3), 19–25 (1973)

    Article  Google Scholar 

  36. Tjoa, E., Guan, C.: A survey on explainable artificial intelligence (XAI): Toward medical XAI. IEEE Transactions on Neural Networks and Learning Systems (2020)

  37. Wachter, S., Mittelstadt, B., Russell, C.: Counterfactual explanations without opening the black box: Automated decisions and the GDPR. Harvard Journal of Law & Technology 31(2) (2018)

  38. Wotawa, F.: On the importance of system testing for assuring safety of AI systems. In: AISafety@IJCAI (2019)

  39. Yilmaz, C., Cohen, M., Porter, A.: Covering arrays for efficient fault characterization in complex configuration spaces. IEEE Trans. Softw. Eng. 32(1), 20–34 (2006)

    Article  Google Scholar 

  40. Zeller, A., Hildebrandt, R.: Simplifying and isolating failure-inducing input. IEEE Trans. Softw. Eng. 28(2), 183–200 (2002)

    Article  Google Scholar 

  41. Zhang, Y., Chen, X.: Explainable recommendation: A survey and new perspectives. arXiv:1804.11192 (2018)

  42. Zhou, B., Khosla, A., Lapedriza, A., Oliva, A., Torralba, A.: Learning deep features for discriminative localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) (2016)

Download references

Acknowledgements

SBA Research (SBA-K1) is a COMET Centre within the framework of COMET - Competence Centers for Excellent Technologies Programme and funded by BMK, BMDW, and the federal state of Vienna. The COMET Programme is managed by FFG. Moreover, this work was performed partly under the following financial assistance award 70NANB18H207 from U.S. Department of Commerce, National Institute of Standards and Technology.

Author information

Authors and Affiliations

  1. SBA Research, Vienna, A-1040, Austria

    Ludwig Kampel & Dimitris E. Simos

  2. National Institute of Standards & Technology, Gaithersburg, MD, USA

    D. Richard Kuhn & Raghu N. Kacker

Authors
  1. Ludwig Kampel
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dimitris E. Simos
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. D. Richard Kuhn
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Raghu N. Kacker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ludwig Kampel.

Additional information

Disclaimer

Any mention of commercial products in this paper is for information only; it does not imply recommendation or endorsement by the National Institute of Standards and Technology (NIST).

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kampel, L., Simos, D.E., Kuhn, D.R. et al. An exploration of combinatorial testing-based approaches to fault localization for explainable AI. Ann Math Artif Intell 90, 951–964 (2022). https://doi.org/10.1007/s10472-021-09772-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10472-021-09772-0

Keywords

Mathematics Subject Classification 2010

Search

Navigation