Skip to main content
SpringerLink
Log in

Enterprise architecture framework evaluation criteria: a literature review and artifact development

  • Original Research Paper
  • Published:
Service Oriented Computing and Applications Aims and scope Submit manuscript

Abstract

Selecting an enterprise architecture framework (EAF) that will best address the needs of their organization is a difficult task for enterprise architecture practitioners. The objective of this study is to make this difficult task easier. To do so, this research first conducts a review of the literature on EAF evaluation criteria. Findings from this review show the shortcomings of this literature, most importantly the lack of a comprehensive set of EAF criteria and adequate measures for their operationalization. Based on these findings, and using the design science research approach, this study then designs and tests an EAF evaluation artifact that identifies, elaborates and operationalizes a comprehensive set of 14 criteria. Results of the experiment that followed show that: (i) 90.87% of the criteria were perceived usable, (ii) 97.62% of them were perceived to be applicable and relevant, and (iii) 90.48% were perceived as correct. This study makes several contributions. First, it provides a much-needed and timely overview of the literature on EAF evaluation criteria. Second, this study is the first to present a comprehensive set of criteria that not only synthesizes previously proposed criteria but also includes new criteria (e.g., SOA models and usability). Third, it answers EA practitioners’ requests for a tool that is both theoretically sound and practical. Finally, and most importantly, this study is the first to propose objective measures to operationalize EAF evaluation criteria.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  1. Hadaya P, Gagnon B (2019) Succeed in your organizational transformations using a business architecture based framework. Technical report, ASATE Group, Montreal

  2. Tallon PP, Pinsonneault A (2011) Competing perspectives on the link between strategic information technology alignment and organizational agility: insights from a mediation model. MIS Q 35(2):463–486

    Article  Google Scholar 

  3. Hadaya P, Gagnon B (2017) Business architecture: the missing link in strategy formulation. Implementation and Execution, ASATE Group

  4. The Open Group (2011) TOGAF Version 9.1

  5. Lankhorst M (2009) Enterprise architecture at work: modelling, communication and analysis. Springer, Berlin

    Book  Google Scholar 

  6. Winter R, Fischer R (2006) Essential layers, artifacts, and dependencies of enterprise architecture. In: 2006 10th IEEE international enterprise distributed object computing conference workshops (EDOCW’06), p. 30

  7. Sessions R (2007) A comparison of the top four enterprise-architecture methodologies. Microsoft Developer Network Architecture Center, pp. 1–31

  8. Urbaczewski L, Mrdalj S (2006) A comparison of enterprise architecture frameworks. Inf Syst J 7(2):18–23

    Google Scholar 

  9. Chen D, Doumeingts G, Vernadat F (2008) Architectures for enterprise integration and interoperability: past, present and future. Comput Ind 59(7):647–659

    Article  Google Scholar 

  10. Cameron BH, McMillan E (2013) Analyzing the current trends in enterprise architecture frameworks. J Enterp archit 9(1):60–71

    Google Scholar 

  11. Hadaya P, Matyas-Balassy I, Marchildon P (2019) Enterprise architecture framework selection criteria: a literature review. In: Proceedings of the ninth international conference on business intelligence and technology, (Venice), p. 8

  12. Hevner AR, March ST, Park J, Ram S (2004) Design science in information systems research. MIS Q Manag Inf Syst 28(1):75–105

    Article  Google Scholar 

  13. Peffers K, Tuunanen T, Rothenberger MA, Chatterjee S (2007) A design science research methodology for information systems research. J Manag Inf Syst 24(3):45–77

    Article  Google Scholar 

  14. Rumrill PD, Fitzgerald SM, Merchant WR (2010) Using scoping literature reviews as a means of understanding and interpreting existing literature. Work (Reading, Mass.) 35(3):399–404

    Google Scholar 

  15. Arksey H, O’Malley L (2005) Scoping studies: towards a methodological framework. Int J Soc Res Methodol 8(1):19–32

    Article  Google Scholar 

  16. Abdallah S, Galal-Edeen G (2006) Towards a framework for enterprise architecture frameworks comparison and selection. In: The fourth international conference on informatics and systems

  17. Franke U, Hook D, Konig J, Lagerstrom R, Narman P, Ullberg J, Gustafsson P, Ekstedt M (2009) EAF2-A framework for categorizing enterprise architecture frameworks. In: 10th ACIS international conference on software engineering, artificial intelligences, networking and parallel/distributed computing, (Daegu, South Korea), pp. 327–332, IEEE

  18. Jonkers H, Lankhorst MM, ter Doest HWL, Arbab F, Bosma H, Wieringa RJ (2006) Enterprise architecture: management tool and blueprint for the organisation. Inf Syst Front 8(2):63

    Article  Google Scholar 

  19. Kaisler SH, Armour F, Valivullah M (2005)“Enterprise architecting: Critical problems. In: Proceedings of the 38th annual Hawaii international conference on system sciences, (Big Island, HI, USA), pp. 224b–224b, IEEE

  20. Leist S, Zellner G (2006) Evaluation of current architecture frameworks. In: Proceedings of the 2006 ACM symposium on applied computing, pp. 1546–1553, ACM

  21. Lim N, Lee T-g, Park S-g (2009) A comparative analysis of enterprise architecture frameworks based on EA quality attributes. In: 10th ACIS international conference on software engineering, artificial intelligences, networking and parallel/distributed computing, pp. 283–288, IEEE

  22. Rouhani BD, Mahrin MN, Nikpay F, Nikfard P (2013) A comparison enterprise architecture implementation methodologies. In: 2013 international conference on informatics and creative multimedia, pp. 1–6, IEEE

  23. Tang A, Han J, Chen P (2004) A comparative analysis of architecture frameworks. In: 11th Asia-pacific software engineering conference, (Busan, South Korea), pp. 640–647, IEEE

  24. Braun C, Winter R (2005) A comprehensive enterprise architecture metamodel and its implementation using a metamodeling platform. In: International workshop on enterprise modelling and information systems architectures, pp. 64–79, Ges. für Informatik

  25. Winter R, Aier S (2011) How are enterprise architecture design principles used?. In: IEEE 15th international enterprise distributed object computing conference workshops, (Helsinki), pp. 314–321, IEEE

  26. Boh WF, Yellin D (2006) Using enterprise architecture standards in managing information technology. J Manag Inf Syst 23(3):163–207

    Article  Google Scholar 

  27. Winter R, Schelp J (2008) Enterprise architecture governance: the need for a business-to-IT approach. In: Proceedings of the 2008 ACM symposium on applied computing, (Fortaleza, Brazil), pp. 548–552

  28. Sobczak A ( 2013) Methods of the assessment of enterprise architecture practice maturity in an organization. In: International conference on business informatics research, pp. 104–111, Springer

  29. Bernaert M, Poels G, Snoeck M, De Backer M (2014) Enterprise architecture for small and medium-sized enterprises: a starting point for bringing EA to SMES, based on adoption models. In: Devos J, van Landeghem H, Deschoolmeester D (eds) Information systems for small and medium-sized enterprises. Progress in IS. Springer, Berlin

    Google Scholar 

  30. Gregor S, Hevner AR (2013) Positioning and presenting design science research for maximum impact. MIS Q 37(2):337–355

    Article  Google Scholar 

  31. Steen MWA, Akehurst DH, Doest HWLter, Lankhorst MM (2004) Supporting viewpoint-oriented enterprise architecture. In: Proceedings. Eighth IEEE international enterprise distributed object computing conference, 2004. EDOC 2004., (Monterey, CA, USA), pp. 201–211, IEEE

  32. The Open Group (2019) ArchiMate 3.1 Specification

  33. Sowa J, Zachman J (1992) Extending and formalizing the framework for information systems architecture. IBM Syst J 31(3):590–616

    Article  Google Scholar 

  34. Object Management Group (OMG) (2016) OMG meta object facility (MOF) core specification

  35. Genero M, Piattini M, Calero C (2002) Empirical validation of class diagram metrics. In: ISESE 2002- proceedings, 2002 international symposium on empirical software engineering

  36. Object Management Group (OMG) (2012) Service oriented architecture modeling language (SoaML) Core Specification

  37. Blal R, Leshob A (2017) A Model-driven service specification approach from BPMN models. In: Proceedings-14th IEEE international conference on e-business engineering, ICEBE

  38. Blal R, Leshob A, Gonzalez-Huerta J, Mili H, Boubaker A (2018) From inter-organizational business process models to service-oriented architecture models. SOCA 12(3):227–245

    Article  Google Scholar 

  39. Schekkerman J (2004) How to survive in the jungle of enterprise architecture frameworks: creating or choosing an enterprise architecture framework. Trafford, Bloomington

    Google Scholar 

  40. Curtis B, Kellner MI, Over J (1992) Process modeling. Commun ACM-Special issue on analysis and modeling in software development 35:75–90

  41. Kettinger WJ, Teng JTC, Guha S (1997) Business process change: a study of methodologies, techniques, and tools. MIS Q 21(1):55–98

    Article  Google Scholar 

  42. Buckl S, Ernst AM, Lankes J, Schneider K, Schweda CM (2007) A pattern based approach for constructing enterprise architecture management information models. In: Wirtschaftinformatik Proceedings 2007, pp. 145–162

  43. ISO (2018) Ergonomics of human–system interaction (ISO 9241-11)

  44. Iivari J (2015) Distinguishing and contrasting two strategies for design science research. Eur J Inf Syst 24(1):107–115

    Article  Google Scholar 

  45. Basili VR, Rombach HD (1988) Tame project: towards improvement-oriented software environments. IEEE Trans Softw Eng 14(6):758–773

    Article  Google Scholar 

  46. Chidamber SR, Kemerer CF (1994) A metrics suite for object oriented design. IEEE Trans Softw Eng 20(6):476–493

    Article  Google Scholar 

  47. Sauro J, Kindlund E (2005) A method to standardize usability metrics into a single score. In: CHI 2005: technology, safety, community: conference proceedings-conference on human factors in computing systems, (Portland, Oregon, USA), pp. 401–409, ACM

  48. Cook TD, Campbell DT (1979) Quasi-experimentation: design and analysis issues for field settings. Houghton Mifflin, Boston

    Google Scholar 

  49. Gorkhali A, Xu LD (2017) Enterprise architecture: a literature review. J Ind Integr Manag 2(02):1750009

    Article  Google Scholar 

  50. Plessius H, Slot R, Pruijt L (2012) On the categorization and measurability of enterprise architecture benefits with the enterprise architecture value framework. Lecture notes in business information processing. Springer, Berlin, pp 79–92

    Google Scholar 

  51. Rouhani BD, Mahrin MNZ, Nikpay F, Ahmad RB, Nikfard P (2015) A systematic literature review on enterprise architecture implementation methodologies. Inf Softw Technol 62(1–20):2015

    Google Scholar 

  52. Zhang M, Chen H, Luo A (2018) A systematic review of business-it alignment research with enterprise architecture. IEEE Access 6:18933–18944

    Article  Google Scholar 

Download references

Acknowledgements

The authors would like to thank all the experts for their involvement in this research. This research was supported by the Natural Sciences and Engineering Research Council of Canada (NSERC).

Author information

Authors and Affiliations

  1. School of Management (ESG), University of Quebec at Montreal (UQAM), Montreal, Canada

    Pierre Hadaya, Abderrahmane Leshob, Philippe Marchildon & Istvan Matyas-Balassy

  2. LATECE Laboratory, University of Quebec at Montreal, Montreal (UQAM), Montreal, Canada

    Abderrahmane Leshob

Authors
  1. Pierre Hadaya
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Abderrahmane Leshob
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Philippe Marchildon
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Istvan Matyas-Balassy
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Abderrahmane Leshob.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hadaya, P., Leshob, A., Marchildon, P. et al. Enterprise architecture framework evaluation criteria: a literature review and artifact development. SOCA 14, 203–222 (2020). https://doi.org/10.1007/s11761-020-00294-x

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11761-020-00294-x

Keywords

Search

Navigation