Abstract
Software process assessments have become big business worldwide. They can be a powerful tool for initiating and sustaining software process improvement (SPI). However, SPI programs sometimes fail. Moreover there still are very few systematic empirical investigations about the conditions under which SPI initiatives vary in their outcomes. In this paper we present the results of a study of factors that influence the success of SPI. The data come from a sample survey of organizations that have performed assessments based on the capability maturity model for software, and was conducted from 1 to 3 years after the assessments (sufficient time had passed for changes to have taken place). The results consist of a multivariate model of the conditions (e.g., how the improvement efforts are organized and funded) that can explain the successes and failures of SPI efforts. The model is constructed using a classification tree algorithm. It identifies the most important factors that affect the outcome of SPI efforts, and describes how those factors interact with each other to influence success or failure.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Breiman, L., Friedman, J., Olshen, R., and Stone, C. 1984. Classification and Regression Trees. Wadsworth and Brooks Cole.
Carmines, E., and Zeller, R. 1979. Reliability and Validity Assessment. Sage Publications.
Chapman, D. 1983. The impact of substitution on survey estimates In: W. Madow, I. Olkin and D. Rubin (eds.): Incomplete Data in Sample Surveys. Vol. 2. Academic Press.
Comrey, A. 1973. A First Course on Factor Analysis. Academic Press.
Cronbach, L. 1951. Coefficient alpha and the internal structure of tests. Psychomterika 16(3): 297-334.
Deephouse, C., Mukhopadhyay, T., Goldenson, D., and Kellner, M. 1995–1996. Software processes and project performance. Journal of Management Information Systems 12(3).
Dion, R. 1999. Starting the climb towards the CMM level 2 plateau. In: K. El-Emam and N. H. Madhavji (eds.): Elements of Software Process Assessment and Improvement, IEEE CS Press.
El-Emam, K., and Birk, A. 2000a. Validating the ISO/IEC 15504 measures of software development process capability. Journal of Systems and Software 51(2): 119-149.
El-Emam, K., and Birk, A. 2000b. Validating the ISO/IEC 15504 measures of software requirements analysis process capability. IEEE Transactions on Software Engineering 26(6): 541-566.
El-Emam, K., Smith, B., and Fusaro, P. 1999. Success factors and barriers in software process improvement: An empirical study. In: R. Messnarz and C. Tully (eds.): Better Software Practice for Business Benefit: Principles and Experiences. IEEE CS Press.
Fowler, P., and Rifkin, S. 1990. Software Engineering Process Group Guide. Software Engineering Institute, CMU/SEI-90-TR-24.
Goldenson, D. R., and Herbsleb, J. 1995. After the Appraisal: A Systematic Survey of Process Improvement, its Benefits, and Factors that Influence Success. Software Engineering Institute, CMU/SEI-95-TR-009.
Goldenson, D., El-Emam, K., Herbsleb, J., and Deephouse, C. 1999. Empirical studies of software process assessment methods. In: K. El-Emam and N. H. Madhavji (eds.): Elements of Software Process Assessment and Improvement. IEEE CS Press.
Herbsleb, J., and Goldenson, D. 1996. A Systematic Survey of CMM Experience and Results. Proceedings of the International Conference on Software Engineering. pp. 25-30.
Herbsleb, J., Carleton, A., Rozum, J., Siegel, J., and Zubrow, D. 1994. Benefits of CMM-based Software Process Improvement: Initial Results. Software Engineering Institute, CMU-SEI-94-TR-13.
Herbsleb, J., Zubrow, D., Goldsenson, D., Hayes, W., and Paulk, M. 1997. Software quality and the capability maturity model. Communications of the ACM 40(6): 30-40.
Humphrey, W., and Sweet, W. 1987. A Method for Assessing the Software Engineering of Contractors. Software Engineering Institute, CMU-SEI-87-TR-0023.
Jones, C. 1999. The economics of software process improvements. In: K. El-Emam and N. H. Madhavji (eds.): Elements of Software Process Assessment and Improvement. IEEE CS Press.
Kim, J., and Mueller, C. 1978. Factor Analysis: Statistical Methods and Practical Issues. Sage Publications.
Krasner, H. 1999. The payoff for software process improvement: What it is and how to get it. In: K. El-Emam and N. H. Madhavji (eds.): Elements of Software Process Assessment and Improvement. IEEE CS Press.
Lanubile, F., and Visaggio, G. 1997. Evaluating predictive quality models derived from software measures: Lessons learned. Journal of Systems and Software 38: 225-234.
Lawlis, P., Flowe, R., and Thordahl, J. 1995. A correlational study of the CMM and software development performance. CrossTalk 21-25.
Little, R., and Rubin, D. 1987. Statistical Analysis With Missing Data. John Wiley & Sons.
Maher, J., and Gremba, J. 1994. Organizational Barriers to PI and Technology Transition. Proceedings of the 1994 SEI Software Engineering Symposium.
McFeeley, B. 1996. IDEAL: A User's Guide for Software Process Improvement. Software Engineering Institute, Handbook CMU-SEI-96-HB-001.
Miller, M., and Goldenson, D. 1992. Software Engineering Process Groups: Results of the 1992 SEPG Workshop and a First Report on SEPG Status. Software Engineering Institute, Special Report CMU/SEI-92-SR-13.
Mingers, J. 1989. An empirical comparison of selection measures for decision-tree induction. Machine Learning 3: 319-342.
Nunnally, J. 1978. Psychometric Theory. McGraw-Hill.
Puffer, J. 1999. Action planning. In: K. El-Emam and N. H. Madhavji (eds.): Elements of Software Process Assessment and Improvement. IEEE CS Press.
Quinlan, J. 1993. C4.5: Programs for Machine Learning. Morgan Kaufmann.
Sande, I. 1983. Hot-deck imputation procedures. In: W. Madow and I. Olkin (eds.): Incomplete Data in Sample Surveys. Vol. 3, Academic Press.
Schneidewind, N. 1994. Validating metrics for ensuring space shuttle flight software quality. IEEE Computer 50-57.
Spector, P. 1992. Summated Rating Scale Construction. Sage Publications.
Stelzer, D., and Mellis, W. 1998. Success factors of organizational change in software process improvement. Software Process: Improvement and Practice 4(4): 227-250.
Weiss, S., and Kulikowski, C. 1991. Computer Systems that Learn: Classification and Prediction Methods from Statistics, Neural Nets, Machine Learning, and Expert Systems. Morgan Kaufmann Publishers.
Zeller, R., and Carmines, E. 1980. Measurement in the Social Sciences. Cambridge University Press.
Author information
Authors and Affiliations
Rights and permissions
About this article
Cite this article
El-Emam, K., Goldenson, D., McCurley, J. et al. Modelling the Likelihood of Software Process Improvement: An Exploratory Study. Empirical Software Engineering 6, 207–229 (2001). https://doi.org/10.1023/A:1011487332587
Issue Date:
DOI: https://doi.org/10.1023/A:1011487332587