Abstract
Replications are commonly considered to be important contributions to investigate the generality of empirical studies. By replicating an original study it may be shown that the results are either valid or invalid in another context, outside the specific environment in which the original study was launched. The results of the replicated study show how much confidence we could possibly have in the original study. We present a replication of a method for selecting software reliability growth models to decide whether to stop testing and release software. We applied the selection method in an empirical study, conducted in a different development environment than the original study. The results of the replication study show that with the changed values of stability and curve fit, the selection method works well on the empirical system test data available, i.e., the method was applicable in an environment that was different from the original one. The application of the SRGMs to failures during functional testing resulted in predictions with low relative error, thus providing a useful approach in giving good estimates of the total number of failures to expect during functional testing.
Similar content being viewed by others
References
Ehrlich W, Lee S, Molisanim R (1990) Applying reliability measurement: a case study. IEEE Softw 7(2):56–64
Ehrlich W, Prasanna B, Stanpfel J, Wu J (1993) Determining the cost of a stop-test decision. IEEE Softw 10(2):33–42
Fenton NE, Pfleeger SL (1997) Software metrics: a rigorous and practical approach, 2nd edn. PWS Publishing Company, Boston
Fujiwara T, Yamada S (2003) A testing-domain-dependent software reliability growth model for imperfect debugging environment and its evaluation of goodness-of-fit. Elec Commun Jap Part 3 86(1):11–18
Gaudoin O, Yang B, Xie M (2003) A simple goodness-of-fit test for the power-law process, based on the Duane plot. IEEE Trans Reliab 52(1):69–74
Goel AL, Okumoto L (1979) A time dependent error detection rate model for software reliability and other performance measures. IEEE Trans Reliab 28(3):206–211
Huang CY (2005) Cost-reliability-optimal release policy for software reliability models incorporating improvements in testing efficiency. J Syst Softw 77(2):139–155
Institute of Electrical and Electronics Engineers (1990) IEEE standard glossary of software engineering terminology, IEEE Std 610.12-1990
International Standards Organisation (2000) Information technology—software product evaluation—quality characteristics and guidelines for their use, ISO/IEC FDIS 9126-1. Geneva, Switzerland
Jeske DR, Zhang X (2005) Some successful approaches to software reliability modeling in industry. J Syst Softw 74(1):85–99
Kececioglu D (1991) Reliability engineering handbook, vol. 2. Prentice-Hall, Englewood Cliffs, NJ
Lyu MR (ed) (1996) Handbook of software reliability engineering. McGraw-Hill, New York
Miller J (2005) Replicating software engineering experiments: a poisoned chalice or the Holy Grail. Inf Softw Technol 47(4):233–244
Montgomery DC (2001) Design and analysis of experiments, 5th edn. Wiley, New York
Musa J (1999) Software reliability engineering. McGraw-Hill, New York
Musa J, Ackerman A (1989) Quantifying software validation: when to stop testing? IEEE Softw 6(3):19–27
Musa J, Iannino A, Okumoto L (1987) Software reliability measurement, prediction, application. McGraw-Hill, New York
Robson C (2002) Real world research. Blackwell Publishers, UK
Siegel S, Castellan NJ (1988) Nonparametric statistics for the behavioral sciences. McGraw-Hill, Singapore
Stringfellow C (2000) An integrated method for improving testing effectiveness and efficiency. PhD Dissertation, Colorado State University
Stringfellow C, Amschler Andrews A (2002) An empirical method for selecting software reliability growth models. Empir Softw Eng 7(4):319–343
Wood A (1996) Predicting software reliability. IEEE Comput 29(11):69–78
Wood A (1997) Software reliability growth models: assumptions vs. reality. Proceedings of the Eighth International Symposium on Software Reliability Engineering, pp136–141
Yamada S, Ohba M, Osaki S (1983) S-shaped reliability growth modeling for software error detection. IEEE Trans Reliab 32(5):475–478
Yamada S, Ohtera H, Narihisa H (1986) Software reliability growth models with testing effort. IEEE Trans Reliab 35(1):19–23
Acknowledgment
The author would like to thank Prof. Catherine Stringfellow for being generous with her time and willing to answer my questions about the selection method. Thanks also to Prof. Anneliese Amschler Andrews and Prof. Per Runeson who provided valuable comments on the paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
Editor: Pankaj Jalote
Rights and permissions
About this article
Cite this article
Andersson, C. A replicated empirical study of a selection method for software reliability growth models. Empir Software Eng 12, 161–182 (2007). https://doi.org/10.1007/s10664-006-9018-0
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10664-006-9018-0