Skip to main content
SpringerLink
Log in

Statistical Packages for Diagnostic Meta-Analysis and Their Application

  • Chapter
  • First Online:
Diagnostic Meta-Analysis

Abstract

The bivariate model has become a de facto standard in diagnostic meta-analysis. Complex iterative algorithms are needed to fit the model, and thus a meta-analysis of diagnostic accuracy data is much aided by appropriate software packages. Also, graphical methods ease exploration, interpretation, and communication in the context of a diagnostic meta-analysis. This chapter reviews existing software and discusses the relative merits of general packages and specialized packages for DTA meta-analysis. The use of software for diagnostic meta-analysis and especially fitting the bivariate model is illustrated with a sample workflow in the open-source statistical framework R. Some ways to extend the bivariate model and software for the case of multiple cutoff values per primary study are discussed.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 129.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 169.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 169.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    We mention in passing that RevMan can plot SROC curves if suitable output is supplied from other packages.

  2. 2.

    Example code for other packages can typically be found in the references in Table 12.1 or in the technical documentation.

  3. 3.

    The package can be installed by typing install.packages(“mada”) at an R-prompt. After this, the package only needs to load once in an R session with library(mada). The most current (development) version of mada is found at http://r-forge.r-project.org/projects/mada/. Some additional functionality of mada is explained in the package vignette that is automatically installed with the package and can be accessed by typing vignette(“mada”) at an R-prompt.

  4. 4.

    Note that the subset function is a convenient way in R to form subsets.

  5. 5.

    Note that for special cases like a binary covariate, plotting SROC curves for the parameters corresponding to each of both levels of the covariates is meaningful. For an example, see Meyer, Frings, Rücker, and Hellwig [68].

  6. 6.

    Note that brms’s syntax is very similar to lme4’s so that the sample code below can be adapted. For similar lme4 code, also consult Partlett and Takwoingi [24].

  7. 7.

    CAMAN is also the backbone for the implementation of the semiparametric mixture approach of Schlattmann, Verba, Dewey, and Walther [69], which extends the bivariate model.

  8. 8.

    At the time of writing, code is found on V. Dukic’s homepage: http://amath.colorado.edu/faculty/vdukic/software/ROC.html

References

  1. The Nordic Cochrane Centre. Review manager (RevMan) [Computer Program], version 5.3. The Cochrane Collaboration, Copenhagen; 2014.

    Google Scholar 

  2. Schwarzer G. meta: an R package for meta-analysis. R News. 2007;7:40–5.

    Google Scholar 

  3. Glas A, Lijmer J, Prins M, Bonsel G, Bossuyt P. The diagnostic odds ratio: a single indicator of test performance. J Clin Epidemiol. 2003;56:1129–35.

    Article  PubMed  Google Scholar 

  4. Reitsma J, Glas A, Rutjes A, Scholten R, Bossuyt P, Zwinderman A. Bivariate analysis of sensitivity and specificity produces informative summary measures in diagnostic reviews. J Clin Epidemiol. 2005;58:982–90.

    Article  PubMed  Google Scholar 

  5. Leeflang M, Deeks J, Gatsonis C, Bossuyt P. Systematic reviews of diagnostic test accuracy. Ann Intern Med. 2008;149:889–97.

    Article  PubMed  PubMed Central  Google Scholar 

  6. Macaskill P, Gatsonis C, Deeks J, Harbord R, Takwoingi Y. Chapter 10: analysing and presenting results. In: Deeks J, Bossuyt P, Gatsonis C, editors. Cochrane handbook for systematic reviews of diagnostic test accuracy version 1.0. The Cochrane Collaboration; 2010. Available from http://srdta.cochrane.org/. Accessed 29 June 2018.

  7. Rutter C, Gatsonis C. A hierarchical regression approach to meta-analysis of diagnostic test accuracy evaluations. Stat Med. 2001;20:2865–84.

    Article  CAS  PubMed  Google Scholar 

  8. Harbord R, Deeks J, Egger M, Whiting P, Sterne J. A unification of models for meta-analysis of diagnostic accuracy studies. Biostatistics. 2007;8:239–51.

    Article  PubMed  Google Scholar 

  9. DerSimonian R, Laird N. Meta-analysis in clinical trials. Control Clin Trials. 1986;7:177–88.

    Article  CAS  Google Scholar 

  10. Viechtbauer W. Conducting meta-analyses in R with the metafor package. J Stat Softw. 2010;36:1–48. Available from http://www.jstatsoft.org/v36/i03/. Accessed 29 June 2018.

  11. Gasparrini A, Armstrong B, Kenward M. Multivariate meta-analysis for non-linear and other multi-parameter associations. Stat Med. 2012;31:3821–39.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  12. R Core Team. R: a language and environment for statistical computing. Vienna: R Foundation for Statistical Computing; 2017. Available from https://www.R-project.org/. Accessed 29 June 2018.

  13. Schwarzer G, Carpenter J, Rücker G. Meta-analysis with r. New York: Springer; 2015. UseR!

    Book  Google Scholar 

  14. Liu Z, Yao Z, Li C, Liu X, Chen H, Gao C. A step-by-step guide to the systematic review and meta-analysis of diagnostic and prognostic test accuracy evaluations. Br J Cancer. 2013;108:2299–303.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  15. Kim K, Lee J, Choi S, Huh J, Park S. Systematic review and meta-analysis of studies evaluating diagnostic test accuracy: a practical review for clinical researchers-part I. General guidance and tips. Korean J Radiol. 2015;16:1175–87.

    Article  PubMed  PubMed Central  Google Scholar 

  16. Lee J, Kim K, Choi S, Huh J, Park S. Systematic review and meta-analysis of studies evaluating diagnostic test accuracy: a practical review for clinical researchers-part II. Statistical methods of meta-analysis. Korean J Radiol. 2015;16:1188–96.

    Article  PubMed  PubMed Central  Google Scholar 

  17. Nikoloulopoulos A. CopulaREMADA: copula mixed effect models for bivariate and trivariate meta-analysis of diagnostic test accuracy studies. R package version 1.0. 2016. Available from https://CRAN.R-project.org/package=CopulaREMADA. Accessed 29 June 2018.

  18. Schiller I, Dendukuri N. HSROC: joint meta-analysis of diagnostic test sensitivity and specificity with or without a gold standard reference test. R package version 2.1.8; 2015.

    Google Scholar 

  19. Verde PE. bamdit: bayesian meta-analysis of diagnostic test data. R package version 3.1.0. 2017. Available from https://CRAN.R-project.org/package=bamdit. Accessed 29 June 2018.

  20. Lunn D, Thomas A, Best N, Spiegelhalter D. WinBUGS—a Bayesian modelling framework: concepts, structure, and extensibility. Stat Comput. 2000;10:325–37.

    Article  Google Scholar 

  21. Lunn D, Spiegelhalter D, Thomas A, Best N. The BUGS project: evolution, critique and future directions. Stat Med. 2009;28:3049–67.

    Article  PubMed  Google Scholar 

  22. Plummer M. rjags: Bayesian graphical models using MCMC. R package version 4-6. 2016. Retrieved from https://CRAN.R-project.org/package=rjags. Accessed 29 June 2018.

  23. Bürkner P-C. brms: an R package for Bayesian multilevel models using Stan. J Stat Softw. 2017;80:1–28.

    Article  Google Scholar 

  24. Partlett C, Takwoingi Y. Meta-analysis of test accuracy studies in R: a summary of user-written programs and step-by-step guide to using glmer. Version 1.0. 2016. Available from http://methods.cochrane.org/sdt/. Accessed 29 June 2018.

  25. Doebler P. mada: meta-analysis of diagnostic accuracy. R package version 0.5.7. 2015. Available from https://CRAN.R-project.org/package=mada. Accessed 29 June 2018.

  26. Guo J, Riebler A. meta4diag: meta-analysis for diagnostic test studies. R package version 2.0.5. 2016. Available from https://CRAN.R-project.org/package=meta4diag. Accessed 29 June 2018.

  27. Huang H. Metatron: meta-analysis for classification data and correction to imperfect reference. R package version 0.1-1. 2014. Available from https://CRAN.R-project.org/package=Metatron. Accessed 29 June 2018.

  28. Botella J, Huang H, Suero M. Multinomial tree models for assessing the status of the reference in studies of the accuracy of tools for binary classification. Front Psychol. 2013;4:694.

    Article  PubMed  PubMed Central  Google Scholar 

  29. Charlton C, Rasbash J, Browne W, Healy M, Cameron, B. Mlwin [computer program] version 3.00. Centre for Multilevel Modelling, University of Bristol; 2017.

    Google Scholar 

  30. Chu H, Cole S. Bivariate meta-analysis of sensitivity and specificity with sparse data: a generalized linear mixed model approach. J Clin Epidemiol. 2006;59:1331–2.

    Article  PubMed  Google Scholar 

  31. Arends L, Hamza T, Van Houwelingen J, Heijenbrok-Kal M, Hunink M, Stijnen T. Bivariate random effects meta-analysis of ROC curves. Med Decis Mak. 2008;28:621–38.

    Article  CAS  Google Scholar 

  32. Menke J. Bivariate random-effects meta-analysis of sensitivity and specificity with sas proc glimmix. Methods Inf Med. 2010;49:54–64.

    PubMed  CAS  Google Scholar 

  33. Takwoingi Y, Deeks J. METADAS: an SAS macro for meta-analysis of diagnostic accuracy studies, version 1.3. Computer program; 2011.

    Google Scholar 

  34. Rabe-Hesketh S, Skrondal A, Pickles A. Generalized multilevel structural equation modeling. Psychometrika. 2004;69:167–90.

    Article  Google Scholar 

  35. Harbord R, Whiting P. Metandi: meta-analysis of diagnostic accuracy using hierarchical logistic regression. Stata J. 2010;9:211–29.

    Google Scholar 

  36. Takwoingi Y. Meta-analysis of test accuracy studies in Stata: a bivariate model approach. Version 1.1. 2016. Available from http://methods.cochrane.org/sdt/. Accessed 29 June 2018.

  37. Dwamena B. midas: Stata module for meta-analytical integration of diagnostic accuracy studies. 2007. Available from: http://econpapers.repec.org/software/bocbocode/s456880.htm. Accessed 29 June 2018.

  38. Patrick D, Cheadle A, Thompson D, Diehr P, Koepsell T, Kinne S. The validity of self-reported smoking: a review and meta-analysis. Am J Public Health. 1994;84:1086–93.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  39. Wickham H, Bryan J. readxl: read excel files. R package version 1.0.0. 2017. Available from https://CRAN.R-project.org/package=readxl. Accessed 29 June 2018.

  40. Phillips B, Stewart L, Sutton A. ‘cross hairs’ plots for diagnostic meta-analysis. Res Synth Methods. 2010;1:308–15.

    Article  PubMed  Google Scholar 

  41. Vogelgesang F, Schlattmann P, Dewey M. The evaluation of bivariate mixed models in meta-analyses of diagnostic accuracy studies with SAS, Stata and R. Methods Inf Med. 2018;57:111–9.

    Article  PubMed  Google Scholar 

  42. Doebler P, Holling H, Böhning D. A mixed model approach to meta-analysis of diagnostic studies with binary test outcome. Psychol Methods. 2012;17:418–36.

    Article  PubMed  Google Scholar 

  43. Karrasch S, Linde K, Rücker G, Sommer H, Karsch-Völk M, Kleijnen J, et al. Accuracy of FENO for diagnosing asthma: a systematic review. Thorax. 2017;72:109–16.

    Article  PubMed  Google Scholar 

  44. Brown H, Prescott R. Applied mixed models in medicine. 3rd ed. Hoboken: John Wiley & Sons; 2015.

    Google Scholar 

  45. Demidenko E. Mixed models: theory and applications. Hoboken: John Wiley & Sons; 2013.

    Google Scholar 

  46. Chu H, Nie L, Cole S, Poole C. Meta-analysis of diagnostic accuracy studies accounting for disease prevalence: alternative parameterizations and model selection. Stat Med. 2009;28:2384–99.

    Article  PubMed  Google Scholar 

  47. Hoyer A, Kuss O. Meta-analysis of diagnostic tests accounting for disease prevalence: a new model using trivariate copulas. Stat Med. 2015;34:1912–24.

    Article  CAS  PubMed  Google Scholar 

  48. Nikoloulopoulos A. A vine copula mixed effect model for trivariate meta-analysis of diagnostic test accuracy studies accounting for disease prevalence. Stat Methods Med Res. 2017;26:2270–86.

    Article  PubMed  Google Scholar 

  49. Bates D, Mächler M, Bolker B, Walker S. Fitting linear mixed-effects models using lme4. J Stat Softw. 2015;67:1–48.

    Article  Google Scholar 

  50. Moses L, Shapiro D, Littenberg B. Combining independent studies of a diagnostic test into a summary ROC curve: data-analytic approaches and some additional considerations. Stat Med. 1993;12:1293–316.

    Article  CAS  PubMed  Google Scholar 

  51. Holling H, Böhning W, Böhning D. Meta-analysis of diagnostic studies based upon SROC-curves: a mixed model approach using the Lehmann family. Stat Model. 2012;12:347–75.

    Article  Google Scholar 

  52. Holling H, Böhning W, Böhning D. Likelihood-based clustering of meta-analytic SROC curves. Psychometrika. 2012;77:106–26.

    Article  Google Scholar 

  53. Schlattmann P, Höhne J, Verba M. CAMAN: finite mixture models and meta-analysis tools–based on C.A.MAN. R package version 0.74. 2016. Available from https://CRAN.R-project.org/package=CAMAN. Accessed 29 June 2018.

  54. Doebler P, Holling H. Meta-analysis of diagnostic accuracy and ROC curves with covariate adjusted semiparametric mixtures. Psychometrika. 2015;80:1084–104.

    Article  PubMed  Google Scholar 

  55. Charoensawat S, Böhning W, Böhning D, Holling H. Meta-analysis and meta-modelling for diagnostic problems. BMC Med Res Methodol. 2014;14:56.

    Article  PubMed  PubMed Central  Google Scholar 

  56. Rücker G, Schumacher M. Summary ROC curve based on a weighted Youden index for selecting an optimal cut point in meta-analysis of diagnostic accuracy. Stat Med. 2010;29:3069–78.

    Article  PubMed  Google Scholar 

  57. Steinhauser S, Schumacher M, Rücker G. Modelling multiple thresholds in meta-analysis of diagnostic test accuracy studies. BMC Med Res Methodol. 2016;16:97.

    Article  PubMed  PubMed Central  Google Scholar 

  58. Levis B, Benedetti A, Levis A, Ioannidis J, Shrier I, Cuijpers P, et al. Selective cutoff reporting in studies of diagnostic test accuracy: a comparison of conventional and individual-patient-data meta-analyses of the Patient Health Questionnaire-9 depression screening tool. Am J Epidemiol. 2017;185:954–64.

    Article  PubMed  PubMed Central  Google Scholar 

  59. Dukic V, Gatsonis C. Meta-analysis of diagnostic test accuracy assessment studies with varying number of thresholds. Biometrics. 2003;59:936–46.

    Article  CAS  PubMed  Google Scholar 

  60. Hamza T, Arends L, van Houwelingen H, Stijnen T. Multivariate random effects meta-analysis of diagnostic tests with multiple thresholds. BMC Med Res Methodol. 2009;9:73.

    Article  PubMed  PubMed Central  Google Scholar 

  61. Putter H, Fiocco M, Stijnen T. Meta-analysis of diagnostic test accuracy studies with multiple thresholds using survival methods. Biom J. 2010;52:95–110.

    Article  CAS  PubMed  Google Scholar 

  62. Simoneau G, Levis B, Cuijpers P, Ioannidis J, Patten S, Shrier I, et al. A comparison of bivariate, multivariate random-effects, and Poisson correlated gamma-frailty models to meta-analyze individual patient data of ordinal scale diagnostic tests. Biom J. 2017;59:1317.

    Article  PubMed  Google Scholar 

  63. Riley R, Ahmed I, Ensor J, Takwoingi Y, Kirkham A, Morris R, et al. Meta-analysis of test accuracy studies: an exploratory method for investigating the impact of missing thresholds. Syst Rev. 2015;4:12.

    Article  PubMed  PubMed Central  Google Scholar 

  64. Hoyer A, Hirt S, Kuss O. Meta-analysis of full ROC curves using bivariate time-to-event models for interval-censored data. Res Synth Methods. 2018;9:62–72.

    Article  PubMed  Google Scholar 

  65. Martínez-Camblor P. Fully non-parametric receiver operating characteristic curve estimation for random-effects meta-analysis. Stat Methods Med Res. 2017;26:5–20.

    Article  PubMed  Google Scholar 

  66. Riley R, Takwoingi Y, Trikalinos T, Guha A, Biswas A, Ensor J, et al. Meta-analysis of test accuracy studies with multiple and missing thresholds: a multivariate-normal model. J Biomet Biostat. 2014;5:196.

    Article  Google Scholar 

  67. Deeks J, Wisniewski S, Davenport C. Chapter 4: guide to the contents of a Cochrane Diagnostic Test Accuracy Protocol. In: Deeks J, Bossuyt P, Gatsonis C, editors Cochrane handbook for systematic reviews of diagnostic test accuracy version 1.0.0. 2013. Available from http://srdta.cochrane.org/. Accessed 29 June 2018.

  68. Meyer P, Frings L, Rücker G, Hellwig S. 18F-FDG PET in parkinsonism: differential diagnosis and cognitive impairment in Parkinson’s disease. J Nucl Med. 2017;58:1888.

    Article  PubMed  Google Scholar 

  69. Schlattmann P, Verba M, Dewey M, Walther M. Mixture models in diagnostic meta-analyses—clustering summary receiver operating characteristic curves accounted for heterogeneity and correlation. J Clin Epidemiol. 2015;68:61–72.

    Article  PubMed  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Statistics, TU Dortmund University, Dortmund, Germany

    Philipp Doebler

  2. Institute of Psychology, Faculty of Psychology and Sport Sciences, University of Münster, Münster, Germany

    Paul-Christian Bürkner

  3. Faculty of Medicine, Institute for Medical Biometry and Statistics, Medical Center – University of Freiburg, Freiburg im Breisgau, Germany

    Gerta Rücker

Authors
  1. Philipp Doebler
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Paul-Christian Bürkner
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Gerta Rücker
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Philipp Doebler .

Editor information

Editors and Affiliations

  1. Department of Medico-Surgical Sciences, Sapienza University of Rome, Latina, Italy

    Giuseppe Biondi-Zoccai

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Doebler, P., Bürkner, PC., Rücker, G. (2018). Statistical Packages for Diagnostic Meta-Analysis and Their Application. In: Biondi-Zoccai, G. (eds) Diagnostic Meta-Analysis. Springer, Cham. https://doi.org/10.1007/978-3-319-78966-8_12

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-78966-8_12

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-78965-1

  • Online ISBN: 978-3-319-78966-8

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation