Skip to main content
SpringerLink
Log in

Umbrella Reviews, Overviews of Reviews, and Meta-epidemiologic Studies: Similarities and Differences

  • Chapter
  • First Online:
Umbrella Reviews

Abstract

This chapter describes umbrella reviews, overviews of reviews, and meta-epidemiologic studies focusing on their definitions, purposes, and classifications where appropriate and then elaborating on their similarities and differences. We may consider umbrella reviews as reviews integrating several types of study designs but typically randomized controlled trials and systematic reviews of such studies in a unifying fashion in order to address a content issue (e.g., whether or not a given drug is superior to another). Overviews of reviews are reviews of systematic reviews and meta-analyses which can focus on content or methodological issues. Finally, meta-epidemiologic studies focus on potentially different types of study designs but most typically on randomized trials and systematic reviews, usually across different content domains (e.g., topics or conditions), and mainly aim at addressing methodological issues.

Is there a magic method of determining when a meta-analysis is likely to be misleading?

C. David Naylor [1]

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 149.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 199.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 199.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Naylor CD. Meta-analysis and the meta-epidemiology of clinical research. Meta-analysis is an important contribution to research and practice but it’s not a panacea. BMJ. 1997;315(7109):617–9. doi:10.1136/bmj.315.7109.617.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  2. Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions. Chichester: Wiley; 2008. doi:10.1002/9780470712184.

    Google Scholar 

  3. Borenstein M, Hedges LV, Higgins JPT, Rothstein HR. Introduction to meta-analysis. Chichester: Wiley; 2009. doi:10.1002/9780470743386.

    Book  Google Scholar 

  4. Savović J, Harris RJ, Wood L, Beynon R, Altman D, Als-Nielsen B, Balk EM, Deeks J, Gluud LL, Gluud C, Ioannidis JPA, Jűni P, Moher D, Pildal J, Schulz KF, Sterne JAC. Development of a combined database for meta-epidemiological research. Res Synth Methods. 2010;1(3–4):212–25. doi:10.1002/jrsm.18.

    Article  PubMed  Google Scholar 

  5. Ioannidis JPA. Integration of evidence from multiple meta-analyses: a primer on umbrella reviews, treatment networks and multiple treatments meta-analyses. Can Med Assoc J. 2009;181(8):488–93. doi:10.1503/cmaj.081086.

    Article  Google Scholar 

  6. Grant MJ, Booth A. A typology of reviews: an analysis of 14 review types and associated methodologies. Health Info Libr J. 2009;26(2):91–108. doi:10.1111/j.1471-1842.2009.00848.x.

    Article  PubMed  Google Scholar 

  7. Moe RH, Haavardsholm EA, Christie A, Jamtvedt G, Dahm KT, Hagen KB. Effectiveness of nonpharmacological and nonsurgical interventions for hip osteoarthritis: an umbrella review of high-quality systematic reviews. Phys Ther. 2007. doi:10.2522/ptj.20070042.

    Google Scholar 

  8. Bastian H, Glasziou P, Chalmers I. Seventy-five trials and eleven systematic reviews a day: how will we ever keep up? PLoS Med. 2010;7(9):e1000326. doi:10.1371/journal.pmed.1000326.

    Article  PubMed Central  PubMed  Google Scholar 

  9. Becker LA, Oxman AD. Overviews of reviews. In: Higgins JPT, Green S, editors. Cochrane handbook for systematic reviews of interventions. Chichester: Wiley; 2008. doi:10.1002/9780470712184.ch22.

    Google Scholar 

  10. Hartling L, Vandermeer B, Fernandes RM. Systematic reviews, overviews of reviews and comparative effectiveness reviews: a discussion of approaches to knowledge synthesis. Evid Based Child Health Cochrane Rev J. 2014;9(2):486–94. doi:10.1002/ebch.1968.

    Article  Google Scholar 

  11. Hartling L, Chisholm A, Thomson D, Dryden DM. A descriptive analysis of overviews of reviews published between 2000 and 2011. PLoS One. 2012;7(11):e49667. doi:10.1371/journal.pone.0049667.

    Article  PubMed Central  CAS  PubMed  Google Scholar 

  12. Thomson D, Russell K, Becker L, Klassen T, Hartling L. The evolution of a new publication type: steps and challenges of producing overviews of reviews. Res Synth Methods. 2010;1(3–4):198–211. doi:10.1002/jrsm.30.

    Article  PubMed  Google Scholar 

  13. Pieper D, Buechter R, Jerinic P, Eikermann M. Overviews of reviews often have limited rigor: a systematic review. J Clin Epidemiol. 2012;65(12):1267–73. doi:10.1016/j.jclinepi.2012.06.015.

    Article  PubMed  Google Scholar 

  14. Thomson D, Foisy M, Oleszczuk M, Wingert A, Chisholm A, Hartling L. Overview of reviews in child health: evidence synthesis and the knowledge base for a specific population. Evid Based Child Health Cochrane Rev J. 2013;8(1):3–10. doi:10.1002/ebch.1897.

    Article  Google Scholar 

  15. Harrold J, Ali S, Oleszczuk M, Lacaze-Masmonteil T, Hartling L. Corticosteroids for the prevention of bronchopulmonary dysplasia in preterm infants: an overview of Cochrane reviews. Evid Based Child Health Cochrane Rev J. 2013;8(6):2063–75. doi:10.1002/ebch.1955.

    Article  Google Scholar 

  16. Adams NP, Jones PW. The dose–response characteristics of inhaled corticosteroids when used to treat asthma: an overview of Cochrane systematic reviews. Respir Med. 2006;100(8):1297–306. doi:10.1016/j.rmed.2006.04.015.

    Article  CAS  PubMed  Google Scholar 

  17. Bloom BS. Effects of continuing medical education on improving physician clinical care and patient health: a review of systematic reviews. Int J Technol Assess Health Care. 2005;21(3):380–5. doi:10.1017/S026646230505049X.

    Article  PubMed  Google Scholar 

  18. de Vet E, de Ridder DTD, de Wit JBF. Environmental correlates of physical activity and dietary behaviours among young people: a systematic review of reviews. Obes Rev. 2011;12(5):e130–42. doi:10.1111/j.1467-789X.2010.00784.x.

    Article  PubMed  Google Scholar 

  19. Cates CJ, Stovold E, Wieland S, Oleszczuk M, Thomson D, Becker L. The Cochrane Library and safety of regular long-acting beta2-agonists in children with asthma: an overview of reviews. Evid Based Child Health Cochrane Rev J. 2012;7(6):1798–806. doi:10.1002/ebch.1889.

    Article  Google Scholar 

  20. Hillberg T, Hamilton-Giachritsis C, Dixon L. Review of meta-analyses on the association between child sexual abuse and adult mental health difficulties: a systematic approach. Trauma Violence Abuse. 2011;12(1):38–49. doi:10.1177/1524838010386812.

    Article  PubMed  Google Scholar 

  21. Trinquart L, Dechartres A, Ravaud P. Commentary: meta-epidemiology, meta-meta-epidemiology or network meta-epidemiology? Int J Epidemiol. 2013;42(4):1131–3. doi:10.1093/ije/dyt137.

    Article  PubMed  Google Scholar 

  22. Schulz KF, Chalmers I, Hayes RJ, Altman DG. Empirical evidence of bias: dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA. 1995;273(5):408–12. doi:10.1001/jama.1995.03520290060030.

    Article  CAS  PubMed  Google Scholar 

  23. Bae J-M. Meta-epidemiology. Epidemiol Health. 2014;36:e2014019. doi:10.4178/epih/e2014019.

    Article  PubMed Central  PubMed  Google Scholar 

  24. Zhang W. Meta-epidemiology: building the bridge from research evidence to clinical practice. Osteoarthritis Cartilage. 2010;18(Supplement 2):S1. doi:10.1016/S1063-4584(10)00293-1.

    Article  CAS  Google Scholar 

  25. Tzoulaki I, Siontis KC, Ioannidis JP. Prognostic effect size of cardiovascular biomarkers in datasets from observational studies versus randomised trials: meta-epidemiology study. BMJ (Clin Res Ed). 2011;343:d6829. doi:10.1136/bmj.d6829.

    Article  Google Scholar 

  26. Savović J, Jones H, Altman D, Harris R, Juni P, Pildal J, Als-Nielsen B, Balk E, Gluud C, Gluud L, Ioannidis J, Schulz K, Beynon R, Welton N, Wood L, Moher D, Deeks J, Sterne J. Influence of reported study design characteristics on intervention effect estimates from randomised controlled trials: combined analysis of meta-epidemiological studies. Health Technol Assess (Winchester, England). 2012;16(35):1–82. doi:10.3310/hta16350.

    Google Scholar 

  27. Savović J, Jones HE, Altman DG, Harris RJ, Jüni P, Pildal J, Als-Nielsen B, Balk EM, Gluud C, Gluud LL, Ioannidis JPA, Schulz KF, Beynon R, Welton NJ, Wood L, Moher D, Deeks JJ, Sterne JAC. Influence of reported study design characteristics on intervention effect estimates from randomized, controlled trials. Ann Intern Med. 2012;157(6):429–38. doi:10.7326/0003-4819-157-6-201209180-00537.

    Article  PubMed  Google Scholar 

  28. Sterne JA, Juni P, Schulz KF, Altman DG, Bartlett C, Egger M. Statistical methods for assessing the influence of study characteristics on treatment effects in ‘meta-epidemiological’ research. Stat Med. 2002;21(11):1513–24. doi:10.1002/sim.1184.

    Article  PubMed  Google Scholar 

  29. Siersma V, Als-Nielsen B, Chen W, Hilden J, Gluud LL, Gluud C. Multivariable modelling for meta-epidemiological assessment of the association between trial quality and treatment effects estimated in randomized clinical trials. Stat Med. 2007;26(14):2745–58. doi:10.1002/sim.2752.

    Article  CAS  PubMed  Google Scholar 

  30. Chaimani A, Vasiliadis HS, Pandis N, Schmid CH, Welton NJ, Salanti G. Effects of study precision and risk of bias in networks of interventions: a network meta-epidemiological study. Int J Epidemiol. 2013;42(4):1120–31. doi:10.1093/ije/dyt074.

    Article  PubMed  Google Scholar 

  31. Salanti G, Dias S, Welton NJ, Ades AE, Golfinopoulos V, Kyrgiou M, Mauri D, Ioannidis JP. Evaluating novel agent effects in multiple-treatments meta-regression. Stat Med. 2010;29(23):2369–83. doi:10.1002/sim.4001.

    PubMed  Google Scholar 

  32. Als-Nielsen B, Chen W, Lotte Gluud L, Siersma V, Hilden J, Gluud C. Are trial size and reported methodological quality associated with treatment effects? Observational study of 523 randomised trials. In: 12th Cochrane Colloquium: bridging the gaps. Cochrane, Ottawa, Canada 2004. 2–6.

    Google Scholar 

  33. Balk EM, Bonis PA, Moskowitz H, Schmid CH, Ioannidis JP, Wang C, Lau J. Correlation of quality measures with estimates of treatment effect in meta-analyses of randomized controlled trials. JAMA. 2002;287(22):2973–82. doi:10.1001/jama.287.22.2973.

    Article  PubMed  Google Scholar 

  34. Contopoulos-Ioannidis DG, Gilbody SM, Trikalinos TA, Churchill R, Wahlbeck K, Ioannidis JPA; Project TE-P. Comparison of large versus smaller randomized trials for mental health-related interventions. Am J Psychiatry. 2005;162(3):578–84. doi:10.1176/appi.ajp.162.3.578.

    Google Scholar 

  35. Egger M, Juni P, Bartlett C, Holenstein F, Sterne J. How important are comprehensive literature searches and the assessment of trial quality in systematic reviews? Empirical study. Health Technol Assess (Winchester, England). 2003;7(1):1–76.

    CAS  Google Scholar 

  36. Kjaergard LL, Villumsen J, Gluud C. Reported methodologic quality and discrepancies between large and small randomized trials in meta-analyses. Ann Intern Med. 2001;135(11):982–9. doi:10.7326/0003-4819-135-11-200112040-00010.

    Article  CAS  PubMed  Google Scholar 

  37. McAuley L, Pham B, Tugwell P, Moher D. Does the inclusion of grey literature influence estimates of intervention effectiveness reported in meta-analyses? Lancet. 2000;356(9237):1228–31. doi:http://dx.doi.org/10.1016/S0140-6736(00)02786-0.

    Article  CAS  PubMed  Google Scholar 

  38. Moja LP, Telaro E, D’Amico R, Moschetti I, Coe L, Liberati A. Assessment of methodological quality of primary studies by systematic reviews: results of the metaquality cross sectional study. BMJ. 2005;330(7499):1053. doi:10.1136/bmj.38414.515938.8F.

    Article  PubMed Central  PubMed  Google Scholar 

  39. Royle P, Milne R. Literature searching for randomized controlled trials used in Cochrane reviews: rapid versus exhaustive searches. Int J Technol Assess Health Care. 2003;19(04):591–603. doi:10.1017/S0266462303000552.

    Article  PubMed  Google Scholar 

  40. Sampson M, Barrowman NJ, Moher D, Klassen TP, Pham B, Platt R, St. John PD, Viola R, Raina P. Should meta-analysts search Embase in addition to Medline? J Clin Epidemiol. 2003;56(10):943–55. doi:http://dx.doi.org/10.1016/S0895-4356(03)00110-0.

    Article  PubMed  Google Scholar 

  41. Wood L, Egger M, Gluud LL, Schulz KF, Juni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA. Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ (Clinical Res Ed). 2008;336(7644):601–5. doi:10.1136/bmj.39465.451748.AD.

    Article  Google Scholar 

  42. Zhang W, Robertson J, Jones AC, Dieppe PA, Doherty M. The placebo effect and its determinants in osteoarthritis: meta-analysis of randomised controlled trials. Ann Rheum Dis. 2008;67(12):1716–23. doi:10.1136/ard.2008.092015.

    Article  CAS  PubMed  Google Scholar 

  43. Nüesch E, Trelle S, Reichenbach S, Rutjes AW, Burgi E, Scherer M, Altman DG, Juni P. The effects of excluding patients from the analysis in randomised controlled trials: meta-epidemiological study. BMJ (Clinical Research Ed). 2009;339:b3244. doi:10.1136/bmj.b3244.

    Article  Google Scholar 

  44. Nüesch E, Trelle S, Reichenbach S, Rutjes AW, Tschannen B, Altman DG, Egger M, Juni P. Small study effects in meta-analyses of osteoarthritis trials: meta-epidemiological study. BMJ (Clinical Res Ed). 2010;341:c3515. doi:10.1136/bmj.c3515.

    Article  Google Scholar 

  45. Valdes AM, Arden NK, Tamm A, Kisand K, Doherty S, Pola E, Cooper C, Muir KR, Kerna I, Hart D, O’Neil F, Zhang W, Spector TD, Maciewicz RA, Doherty M. A meta-analysis of interleukin-6 promoter polymorphisms on risk of hip and knee osteoarthritis. Osteoarthritis Cartilage. 2010;18(5):699–704. doi:http://dx.doi.org/10.1016/j.joca.2009.12.012.

    Article  CAS  PubMed  Google Scholar 

  46. Oliver S, Bagnall AM, Thomas J, Shepherd J, Sowden A, White I, Dinnes J, Rees R, Colquitt J, Oliver K, Garrett Z. Randomised controlled trials for policy interventions: a review of reviews and meta-regression. Health Technol Assess (Winchester, England). 2010;14(16):1–165, iii. doi:10.3310/hta14160.

    CAS  Google Scholar 

  47. Herbison P, Hay-Smith J, Gillespie WJ. Different methods of allocation to groups in randomized trials are associated with different levels of bias. A meta-epidemiological study. J Clin Epidemiol. 2011;64(10):1070–5. doi:10.1016/j.jclinepi.2010.12.018.

    Article  PubMed  Google Scholar 

  48. Dechartres A, Trinquart L, Boutron I, Ravaud P. Influence of trial sample size on treatment effect estimates: meta-epidemiological study. BMJ (Clinical Res Ed). 2013;346:f2304. doi:10.1136/bmj.f2304.

    Google Scholar 

  49. Dechartres A, Boutron I, Trinquart L, Charles P, Ravaud P. Single-center trials show larger treatment effects than multicenter trials: evidence from a meta-epidemiologic study. Ann Intern Med. 2011;155(1):39–51. doi:10.7326/0003-4819-155-1-201107050-00006.

    Article  PubMed  Google Scholar 

  50. Zhang Z, Xu X, Ni H. Small studies may overestimate the effect sizes in critical care meta-analyses: a meta-epidemiological study. Crit Care (London, England). 2013;17(1):R2. doi:10.1186/cc11919.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, University of Crete, Heraklion, Greece

    Michail Tsagris

  2. Division of Medicine, University College London, London, UK

    Konstantinos C. Fragkos

Authors
  1. Michail Tsagris
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Konstantinos C. Fragkos
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Michail Tsagris .

Editor information

Editors and Affiliations

  1. Dept of Medico-Surgical Sci, Sapienza University of Rome, Latina, Italy

    Giuseppe Biondi-Zoccai

Rights and permissions

Reprints and permissions

Copyright information

© 2016 Springer International Publishing Switzerland

About this chapter

Cite this chapter

Tsagris, M., Fragkos, K.C. (2016). Umbrella Reviews, Overviews of Reviews, and Meta-epidemiologic Studies: Similarities and Differences. In: Biondi-Zoccai, G. (eds) Umbrella Reviews. Springer, Cham. https://doi.org/10.1007/978-3-319-25655-9_4

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-25655-9_4

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-25653-5

  • Online ISBN: 978-3-319-25655-9

  • eBook Packages: MedicineMedicine (R0)

Publish with us

Policies and ethics

Search

Navigation