Skip to main content

Advertisement

SpringerLink
Log in

Evidence-based recommendations for blinding in surgical trials

  • Systematic Reviews and Meta-analysis
  • Published:
Langenbeck's Archives of Surgery Aims and scope Submit manuscript

Abstract

Rationale

Blinding reduces performance and detection bias in randomized controlled trials (RCT). There is evidence that lack of blinding leads to overestimation of treatment effects in pharmacological trials. Since surgical trials use interventions with a physical component, blinding is often complicated. The aim of this study was to analyze, in general and abdominal surgery RCT, the status of blinding, the potential for blinding, and the influence of blinding on outcomes.

Methods

A systematic search of the literature in CENTRAL, MEDLINE, and Web of Science was conducted to identify RCT with a surgical intervention, starting in 1996, the year when the first CONSORT statement was published. Information on general study characteristics and blinding methods was extracted. The presence or absence of blinding of the study contributors—patients, surgeons, data collectors, outcome assessors, and data analysts—was analyzed. The association of blinding with the trial outcome was investigated for every study contributor.

Results

Out of 29,119 articles, 378 RCT were included in the analysis. These investigated a total of 62,522 patients, of whom 15,025 were blinded (24.0%). Contributors could have been blinded in far more trials, as the potential for blinding measures ranged from 69% for outcome assessors to 98% for data analysts. If blinding of surgeons would have been possible but had not been performed, this was associated with more significant trial outcomes (OR 13.670; 95% CI 1.308 to 142.840; p = 0.0289).

Discussion

The potential of blinding, an important quality measure in surgical RCT, has not been exhausted. This study summarizes the existing evidence on blinding in surgical trials and gives evidence-based recommendations for the use of blinding in future surgical trials.

Systematic review registration

PROSPERO 2015:CRD42015026837.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

Abbreviations

CI:

confidence interval

ITT:

intention to treat

OR:

odds ratio

PICO:

Participants, Interventions, Comparisons, and Outcomes

RCT:

randomized controlled trials

References

  1. Sackett DL, Rosenberg WM, Gray JA, Haynes RB, Richardson WS (1996) Evidence based medicine: what it is and what it isn’t. BMJ 312(7023):71–72

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  2. Phillips B, Ball C, Sackett D, Badenoch D, Straus S, Haynes B, Dawes M (2009) Oxford centre for evidence-based medicine—levels of evidence. Available from: www.cebm.net/oxford-centre-evidence-based-medicine-levels-evidence-march-2009/. Last accessed 30.09.2016

  3. Solomon MJ, McLeod RS (1995) Should we be performing more randomized controlled trials in evaluating surgical procedures? Surgery 118:459–467

    Article  CAS  Google Scholar 

  4. Cochrane Handbook, Higgins JPT (2011) Cochrane handbook for systematic reviews of interventions version 5.1.0 [updated March 2011]. The Cochrane Collaboration. Available from www.cochrane-handbook.org. Accessed 30.09.2016

  5. Schulz KF, Altman DG, Moher D, CONSORT Group (2010) CONSORT 2010 statement: updated guidelines for reporting parallel group randomised trials. Trials 11:32

    Article  PubMed  PubMed Central  Google Scholar 

  6. Devereaux PJ, Manns BJ, Ghali WA, Quan H, Lacchetti C, Mouton VM et al (2001) Physician interpretations and textbook definitions of blinding terminology in randomized controlled trials. JAMA 285:2000–2003

    Article  CAS  Google Scholar 

  7. Jüni P, Altman DG, Egger M (2001) Systematic reviews in health care: assessing the quality of controlled clinical trials. BMJ 323(7303):42–46

    Article  PubMed  PubMed Central  Google Scholar 

  8. Probst P, Grummich K, Harnoss JC, Hüttner FJ, Jensen K, Braun S, Kieser M, Ulrich A, Büchler MW, Diener MK (2016) Placebo-controlled trials in surgery: a systematic review and meta-analysis. Medicine (Baltimore) 95(17):e3516

    Article  CAS  Google Scholar 

  9. Boutron I, Tubach F, Giraudeau B, Ravaud P (2004) Blinding was judged more difficult to achieve and maintain in nonpharmacologic than pharmacologic trials. J Clin Epidemiol 57(6):543–550

    Article  PubMed  Google Scholar 

  10. Boutron I, Guittet L, Estellat C, Moher D, Hróbjartsson A, Ravaud P (2007) Reporting methods of blinding in randomized trials assessing nonpharmacological treatments. PLoS Med 4(2):e61

    Article  PubMed  PubMed Central  Google Scholar 

  11. Moher D, Liberati A, Tetzlaff J, Altman DG, The PRISMA Group (2010) Preferred reporting items for systematic reviews and meta-analyses: the PRISMA statement. Int J Surg 8(5):336–341

    Article  PubMed  Google Scholar 

  12. Probst P, Grummich K, Heger P, Zaschke S, Knebel P, Ulrich A, Büchler MW, Diener MK (2016) Blinding in randomized controlled trials in general and abdominal surgery: protocol for a systematic review and empirical study. Syst Rev 5:48

    Article  PubMed  PubMed Central  Google Scholar 

  13. Goossen K, Tenckhoff S, Probst P, Grummich K, Mihaljevic AL, Büchler MW, Diener MK (2018) Optimal literature search for systematic reviews in surgery. Langenbeck’s Arch Surg 403(1):119–129

    Article  Google Scholar 

  14. Begg CB, Cho MK, Eastwood S et al (1996) Improving the quality of reporting of randomized controlled trials: the CONSORT statement. JAMA 276:637–639

    Article  CAS  Google Scholar 

  15. Chan AW, Hróbjartsson A, Haahr MT, Gøtzsche PC, Altman DG (2004) Empirical evidence for selective reporting of outcomes in randomized trials: comparison of protocols to published articles. JAMA 291(20):2457–2465

    Article  CAS  Google Scholar 

  16. Probst P, Grummich K, Tenckhoff S, Ulrich A, Büchler MW, Knebel P, Diener MK (2016) Industry bias in randomized controlled trials in general and abdominal surgery: an empirical study. Ann Surg 264(1):87–92

    Article  PubMed  Google Scholar 

  17. Buscemi N, Hartling L, Vandermeer B, Tjosvold L, Klassen TP (2006) Single data extraction generated more errors than double data extraction in systematic reviews. J Clin Epidemiol 59(7):697–703

    Article  PubMed  Google Scholar 

  18. R Core Team (2013) R: a language and environment for statistical computing. R Foundation for Statistical Computing, Vienna http://www.R-project.org. Accessed 30.09.2016

    Google Scholar 

  19. Park JS, Kim JH, Kim JK, Yoon DS (2015) The role of abdominal drainage to prevent of intra-abdominal complications after laparoscopic cholecystectomy for acute cholecystitis: prospective randomized trial. Surg Endosc 29(2):453–457

    Article  PubMed  Google Scholar 

  20. Diener MK, Knebel P, Kieser M, Schüler P, Schiergens TS, Atanassov V, Neudecker J, Stein E, Thielemann H, Kunz R, von Frankenberg M, Schernikau U, Bunse J, Jansen-Winkeln B, Partecke LI, Prechtl G, Pochhammer J, Bouchard R, Hodina R, Beckurts KT, Leißner L, Lemmens HP, Kallinowski F, Thomusch O, Seehofer D, Simon T, Hyhlik-Dürr A, Seiler CM, Hackert T, Reissfelder C, Hennig R, Doerr-Harim C, Klose C, Ulrich A, Büchler MW (2014) Effectiveness of triclosan-coated PDS Plus versus uncoated PDS II sutures for prevention of surgical site infection after abdominal wall closure: the randomised controlled PROUD trial. Lanct 384(9938):142–152

    Article  CAS  Google Scholar 

  21. Schulz KF, Chalmers I, Hayes RJ, Altman DG (1995) Empirical evidence of bias. Dimensions of methodological quality associated with estimates of treatment effects in controlled trials. JAMA 273:408–412

    Article  CAS  PubMed  Google Scholar 

  22. Montori VM, Bhandari M, Devereaux PJ, Manns BJ, Ghali WA, Guyatt GH (2002) In the dark: the reporting of blinding status in randomized controlled trials. J Clin Epidemiol 55:787–790

    Article  PubMed  Google Scholar 

  23. Boutron I, Estellat C, Guittet L, Dechartres A, Sackett DL, Hrobjartsson A, Ravaud P (2006) Methods of blinding in reports of randomized controlled trials assessing pharmacologic treatments: a systematic review. PLoS Med 3:e425. https://doi.org/10.1371/journal.pmed.0030425

    Article  PubMed  PubMed Central  Google Scholar 

  24. Poolman RW, Struijs PA, Krips R, Sierevelt IN, Marti RK, Farrokhyar F, Bhandari M (2007) Reporting of outcomes in orthopaedic randomized trials: does blinding of outcome assessors matter? J Bone Joint Surg Am 89:550–558. https://doi.org/10.2106/JBJS.F.00683

    Article  PubMed  Google Scholar 

  25. Wood L, Egger M, Gluud LL, Schulz KF, Juni P, Altman DG, Gluud C, Martin RM, Wood AJ, Sterne JA (2008) Empirical evidence of bias in treatment effect estimates in controlled trials with different interventions and outcomes: meta-epidemiological study. BMJ 336:601–605. https://doi.org/10.1136/bmj.39465.451748.AD

    Article  PubMed  PubMed Central  Google Scholar 

  26. Karanicolas PJ, Bhandari M, Taromi B, Akl EA, Bassler D, Alonso-Coello P, Rigau D, Bryant D, Smith SE, Walter SD u G, H G (2008) Blinding of outcomes in trials of orthopaedic trauma: an opportunity to enhance the validity of clinical trials. J Bone Joint Surg Am 90:1026–1033. https://doi.org/10.2106/JBJS.G.00963

    Article  PubMed  Google Scholar 

  27. Karanicolas PJ, Farrokhyar F, Bhandari M (2010) Practical tips for surgical research: blinding: who, what, when, why, how? Can J Surg 53:345–348

    PubMed  PubMed Central  Google Scholar 

  28. Farrokhyar F, Bajammal S, Kahnamoui K, Bhandari M (2010) Practical tips for surgical research. Ensuring balanced groups in surgical trials. Can J Surg 53:418–423

    PubMed  PubMed Central  Google Scholar 

  29. Miller LE, Stewart ME (2011) The blind leading the blind: use and misuse of blinding in randomized controlled trials. Contemp Clin Trials 32:240–243. https://doi.org/10.1016/j.cct.2010.11.004

    Article  PubMed  Google Scholar 

  30. Higgins JP, Altman DG, Gotzsche PC, Juni P, Moher D, Oxman AD, Savovic J, Schulz KF, Weeks L, Sterne JA, Cochrane Bias Methods G, Cochrane Statistical Methods G (2011) The Cochrane Collaboration’s tool for assessing risk of bias in randomised trials. BMJ 343:d5928. https://doi.org/10.1136/bmj.d5928

    Article  PubMed  PubMed Central  Google Scholar 

  31. Hrobjartsson A, Thomsen AS, Emanuelsson F, Tendal B, Hilden J, Boutron I, Ravaud P, Brorson S (2012) Observer bias in randomised clinical trials with binary outcomes: systematic review of trials with both blinded and non-blinded outcome assessors. BMJ 344:e1119. https://doi.org/10.1136/bmj.e1119

    Article  PubMed  Google Scholar 

  32. Boulind CE, Ewings P, Bulley SH, Reid JM, Jenkins JT, Blazeby JM, Francis NK (2013) Feasibility study of analgesia via epidural versus continuous wound infusion after laparoscopic colorectal resection. Br J Surg 100:395–402. https://doi.org/10.1002/bjs.8999

    Article  CAS  PubMed  Google Scholar 

  33. Baethge C, Assall OP, Baldessarini RJ (2013) Systematic review of blinding assessment in randomized controlled trials in schizophrenia and affective disorders 2000–2010. Psychother Psychosom 82:152–160. https://doi.org/10.1159/000346144

    Article  PubMed  Google Scholar 

  34. Berkman ND, Santaguida PL, Viswanathan M, Morton SC (2014) In: The empirical evidence of Bias in trials measuring treatment differences, Rockville (MD): Agency for healthcare research and quality (US), Report No.: 14-EHC050-EF. AHRQ Methods for effective health care.

  35. Feys F, Bekkering GE, Singh K, Devroey D (2014) Do randomized clinical trials with inadequate blinding report enhanced placebo effects for intervention groups and nocebo effects for placebo groups? Syst Rev 3:14. https://doi.org/10.1186/2046-4053-3-14

    Article  PubMed  PubMed Central  Google Scholar 

  36. Cuijpers P, Karyotaki E, Andersson G, Li J, Mergl R, Hegerl U (2015) The effects of blinding on the outcomes of psychotherapy and pharmacotherapy for adult depression: a meta-analysis. Eur Psychiatry 30:685–693. https://doi.org/10.1016/j.eurpsy.2015.06.005

    Article  CAS  PubMed  Google Scholar 

  37. Kahan BC, Rehal S, Cro S (2015) Blinded outcome assessment was infrequently used and poorly reported in open trials. PLoS One 10:e0131926. https://doi.org/10.1371/journal.pone.0131926

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  38. Metcalfe C, Avery K, Berrisford R, Barham P, Noble SM, Fernandez AM, Hanna G, Goldin R, Elliott J, Wheatley T, Sanders G, Hollowood A, Falk S, Titcomb D, Streets C, Donovan JL, Blazeby JM (2016) Comparing open and minimally invasive surgical procedures for oesophagectomy in the treatment of cancer: the ROMIO (Randomised Oesophagectomy: Minimally Invasive or Open) feasibility study and pilot trial. Health Technol Assess 20:1–68. https://doi.org/10.3310/hta20480

    Article  PubMed  PubMed Central  Google Scholar 

  39. Blencowe NS, Cook JA, Pinkney T, Rogers C, Reeves BC, Blazeby JM (2017) Delivering successful randomized controlled trials in surgery: Methods to optimize collaboration and study design. Clin Trials. https://doi.org/10.1177/1740774516687272

  40. Sim J (2017) Outcome-adaptive randomization: some ethical issues. Trials 18(Suppl 1):O42

    Google Scholar 

  41. Elbers R, Savovic J, Blencowe N, Higgins JPT, Sterne JAC (2017) Performance bias in trials that cannot blind participants and healthcare providers to assigned interventions: implications for trial conduct. Trials 18(Suppl 1):P478

    Google Scholar 

  42. Cousins S, Chalmers K, Avery K, Blencowe N, Brookes S, Blazeby J, Kobetic M, Munder T (2017) Minimising bias in surgical RCTs through blinding: a systematic review. Trials 18(Suppl 1):P377

    Google Scholar 

  43. Blencowe N, Main BG, Blazeby JM (2017) Beyond blinding: a systematic review to explore performance bias in surgical RCTs. Trials 18(Suppl 1):P224

    Google Scholar 

  44. Bassi C, Marchegiani G, Dervenis C, Sarr M, Abu Hilal M, Adham M, Allen P, Andersson R, Asbun HJ, Besselink MG, Conlon K, Del Chiaro M, Falconi M, Fernandez-Cruz L, Fernandez-Del Castillo C, Fingerhut A, Friess H, Gouma DJ, Hackert T, Izbicki J, Lillemoe KD, Neoptolemos JP, Olah A, Schulick R, Shrikhande SV, Takada T, Takaori K, Traverso W, Vollmer CR, Wolfgang CL, Yeo CJ, Salvia R, Buchler M, International Study Group on Pancreatic Surgery (ISGPS) (2017) The 2016 update of the International Study Group (ISGPS) definition and grading of postoperative pancreatic fistula: 11 years after. Surgery 161(3):584–591

    Article  Google Scholar 

Download references

Acknowledgements

Furthermore, the authors thank the following people for translation of full-text articles: Anja Truong (Japanese), Defan Chen (Chinese), and Solveig Tenckhoff (Italian and Spanish). Especially, the authors thank the staff of the university library for acquisition of full-text articles. Large parts of the “Background” and “Methods” section formed part of the protocol [12] and were published under the terms of the Creative Commons Attribution 4.0 International License.

Funding

No additional funding source is available. However, the resources and facilities of the University of Heidelberg were used in conducting this review.

Author information

Authors and Affiliations

  1. Department of General, Visceral and Transplantation Surgery, University of Heidelberg, Im Neuenheimer Feld 110, 69120, Heidelberg, Germany

    Pascal Probst, Steffen Zaschke, Patrick Heger, Julian C. Harnoss, Felix J. Hüttner, André L. Mihaljevic, Phillip Knebel & Markus K. Diener

  2. The Study Center of the German Surgical Society (SDGC), University of Heidelberg, Im Neuenheimer Feld 110, 69120, Heidelberg, Germany

    Pascal Probst, Steffen Zaschke, Patrick Heger, Julian C. Harnoss, Felix J. Hüttner, André L. Mihaljevic, Phillip Knebel & Markus K. Diener

Authors
  1. Pascal Probst
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Steffen Zaschke
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Patrick Heger
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Julian C. Harnoss
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Felix J. Hüttner
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. André L. Mihaljevic
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Phillip Knebel
    View author publications

    You can also search for this author in PubMed Google Scholar

  8. Markus K. Diener
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

PP and SZ developed the study concept, acquired, analyzed, interpreted the data, and drafted the manuscript. PH, JCH, and FJH acquired literature and revised the manuscript. ALM, PK, and MKD developed the study concept, interpreted data, and revised the manuscript. All authors read and approved the final manuscript. Furthermore, all authors agreed to be accountable for all aspects of the work in ensuring that questions related to the accuracy or integrity of any part of the work are appropriately investigated and resolved.

Corresponding author

Correspondence to Pascal Probst.

Ethics declarations

Conflict of interest

The authors declare that they have no competing interests.

Ethical approval

This article does not contain any studies with human participants or animals performed by any of the authors.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic supplementary material

ESM 1

(DOCX 18 kb)

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Probst, P., Zaschke, S., Heger, P. et al. Evidence-based recommendations for blinding in surgical trials. Langenbecks Arch Surg 404, 273–284 (2019). https://doi.org/10.1007/s00423-019-01761-6

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00423-019-01761-6

Keywords

Search

Navigation