Skip to main content
SpringerLink
Log in

Development and validation of a theoretical test of proficiency for video-assisted thoracoscopic surgery (VATS) lobectomy

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Testing stimulates learning, improves long-term retention, and promotes technical performance. No purpose-orientated test of competence in the theoretical aspects of VATS lobectomy has previously been presented. The purpose of this study was, therefore, to develop and gather validity evidence for a theoretical test on VATS lobectomy consisting of multiple-choice questions.

Methods

Four European VATS lobectomy experts were interviewed to explore their views on important theoretical VATS lobectomy knowledge (step 1). This information was used to construct the test items in compliance with existing guidelines for multiple-choice questions (step 2). The experts rated the relevance of the items to confirm content validity in a modified Delphi approach (step 3). Finally, the test was administered to physicians, who were categorised into different experience levels based on their experience in VATS procedures overall and in VATS lobectomies specifically. Their answers were used to achieve construct validity (step 4).

Results

Initially, 81 items were constructed and two Delphi iterations reduced the test to 50 items. Item analysis led to the exclusion of 19 items and the mean discrimination index of the 31 final items was 0.26. Cronbach’s alpha for internal consistency was 0.75. The mean item difficulty was calculated to 0.63. According to performed VATS procedures, significantly different test performances were detected when comparing the group performances (p = 0.002) and the experts performed significantly better than the novices (p < 0.001) and intermediates (p = 0.01). In the category of performed VATS lobectomies, significant group performances were also found. In this category, the experts were also significantly better than the novices (p < 0.001), the trainees (p = 0.002), and the intermediates (p = 0.01).

Conclusions

This study led to the development of a theoretical test on VATS lobectomy consisting of multiple-choice questions. Both content and construct validity evidence were established.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

Similar content being viewed by others

References

  1. Youlden DR, Cramb SM, Baade PD (2008) The international epidemiology of lung cancer: geographical distribution and secular trends. J Thorac Oncol 3:819–831

    Article  PubMed  Google Scholar 

  2. McKenna RJ Jr, Houck W, Fuller CB (2006) Video-assisted thoracic surgery lobectomy: experience with 1,100 cases. Ann Thorac Surg 81:421–425 discussion 425-426

    Article  PubMed  Google Scholar 

  3. Walker WS, Codispoti M, Soon SY, Stamenkovic S, Carnochan F, Pugh G (2003) Long-term outcomes following VATS lobectomy for non-small cell bronchogenic carcinoma. Eur J Cardiothorac Surg 23:397–402

    Article  PubMed  Google Scholar 

  4. Hansen HJ, Petersen RH, Christensen M (2011) Video-assisted thoracoscopic surgery (VATS) lobectomy using a standardized anterior approach. Surg Endosc 25:1263–1269

    Article  PubMed  Google Scholar 

  5. Yan TD, Black D, Bannon PG, McCaughan BC (2009) Systematic review and meta-analysis of randomized and nonrandomized trials on safety and efficacy of video-assisted thoracic surgery lobectomy for early-stage non-small-cell lung cancer. J Clin Oncol 27:2553–2562

    Article  PubMed  Google Scholar 

  6. Whitson BA, Groth SS, Duval SJ, Swanson SJ, Maddaus MA (2008) Surgery for early-stage non-small cell lung cancer: a systematic review of the video-assisted thoracoscopic surgery versus thoracotomy approaches to lobectomy. Ann Thorac Surg 86:2008–2016 discussion 2016-2008

    Article  PubMed  Google Scholar 

  7. Daniels LJ, Balderson SS, Onaitis MW, D’Amico TA (2002) Thoracoscopic lobectomy: a safe and effective strategy for patients with stage I lung cancer. Ann Thorac Surg 74:860–864

    Article  PubMed  Google Scholar 

  8. Detterbeck FC, Lewis SZ, Diekemper R, Addrizzo-Harris D, Alberts WM (2013) Executive Summary: diagnosis and management of lung cancer, 3rd ed: American College of Chest Physicians evidence-based clinical practice guidelines. Chest 143:7S–37S

    Article  CAS  PubMed  Google Scholar 

  9. Downey RJ, Cheng D, Kernstine K, Stanbridge R, Shennib H, Wolf R, Ohtsuka T, Schmid R, Waller D, Fernando H, Yim A, Martin J (2007) Video-assisted thoracic surgery for lung cancer resection: a consensus statement of the International Society Of Minimally Invasive Cardiothoracic Surgery (ISMICS) 2007. Innovations (Phila) 2:293–302

    Article  Google Scholar 

  10. Kaseda S, Aoki T, Hangai N, Shimizu K (2000) Better pulmonary function and prognosis with video-assisted thoracic surgery than with thoracotomy. Ann Thorac Surg 70:1644–1646

    Article  CAS  PubMed  Google Scholar 

  11. Balduyck B, Hendriks J, Lauwers P, Van Schil P (2007) Quality of life evolution after lung cancer surgery: a prospective study in 100 patients. Lung Cancer 56:423–431

    Article  CAS  PubMed  Google Scholar 

  12. Yim AP, Wan S, Lee TW, Arifi AA (2000) VATS lobectomy reduces cytokine responses compared with conventional surgery. Ann Thorac Surg 70:243–247

    Article  CAS  PubMed  Google Scholar 

  13. Petersen RP, Pham D, Burfeind WR, Hanish SI, Toloza EM, Harpole DH Jr, D’Amico TA (2007) Thoracoscopic lobectomy facilitates the delivery of chemotherapy after resection for lung cancer. Ann Thorac Surg 83:1245–1249 discussion 1250

    Article  PubMed  Google Scholar 

  14. Casali G, Walker WS (2009) Video-assisted thoracic surgery lobectomy: can we afford it? Eur J Cardiothorac Surg 35:423–428

    Article  PubMed  Google Scholar 

  15. Petersen RH, Hansen HJ (2010) Learning thoracoscopic lobectomy. Eur J Cardiothorac Surg 37:516–520

    Article  PubMed  Google Scholar 

  16. Konge L, Petersen RH, Hansen HJ, Ringsted C (2012) No extensive experience in open procedures is needed to learn lobectomy by video-assisted thoracic surgery. Interact CardioVasc Thorac Surg 15:961–965

    Article  PubMed Central  PubMed  Google Scholar 

  17. Konge L, Lehnert P, Hansen HJ, Petersen RH, Ringsted C (2012) Reliable and valid assessment of performance in thoracoscopy. Surg Endosc 26:1624–1628

    Article  PubMed  Google Scholar 

  18. Jensen K, Ringsted C, Hansen HJ, Petersen RH, Konge L (2014) Simulation-based training for thoracoscopic lobectomy: a randomized controlled trial: virtual-reality versus black-box simulation. Surg Endosc 28:1821–1829

    Article  PubMed  Google Scholar 

  19. Bjurstrom JM, Konge L, Lehnert P, Krogh CL, Hansen HJ, Petersen RH, Ringsted C (2013) Simulation-based training for thoracoscopy. Simul Healthc 8:317–323

    Article  PubMed  Google Scholar 

  20. Ferguson J, Walker W (2006) Developing a VATS lobectomy programme—can VATS lobectomy be taught? Eur J Cardiothorac Surg 29:806–809

    Article  PubMed  Google Scholar 

  21. Wan IY, Thung KH, Hsin MK, Underwood MJ, Yim AP (2008) Video-assisted thoracic surgery major lung resection can be safely taught to trainees. Ann Thorac Surg 85:416–419

    Article  PubMed  Google Scholar 

  22. Li X, Wang J, Ferguson MK (2014) Competence versus mastery: the time course for developing proficiency in video-assisted thoracoscopic lobectomy. J Thorac Cardiovasc Surg 147:1150–1154

    Article  PubMed  Google Scholar 

  23. Carrott PW Jr, Jones DR (2013) Teaching video-assisted thoracic surgery (VATS) lobectomy. J Thorac Dis 5:S207–S211

    PubMed Central  PubMed  Google Scholar 

  24. Kohls-Gatzoulis JA, Regehr G, Hutchison C (2004) Teaching cognitive skills improves learning in surgical skills courses: a blinded, prospective, randomized study. Can J Surg 47:277–283

    PubMed Central  PubMed  Google Scholar 

  25. Roediger HL, Karpicke JD (2006) Test-enhanced learning: taking memory tests improves long-term retention. Psychol Sci 17:249–255

    Article  PubMed  Google Scholar 

  26. Kromann CB, Jensen ML, Ringsted C (2009) The effect of testing on skills learning. Med Educ 43:21–27

    Article  PubMed  Google Scholar 

  27. Karpicke JD, Roediger HL 3rd (2008) The critical importance of retrieval for learning. Science 319:966–968

    Article  CAS  PubMed  Google Scholar 

  28. Case SM, Swanson DB (2001) constructing written test questions for the basic and clinical sciences, 3rd edn. National Board of Medical Examiners, Philadelphia

    Google Scholar 

  29. Schuwirth LW, van der Vleuten CP (2003) ABC of learning and teaching in medicine: written assessment. BMJ 326:643–645

    Article  PubMed Central  PubMed  Google Scholar 

  30. Downing SM (2009) Written Tests. In: Downing SM, Yudkowsky R (eds) Assessment in health professions education. Routledge, New York, pp 149–181

    Google Scholar 

  31. van der Vleuten CPM, Schuwirth LWT (2009) Written Assessments. In: Dent JA, Harden RM (eds) A practical guide for medical teachers. Churchill Livingstone, London, pp 323–331

    Google Scholar 

  32. McKenna RJ Jr (2008) Complications and learning curves for video-assisted thoracic surgery lobectomy. Thorac Surg Clin 18:275–280

    Article  PubMed  Google Scholar 

  33. Boffa DJ, Gangadharan S, Kent M, Kerendi F, Onaitis M, Verrier E, Roselli E (2012) Self-perceived video-assisted thoracic surgery lobectomy proficiency by recent graduates of North American thoracic residencies. Interact CardioVasc Thorac Surg 14:797–800

    Article  PubMed Central  PubMed  Google Scholar 

  34. Savran MM, Clementsen PF, Annema JT, Minddal V, Larsen KR, Park YS, Konge L (2014) Development and validation of a theoretical test in endosonography for pulmonary diseases. Respiration 88:67–73

    Article  PubMed  Google Scholar 

  35. Schubert S, Ortwein H, Dumitsch A, Schwantes U, Wilhelm O, Kiessling C (2008) A situational judgement test of professional behaviour: development and validation. Med Teach 30:528–533

    Article  PubMed  Google Scholar 

  36. Strandbygaard J, Maagaard M, Larsen CR, Schouenborg L, Ottosen C, Ringsted C, Grantcharov T, Ottesen B, Sorensen JL (2013) Development and validation of a theoretical test in basic laparoscopy. Surg Endosc 27:1353–1359

    Article  PubMed  Google Scholar 

  37. Streiner DL, Norman GR (2008) Validity. In: Streiner DL, Norman GR (eds) Health Measurement Scales: a practical guide to their development and use. Oxford University Press, Oxford, pp 247–276

    Chapter  Google Scholar 

  38. Graham B, Regehr G, Wright JG (2003) Delphi as a method to establish consensus for diagnostic criteria. J Clin Epidemiol 56:1150–1156

    Article  PubMed  Google Scholar 

  39. Haladyna TM (2004) Guidelines for developing MC items. In: Haladyna TM (ed) Developing and validating multiple-choice test items. Routledge, London and New York, pp 97–126

    Google Scholar 

  40. Downing SM (2009) Statistics of testing. In: Downing SM, Yudkowsky R (eds) Assessment in health professions education. Routledge, New York, pp 93–117

    Google Scholar 

  41. Haladyna TM (2004) Validity evidence coming from statistical study of item responses. In: Haladyna TM (ed) Developing and validating multiple-choice test items. Routledge, London, pp 202–229

    Google Scholar 

  42. Gruber H (2001) Acquisition of expertise. In: Smelser NJ, Baltes PB (eds) International encyclopedia of the social & behavioral sciences. Elsevier, Amsterdam, p 5145–5150

  43. Patel VL, Kaufman DR (2001) Cognitive psychology of medical expertise. In: Smelser NJ, Baltes PB (eds) International encyclopedia of the social & behavioral sciences 9515–9517

  44. Streiner DL, Norman GR (2008) Divising the items. In: Streiner DL, Norman GR (eds) Health Measurement Scales: a practical guide to their development and use. Oxford University Press, Oxford, pp 18–38

    Chapter  Google Scholar 

  45. Hawkins RE, Swanson DB (2008) Using written examinations to assess medical knowledge and its application. In: Holmboe ES, Hawkins RE (eds) Practical guide to the evaluation of clinical competence. Mosby, Philadelphia, pp 42–59

    Google Scholar 

  46. Keeney S, Hasson F, McKenna HP (2001) A critical review of the Delphi technique as a research methodology for nursing. Int J Nurs Stud 38:195–200

    Article  CAS  PubMed  Google Scholar 

  47. Hasson F, Keeney S, McKenna H (2000) Research guidelines for the Delphi survey technique. J Adv Nurs 32:1008–1015

    CAS  PubMed  Google Scholar 

  48. Streiner DL, Norman GR (2008) Selecting the items. In: Streiner DL, Norman GR (eds) Health Measurement Scales: a practical guide to their development and use. Oxford University Press, Oxford, pp 78–104

    Chapter  Google Scholar 

  49. Cook DA, Thompson WG, Thomas KG (2014) Test-enhanced web-based learning: optimizing the number of questions (a randomized crossover trial). Acad Med 89:169–175

    Article  PubMed  Google Scholar 

  50. Ashley EA (2000) Medical education—beyond tomorrow? The new doctor—Asclepiad or Logiatros? Med Educ 34:455–459

    Article  CAS  PubMed  Google Scholar 

  51. Van der Vleuten CPM, Dolmans DHJM, Scherpbier AJJA (2000) The need for evidence in education. Med Teach 22:246–250

    Article  Google Scholar 

  52. McGaghie WC, Issenberg SB, Barsuk JH, Wayne DB (2014) A critical review of simulation-based mastery learning with translational outcomes. Med Educ 48:375–385

    Article  PubMed  Google Scholar 

  53. Miller GE (1990) The assessment of clinical skills/competence/performance. Acad Med 65:S63–S67

    Article  CAS  PubMed  Google Scholar 

Download references

Disclosures

Savran, Hansen, Petersen, Walker, Schmid, Bojsen, and Konge have no conflicts of interest or financial ties to disclose.

Author information

Authors and Affiliations

  1. Centre for Clinical Education, University of Copenhagen and the Capital Region of Denmark, Blegdamsvej 9, 2100, Copenhagen, Denmark

    Mona Meral Savran, Signe Rolskov Bojsen & Lars Konge

  2. Department of Cardiothoracic Surgery, Rigshospitalet, University of Copenhagen, Copenhagen, Denmark

    Henrik Jessen Hansen & René Horsleben Petersen

  3. Department of Cardiothoracic Surgery, Royal Infirmary of Edinburgh, Edinburgh, Scotland

    William Walker

  4. Department of Visceral, Transplant and Thoracic Surgery, Centre of Operative Medicine, Innsbruck, Austria

    Thomas Schmid

Authors
  1. Mona Meral Savran
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Henrik Jessen Hansen
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. René Horsleben Petersen
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. William Walker
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Thomas Schmid
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Signe Rolskov Bojsen
    View author publications

    You can also search for this author in PubMed Google Scholar

  7. Lars Konge
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Mona Meral Savran.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Savran, M.M., Hansen, H.J., Petersen, R.H. et al. Development and validation of a theoretical test of proficiency for video-assisted thoracoscopic surgery (VATS) lobectomy. Surg Endosc 29, 2598–2604 (2015). https://doi.org/10.1007/s00464-014-3975-y

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-014-3975-y

Keywords

Search

Navigation