Skip to main content
SpringerLink
Log in

Objective assessment of surgical skill transfer using non-invasive brain imaging

  • Published:
Surgical Endoscopy Aims and scope Submit manuscript

Abstract

Background

Physical and virtual surgical simulators are increasingly being used in training technical surgical skills. However, metrics such as completion time or subjective performance checklists often show poor correlation to transfer of skills into clinical settings. We hypothesize that non-invasive brain imaging can objectively differentiate and classify surgical skill transfer, with higher accuracy than established metrics, for subjects based on motor skill levels.

Study design

18 medical students at University at Buffalo were randomly assigned into control, physical surgical trainer, or virtual trainer groups. Training groups practiced a surgical technical task on respective simulators for 12 consecutive days. To measure skill transfer post-training, all subjects performed the technical task in an ex-vivo environment. Cortical activation was measured using functional near-infrared spectroscopy (fNIRS) in the prefrontal cortex, primary motor cortex, and supplementary motor area, due to their direct impact on motor skill learning.

Results

Classification between simulator trained and untrained subjects based on traditional metrics is poor, where misclassification errors range from 20 to 41%. Conversely, fNIRS metrics can successfully classify physical or virtual trained subjects from untrained subjects with misclassification errors of 2.2% and 8.9%, respectively. More importantly, untrained subjects are successfully classified from physical or virtual simulator trained subjects with misclassification errors of 2.7% and 9.1%, respectively.

Conclusion

fNIRS metrics are significantly more accurate than current established metrics in classifying different levels of surgical motor skill transfer. Our approach brings robustness, objectivity, and accuracy in validating the effectiveness of future surgical trainers in translating surgical skills to clinically relevant environments.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3
Fig. 4

Similar content being viewed by others

References

  1. Dawe SR, Windsor JA, Broeders JAJL, Cregan PC, Hewett PJ, Maddern GJ (2014) A systematic review of surgical skills transfer after simulation-based training. Ann Surg 259:236–248. https://doi.org/10.1097/SLA.0000000000000245

    Article  PubMed  Google Scholar 

  2. Fraser SA, Klassen DR, Feldman LS, Ghitulescu GA, Stanbridge D, Fried GM (2003) Evaluating laparoscopic skills: Setting the pass/fail score for the MISTELS system. Surg Endosc 17:964–967. https://doi.org/10.1007/s00464-002-8828-4

    Article  CAS  PubMed  Google Scholar 

  3. Fraser SA, Feldman LS, Stanbridge D, Fried GM (2005) Characterizing the learning curve for a basic laparoscopic drill. Surg Endosc 19:1572–1578. https://doi.org/10.1007/s00464-005-0150-5

    Article  CAS  PubMed  Google Scholar 

  4. Fried GM, Feldman LS, Vassiliou MC, Fraser SA, Stanbridge D, Ghitulescu G, Andrew CG (2004) Proving the value of simulation in laparoscopic surgery. Ann Surg 240:518–525. https://doi.org/10.1097/01.SLA.0000136941.46529.56

    Article  PubMed  PubMed Central  Google Scholar 

  5. McCluney AL, Vassiliou MC, Kaneva PA, Cao J, Stanbridge DD, Feldman LS, Fried GM (2007) FLS simulator performance predicts intraoperative laparoscopic skill. Surg Endosc 21:1991–1995. https://doi.org/10.1007/s00464-007-9451-1

    Article  CAS  PubMed  Google Scholar 

  6. Scott DJ, Ritter EM, Tesfay ST, Pimentel EA, Nagji A, Fried GM (2008) Certification pass rate of 100% for fundamentals of laparoscopic surgery skills after proficiency-based training. Surg Endosc 22:1887–1893. https://doi.org/10.1007/s00464-008-9745-y

    Article  PubMed  Google Scholar 

  7. Satava RM (2010) Emerging trends that herald the future of surgical simulation. Surg Clin North Am 90:623–633. https://doi.org/10.1016/j.suc.2010.02.002

    Article  PubMed  Google Scholar 

  8. Fried GM (2008) FLS assessment of competency using simulated laparoscopic tasks. J Gastrointest Surg 12:210–212. https://doi.org/10.1007/s11605-007-0355-0

    Article  PubMed  Google Scholar 

  9. Poulose BK, Vassiliou MC, Dunkin BJ, Mellinger JD, Fanelli RD, Martinez JM, Hazey JW, Sillin LF, Delaney CP, Velanovich V, Fried GM, Korndorffer JR, Marks JM (2014) Fundamentals of endoscopic surgery cognitive examination: development and validity evidence. Surg Endosc 28:631–638. https://doi.org/10.1007/s00464-013-3220-0

    Article  PubMed  Google Scholar 

  10. Vassiliou MC, Dunkin BJ, Fried GM, Mellinger JD, Trus T, Kaneva P, Lyons C, Korndorffer JR, Ujiki M, Velanovich V, Kochman ML, Tsuda S, Martinez J, Scott DJ, Korus G, Park A, Marks JM (2014) Fundamentals of endoscopic surgery: creation and validation of the hands-on test. Surg Endosc 28:704–711. https://doi.org/10.1007/s00464-013-3298-4

    Article  PubMed  Google Scholar 

  11. McDougall EM (2007) Validation of surgical simulators. J Endourol 21:244–247. https://doi.org/10.1089/end.2007.9985

    Article  PubMed  Google Scholar 

  12. Scott DJ, Pugh CM, Ritter EM, Jacobs LM, Pellegrini CA, Sachdeva AK (2011) New directions in simulation-based surgical education and training: validation and transfer of surgical skills, use of nonsurgeons as faculty, use of simulation to screen and select surgery residents, and long-term follow-up of learners. Surgery 149:735–744. https://doi.org/10.1016/j.surg.2010.11.010

    Article  PubMed  Google Scholar 

  13. Aggarwal R, Grantcharov T, Moorthy K, Milland T, Papasavas P, Dosis A, Bello F, Darzi A (2007) An evaluation of the feasibility, validity, and reliability of laparoscopic skills assessment in the operating room. Ann Surg 245:992–999. https://doi.org/10.1097/01.sla.0000262780.17950.e5

    Article  PubMed  PubMed Central  Google Scholar 

  14. Vassiliou MC, Feldman LS, Andrew CG, Bergman S, Leffondré K, Stanbridge D, Fried GM (2005) A global assessment tool for evaluation of intraoperative laparoscopic skills. Am J Surg 190:107–113. https://doi.org/10.1016/j.amjsurg.2005.04.004

    Article  Google Scholar 

  15. Doyle JD, Webber EM, Sidhu RS (2007) A universal global rating scale for the evaluation of technical skills in the operating room. Am J Surg 193:551–555. https://doi.org/10.1016/j.amjsurg.2007.02.003

    Article  Google Scholar 

  16. Dawe SR, Pena GN, Windsor JA, Broeders JAJL, Cregan PC, Hewett PJ, Maddern GJ (2014) Systematic review of skills transfer after surgical simulation-based training. Br J Surg 101:1063–1076. https://doi.org/10.1002/bjs.9482

    Article  CAS  Google Scholar 

  17. Moorthy K, Munz Y (2003) Objective assessment of technical skills in surgery. Br Med J 327:1032–1037. https://doi.org/10.1136/bmj.327.7422.1032

    Article  Google Scholar 

  18. Hogle NJ, Chang L, Strong VEM, Welcome AOU, Sinaan M, Bailey R, Fowler DL (2009) Validation of laparoscopic surgical skills training outside the operating room: a long road. Surg Endosc 23:1476–1482. https://doi.org/10.1007/s00464-009-0379-5

    Article  CAS  PubMed  Google Scholar 

  19. Bosecker C, Dipietro L, Volpe B, Krebs HI (2010) Kinematic robot-based evaluation scales and clinical counterparts to measure upper limb motor performance in patients with chronic stroke. Neurorehabil Neural Repair 24:62–69. https://doi.org/10.1177/1545968309343214

    Article  PubMed  Google Scholar 

  20. Buckley CE, Kavanagh DO, Traynor O, Neary PC (2014) Is the skillset obtained in surgical simulation transferable to the operating theatre? Am J Surg 207:146–157. https://doi.org/10.1016/j.amjsurg.2013.06.017

    Article  PubMed  Google Scholar 

  21. Dai X, Zhang T, Yang H, Tang J, Carney PR, Jiang H (2017) Fast non-invasive functional diffuse optical tomography for brain imaging. J Biophotonics. https://doi.org/10.1002/jbio.201600267

    Article  PubMed  Google Scholar 

  22. Leff DR, Orihuela-Espina F, Elwell CE, Athanasiou T, Delpy DT, Darzi AW, Yang G-Z (2011) Assessment of the cerebral cortex during motor task behaviours in adults: a systematic review of functional near infrared spectroscopy (fNIRS) studies. Neuroimage 54:2922–2936. https://doi.org/10.1016/j.neuroimage.2010.10.058

    Article  PubMed  Google Scholar 

  23. Leff DR, Elwell CE, Orihuela-Espina F, Atallah L, Delpy DT, Darzi AW, Yang GZ (2008) Changes in prefrontal cortical behaviour depend upon familiarity on a bimanual co-ordination task: an fNIRS study. Neuroimage 39:805–813. https://doi.org/10.1016/j.neuroimage.2007.09.032

    Article  PubMed  Google Scholar 

  24. James DRC, Orihuela-Espina F, Leff DR, Sodergren MH, Athanasiou T, Darzi AW, Yang G-Z (2011) The ergonomics of natural orifice translumenal endoscopic surgery (NOTES) navigation in terms of performance, stress, and cognitive behavior. Surgery 149:525–533. https://doi.org/10.1016/j.surg.2010.11.019

    Article  PubMed  Google Scholar 

  25. Ohuchida K, Kenmotsu H, Yamamoto A, Sawada K, Hayami T, Morooka K, Takasugi S, Konishi K, Ieiri S, Tanoue K, Iwamoto Y, Tanaka M, Hashizume M (2009) The frontal cortex is activated during learning of endoscopic procedures. Surg Endosc 23:2296–2301. https://doi.org/10.1007/s00464-008-0316-z

    Article  PubMed  Google Scholar 

  26. Shewokis PA, Shariff FU, Liu Y, Ayaz H, Castellanos A, Lind DS (2017) Acquisition, retention and transfer of simulated laparoscopic tasks using fNIR and a contextual interference paradigm. Am J Surg 213:336–345. https://doi.org/10.1016/j.amjsurg.2016.11.043

    Article  PubMed  Google Scholar 

  27. Andreu-Perez J, Leff DR, Shetty K, Darzi A, Yang G-Z (2016) Disparity in frontal lobe connectivity on a complex bimanual motor task aids in classification of operator skill level. Brain Connect 6:375–388. https://doi.org/10.1089/brain.2015.0350

    Article  PubMed  Google Scholar 

  28. Modi HN, Singh H, Yang G-Z, Darzi A, Leff DR, Mansur A, Yang AE, Darzi G-Z, Leff A DR (2017) A decade of imaging surgeons’ brain function (part I): terminology, techniques, and clinical translation. Surgery 28:2189–2198. https://doi.org/10.1016/j.surg.2017.05.021

    Article  Google Scholar 

  29. Nemani A, Kruger U, Intes X, De S (2017) Increased sensitivity in discriminating surgical motor skills using prefrontal cortex activation over established metrics. In: Optics in the life sciences congress. OSA, San Diego, p 11

    Google Scholar 

  30. Soper NJ, Fried GM (2008) The fundamentals of laparoscopic surgery: its time has come. Bull Am Coll Surg 93:30–32

    PubMed  Google Scholar 

  31. Peters JH, Fried GM, Swanstrom LL, Soper NJ, Sillin LF, Schirmer B, Hoffman K, the SAGES FLS Committee the SF (2004) Development and validation of a comprehensive program of education and assessment of the basic fundamentals of laparoscopic surgery. Surgery 135:21–27. https://doi.org/10.1016/S0039-6060(03)00156-9

    Article  PubMed  Google Scholar 

  32. Zhang L, Sankaranarayanan G, Arikatla VS, Ahn W, Grosdemouge C, Rideout JM, Epstein SK, De S, Schwaitzberg SD, Jones DB, Cao CGL (2013) Characterizing the learning curve of the VBLaST-PT(©) (Virtual basic laparoscopic skill trainer). Surg Endosc 27:3603–3615. https://doi.org/10.1007/s00464-013-2932-5

    Article  PubMed  PubMed Central  Google Scholar 

  33. Maciel A, Liu Y, Ahn W, Singh TP, Dunnican W, De S (2008) Development of the VBLaST: a virtual basic laparoscopic skill trainer. Int J Med Robot Comput Assist Surg 4:131–138. https://doi.org/10.1002/rcs.185

    Article  Google Scholar 

  34. Arikatla VS, Sankaranarayanan G, Ahn W, Chellali A, De S, Caroline GL, Hwabejire J, DeMoya M, Schwaitzberg S, Jones DB (2013) Face and construct validation of a virtual peg transfer simulator. Surg Endosc 27:1721–1729. https://doi.org/10.1007/s00464-012-2664-y

    Article  PubMed  Google Scholar 

  35. Sankaranarayanan G, Lin H, Arikatla VS, Mulcare M, Zhang L, Derevianko A, Lim R, Fobert D, Cao C, Schwaitzberg SD, Jones DB, De S (2010) Preliminary face and construct validation study of a virtual basic laparoscopic skill trainer. J Laparoendosc Adv Surg Tech 20:153–157. https://doi.org/10.1089/lap.2009.0030

    Article  Google Scholar 

  36. Chellali A, Ahn W, Sankaranarayanan G, Flinn JT, Schwaitzberg SD, Jones DB, De S, Cao CGL (2015) Preliminary evaluation of the pattern cutting and the ligating loop virtual laparoscopic trainers. Surg Endosc 29:815–821. https://doi.org/10.1007/s00464-014-3764-7

    Article  CAS  PubMed  Google Scholar 

  37. Nemani A, Ahn W, Cooper C, Schwaitzberg S, De S Convergent validation and transfer of learning studies of a virtual reality-based pattern cutting simulator. Surg Endosc. https://doi.org/10.1007/s00464-017-5802-8

  38. Hikosaka O, Nakamura K, Sakai K, Nakahara H (2002) Central mechanisms of motor skill learning. Curr Opin Neurobiol 12:217–222. https://doi.org/10.1016/S0959-4388(02)00307-0

    Article  CAS  PubMed  Google Scholar 

  39. Wolpert DM, Diedrichsen J, Flanagan JR (2011) Principles of sensorimotor learning. Nat Rev Neurosci 12:739–751. https://doi.org/10.1038/nrn3112

    Article  CAS  PubMed  Google Scholar 

  40. Swinnen SP (2002) Intermanual coordination: from behavioural principles to neural-network interactions. Nat Rev Neurosci 3:348–359. https://doi.org/10.1038/nrn807

    Article  CAS  PubMed  Google Scholar 

  41. Swinnen SP, Gooijers J (2015) Bimanual Coordination. In: Brain mapp, 1st edn. Elsevier, San Diego, pp 475–482

    Chapter  Google Scholar 

  42. Swinnen SP, Wenderoth N (2004) Two hands, one brain: cognitive neuroscience of bimanual skill. Trends Cogn Sci 8:18–25

    Article  PubMed  Google Scholar 

  43. Nemani A, Intes X, De S (2014) Monte Carlo based simulation of sensitivity curvature for evaluating optimal probe geometry. In: Biomedical optics 2014. OSA, Washington, D.C., p BM3A.36

    Chapter  Google Scholar 

  44. Cope M, Delpy DT (1988) System for long-term measurement of cerebral blood and tissue oxygenation on newborn infants by near infra-red transillumination. Med Biol Eng Comput 26:289–294. https://doi.org/10.1007/BF02447083

    Article  CAS  PubMed  Google Scholar 

  45. Delpy DT, Cope M, Zee P, van der Arridge S, Wray S, Wyatt J (1988) Estimation of optical pathlength through tissue from direct time of flight measurement. Phys Med Biol 33:1433–1442. https://doi.org/10.1088/0031-9155/33/12/008

    Article  CAS  PubMed  Google Scholar 

  46. Huppert TJ, Diamond SG, Franceschini MA, Boas DA (2009) Homer: a review of time-series analysis methods for near-infrared spectroscopy of the brain. Appl Opt 48:280–298. https://doi.org/10.1364/AO.48.00D280

    Article  Google Scholar 

  47. Franceschini MA, Joseph DK, Huppert TJ, Diamond SG, Boas DA (2006) Diffuse optical imaging of the whole head. J Biomed Opt 11:1–22. https://doi.org/10.1117/1.2363365

    Article  Google Scholar 

  48. Zhang Y, Brooks DH, Franceschini MA, Boas DA (2005) Eigenvector-based spatial filtering for reduction of physiological interference in diffuse optical imaging. J Biomed Opt 10:1–11. https://doi.org/10.1117/1.1852552

    Article  Google Scholar 

  49. Gagnon L, Yücel MA, Boas DA, Cooper RJ (2014) Further improvement in reducing superficial contamination in NIRS using double short separation measurements. Neuroimage 85:127–135. https://doi.org/10.1016/j.neuroimage.2013.01.073

    Article  PubMed  Google Scholar 

  50. Tabachnick BG, Fidell LS (2007) Using multivariate statistics, 5th edn. Allyn & Bacon, Needham Heights

    Google Scholar 

  51. Silverman BW (1986) Density estimation for statistics and data analysis. CRC press, Boca Raton

    Book  Google Scholar 

  52. Ahlberg G, Enochsson L, Gallagher AG, Hedman L, Hogman C, McClusky DA, Ramel S, Smith CD, Arvidsson D (2007) Proficiency-based virtual reality training significantly reduces the error rate for residents during their first 10 laparoscopic cholecystectomies. Am J Surg 193:797–804. https://doi.org/10.1016/j.amjsurg.2006.06.050

    Article  PubMed  Google Scholar 

  53. Seymour NE, Gallagher AG, Roman SA, O’Brien MK, Bansal VK, Andersen DK, Satava RM (2002) Virtual reality training improves operating room performance: results of a randomized, double-blinded study. Ann Surg 236:458–463. https://doi.org/10.1097/01.SLA.0000028969.51489.B4

    Article  PubMed  PubMed Central  Google Scholar 

  54. Ahlberg G, Hultcrantz R, Jaramillo E, Lindblom A, Arvidsson D (2005) Virtual reality colonoscopy simulation: a compulsory practice for the future colonoscopist? Endoscopy 37:1198–1204. https://doi.org/10.1055/s-2005-921049

    Article  CAS  PubMed  Google Scholar 

  55. Grantcharov TP, Kristiansen VB, Bendix J, Bardram L, Rosenberg J, Funch-Jensen P (2004) Randomized clinical trial of virtual reality simulation for laparoscopic skills training. Br J Surg 91:146–150. https://doi.org/10.1002/bjs.4407

    Article  CAS  Google Scholar 

  56. Sakai K, Hikosaka O, Miyauchi S, Takino R, Sasaki Y, Pütz B (1998) Transition of brain activation from frontal to parietal areas in visuomotor sequence learning. J Neurosci 18:1827–1840

    Article  CAS  PubMed  Google Scholar 

  57. Honda M (1998) Dynamic cortical involvement in implicit and explicit motor sequence learning: a PET study. Brain 121:2159–2173. https://doi.org/10.1093/brain/121.11.2159

    Article  PubMed  Google Scholar 

  58. Nakamura K, Sakai K, Hikosaka O (1998) Neuronal activity in medial frontal cortex during learning of sequential procedures. J Neurophysiol 80:2671–2687

    Article  CAS  PubMed  Google Scholar 

  59. Klintsova AY, Greenough WT (1999) Synaptic plasticity in cortical systems. Curr Opin Neurobiol 9:203–208

    Article  CAS  PubMed  Google Scholar 

  60. Wolpert DM, Ghahramani Z, Flanagan JR (2001) Perspectives and problems in motor learning. Trends Cogn Sci 5:487–494. https://doi.org/10.1016/S1364-6613(00)01773-3

    Article  PubMed  Google Scholar 

  61. Serrien DJ, Ivry RB, Swinnen SP (2006) Dynamics of hemispheric specialization and integration in the context of motor control. Nat Rev Neurosci 7:160–166. https://doi.org/10.1038/nrn1849

    Article  CAS  PubMed  Google Scholar 

  62. Gagnon L, Cooper RJ, Yücel MA, Perdue KL, Greve DN, Boas DA (2012) Short separation channel location impacts the performance of short channel regression in NIRS. Neuroimage 59:2518–2528. https://doi.org/10.1016/j.neuroimage.2011.08.095

    Article  PubMed  Google Scholar 

Download references

Acknowledgements

The authors would like to thank the medical student subjects and their dedication for this study. The authors would also like to thank the anatomical gift program and the gross anatomy lab at University at Buffalo for their support regarding the ex-vivo cadaveric samples. We would also like to thank Arthur “Buzz” DiMartino and his team at TechEn in graciously providing us support with the CW6 spectrometer.

Funding

This work is supported by NIBIB 1R01EB014305, NHBLI 1R01HL119248, and NCI 1R01CA197491 grants that were awarded to Suvranu De.

Author information

Authors and Affiliations

  1. Rensselaer Polytechnic Institute, 110, 8th Street, Troy, NY, 12180, USA

    Arun Nemani, Uwe Kruger, Xavier Intes & Suvranu De

  2. University at Buffalo School of Medicine and Biomedical Sciences, Buffalo, NY, 14228, USA

    Clairice A. Cooper & Steven D. Schwaitzberg

Authors
  1. Arun Nemani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Uwe Kruger
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Clairice A. Cooper
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Steven D. Schwaitzberg
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xavier Intes
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Suvranu De
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Suvranu De.

Ethics declarations

Disclosures

Drs. Arun Nemani, Uwe Kruger, Meryem Yucel, Clairice Cooper, Steven Schwaitzberg, Xavier Intes, and Suvranu De have no conflicts of interest or financial ties to disclose.

Additional information

Presented at ACS Clinical Congress 2017, San Diego, CA.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Nemani, A., Kruger, U., Cooper, C.A. et al. Objective assessment of surgical skill transfer using non-invasive brain imaging. Surg Endosc 33, 2485–2494 (2019). https://doi.org/10.1007/s00464-018-6535-z

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00464-018-6535-z

Keywords

Search

Navigation