Abstract
Purpose
The use of projector-based augmented reality (AR) in surgery may enable surgeons to directly view anatomical models and surgical data from the patient’s surface (skin). It has the advantages of a consistent viewing focus on the patient, an extended field of view and augmented interaction. This paper presents an AR guidance mechanism with a projector-camera system to provide the surgeon with direct visual feedback for supervision of robotic needle insertion in radiofrequency (RF) ablation treatment.
Methods
The registration of target organ models to specific positions on the patient body is performed using a surface-matching algorithm and point-based registration. An algorithm based on the extended Kalman filter and spatial transformation is used to intraoperatively compute the virtual needle’s depth in the patient’s body for AR display.
Results
Experiments of this AR system on a mannequin were conducted to evaluate AR visualization and accuracy of virtual RF needle insertion. The average accuracy of 1.86 mm for virtual needle insertion met the clinical requirement of 2 mm or better. The feasibility of augmented interaction with a surgical robot using the proposed open AR interface with active visual feedback was demonstrated.
Conclusions
The experimental results demonstrate that this guidance system is effective in assisting a surgeon to perform a robot-assisted radiofrequency ablation procedure. The novelty of the work lies in establishing a navigational procedure for percutaneous surgical augmented intervention integrating a projection-based AR guidance and robotic implementation for surgical needle insertion.
Similar content being viewed by others
References
Yamakado K, Hase S, Matsuoka T, Tanigawa N, Nakatsuka A, Takaki H, Takao M, Inoue Y, Kanazawa S, Inoue Y, Sawada S, Kusunoki M, Takeda K (2007) Radiofrequency ablation for the treatment of unresectable lung metastases in patients with colorectal cancer: a multicenter study in Japan. J Vasc Interv Radiol 18(3):393–398
Benoist S, Nordlinger B (2004) Radiofrequency ablation in liver tumours. Ann Oncol 15(Suppl. 4):iv313–iv317
Park BK, Kim CK, Choi HY, Lee HM, Jeon SS, Seo S, Han DH (2010) Limitation for performing ultrasound-guided radiofrequency ablation of small renal masses. Eur J Radiol 75(2):248–252
Cura J, Zabala R, Iriarte J, Unda M (2010) Treatment of renal tumors by percutaneous ultrasound-guided radiofrequency ablation using a multitined electrode: effectiveness and complications. Eur Urol 57:459–465
Gandhi NS, Dupuy DE (2005) Image-guided radiofrequency ablation as a new treatment option for patients with lung cancer. Semin Roentgenol 40(2):171–181
Bosch M, Daniel B, Rieke V, Butts-Pauly K, Kermit E, Jeffrey S (2008) MRI-guided radiofrequency ablation of breast cancer: preliminary clinical experience. J Magn Reson Imaging 27(1):204–208
Wacker FK, Nour SG, Eisenberg R, Duerk JL, Lewin JS (2004) MRI-guided radiofrequency thermal ablation of normal lung tissue: in vivo study in a rabbit model. AJR Am J Roentgenol 183(3):599–603
Wang X, Zhang Q, Han Q, Yang R, Carswell M, Seales B, Sutton E (2010) Endoscopic video texture mapping on pre-built 3-d anatomical objects without camera tracking. IEEE Trans Medical Imaging 29(6):1213–1223
Traub J, Sielhorst T, Heining S-M, Navab N (2008) Advanced display and visualization concepts for image guided surgery. J Disp Technol 4(4):483–490
Kellner F, Bolte B, Bruder G, Rautenberg U, Steinicke F, Lappe M, Koch R (2012) Geometric calibration of head-mounted displays and its effects on distance estimation. IEEE Trans Vis Comput Graph 18(4):589–596
Ferrari V, Megali G, Troia E, Pietrabissa A, Mosca F (2009) A 3-d mixed-reality system for stereoscopic visualization of medical dataset. IEEE Trans Biomed Eng 56(11):2627–2633
Fichtinger G, Deguet A, Masamune K, Balogh E, Fischer GS, Mathieu H, Taylor RH, Zinreich SJ, Fayad LM (2005) Image overlay guidance for needle insertion in CT scanner. IEEE Trans Biomed Eng 52(8):1415–1424
Gavaghan KA, Peterhans M, Santos TO, Weber S (2011) A portable image overlay projection device for computer-aided open liver surgery. IEEE Trans Biomed Eng 58(6):1855–1864
Krempien R, Hoppe H, Kahrs L, Daeuber S, Schorr O, Eggers G, Bischof M, Munter MW, Debus J, Harms W (2008) Projector-based augmented reality for intuitive intraoperative guidance in image-guided 3d interstitial brachytherapy. Int J Radiat Oncol Biol Phys 70(3):944–952
Wen R, Chui C-K, Lim K-B (2010) Intraoperative visual guidance and control interface for augmented reality robotic surgery. In Augmented reality-some emerging application areas. InTech, pp 191–208
Juang R, Majumder A (2007) Photometric self-calibration of a projector-camera system. In: Proceedings of IEEE conference computer vision and pattern recognition (CVPR), California, pp 1–8
Fujii K, Grossberg MD, Nayar SK (2005) A projector-camera system with real-time photometric adaptation for dynamic environments. In: Proceedings of IEEE conference computer vision and pattern recognition (CVPR), Yokosuka, pp 20–25
Griesser A, Gool LV (2006) Automatic interactive calibration of multi-projector-camera systems. In: Proceedings of IEEE conference computer vision and pattern recognition workshop (CVPRW), Zurich, pp 8–8
Taylor RH, Stoianovici D (2003) Medical robotics in computer-integrated surgery. IEEE Trans Robotics Autom 5:765–781
Yang L, Chui C-K, Chang S (2009) Design and development of an augmented reality robotic system for large tumor ablation. Int J Virtual Real 8(1):27–35
Yang L, Wen R, Qin J, Chui C-K, Lim K-B, Chang SK-Y (2010) A robotic system for overlapping radiofrequency ablation in large tumor treatment. IEEE/ASME Trans Mechatro 15(6):887–897
Feuerstein M, Mussack T, Heining SM, Navab N (2008) Intraoperative laparoscope augmentation for port placement and resection planning in minimally invasive liver resection. IEEE Trans Med Imaging 27(3):355–369
Rieder C, Kroeger T, Schumann C, Hahn HK (2011) GPU-based real-time approximation of the ablation zone for radiofrequency ablation. IEEE Trans Vis Comput Graph 17(12):1812–1821
Wen R, Yang L, Chui C-K, Lim K-B, Chang S (2010) Intraoperative visual guidance and control interface for augmented reality robotic surgery. In: Proceedings of IEEE international conference control and automation (ICCA), Singapore, pp 947–952
Bouguet JY (2010) Camera calibration toolbox for Matlab. http://www.vision.caltech.edu/bouguetj/calib_doc/index.html
Frankowski G, Hainich R (2009) DLP-based 3D metrology by structured light or projected fringe technology for life sciences and industrial metrology. In: Proceedings of SPIE, California, vol 7210. pp 72100C1–72100C12
Gao J, Kosaka A, Kak AC (2005) A multi-Kalman filtering approach for video tracking of human-delineated objects in cluttered environments. Comput Vis Image Underst 99(1):1–57
Garstka J, Peters G (2011) View-Dependent 3D Projection Using Depth-Image-based Head Tracking. In: Proceedings of IEEE interenational conference workshop. Projector-Camera Systems (PROCAMS), Colorado, Available: http://www.cs.ubc.ca/labs/imager/PROCAMS2011/program4.php
Hartley R, Zisserman A (2003) Multiple view geometry in computer vision, 2nd edn. Cambridge University Press, Cambridge, pp 24–38
Bar-Shaloom Y, Fortmann TE (1988) Tracking and data association. Academic, New York
Hostettler A, Nicolau SA, Rémond Y, Marescaux J, Soler L (2010) A real-time predictive simulation of abdominal viscera positions during quiet free breathing. Prog Biophys Mol Biol 103(2–3):169–184
Tungjitkusolmun S, Staelin ST, Haemmerich D, Tsai J-Z, Cao H, Webster JG, Lee FT Jr, Mahvi DM, Vorperian VR (2002) Three-dimensional finite-element analyses for radio-frequency hepatic tumor ablation. IEEE Trans Biomed Eng 49(1):3–9
Wen R, Chng CB, Chui CK, Lim KB, Ong SH, Chang SK (2012) Robot-assisted RF ablation with interactive planning and mixed reality guidance. IEEE/SICE international symposium on system integration (SII), Fukuoka, Japan, pp 31–36
Acknowledgments
This work was supported in part by the National University of Singapore, under Grants R-265-000-270-112 and R-265-000-270-127. The authors would like to thank Chng Chin Boon for his expertise and support in the surgical robot setup.
Conflict of Interest
None.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wen, R., Chui, CK., Ong, SH. et al. Projection-based visual guidance for robot-aided RF needle insertion. Int J CARS 8, 1015–1025 (2013). https://doi.org/10.1007/s11548-013-0897-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-013-0897-4