Abstract
Purpose
In clinical examinations, the tissue surface topology is an important feature for detecting the tissue pathology and implementing augmented reality. We have previously presented a miniaturised structured light (SL) system for recovery of tissue surface shape in minimally invasive surgery (MIS), based on a flexible multispectral structured illumination probe (1.9 mm diameter) (Clancy et al. in Biomed Opt Express 2(11):3119–3128, 2011. doi:10.1364/BOE.2.003119). This paper reports further hardware and analytical developments to improve the light pattern decoding result and increase the reconstruction accuracy.
Methods
The feasibility of using an 8-band multispectral camera with higher pattern-colour discrimination ability than normal RGB camera in this system was studied. Additionally, the “normalised cut” algorithm was investigated to improve pattern segmentation.
Results
The whole SL system was evaluated by phantom and in vivo experiments. Higher pattern identification performance than that of an RGB camera was recorded by using the multispectral camera (average precision >97 %, average sensitivity >62 %). An average of \(0.88\,\pm \,0.67\,\hbox {mm}\) reconstruction error was achieved using the proposed pattern decoding method on a heart phantom at a working distance of approximately 10 cm.
Conclusions
The experiment showed the superiority of the multispectral camera over the RGB camera in the spot identification step. The proposed pattern decoding algorithm underwent evaluations using different experiments, showing that it provided promising reconstruction results. The potential of using this system in MIS environments has been demonstrated.
Similar content being viewed by others
References
Clancy NT, Stoyanov D, Maier-Hein L, Groch A, Yang G-Z, Elson DS (2011) Spectrally encoded fiber-based structured lighting probe for intraoperative 3D imaging. Biomed Opt Express 2(11):3119–3128. doi:10.1364/BOE.2.003119
Schwartz JJ, Lichtenstein GR (2004) Magnification endoscopy, chromoendoscopy and other novel techniques in evaluation of patients with IBD. Tech Gastrointest Endosc 6(4):182–188. doi:10.1016/j.tgie.2004.09.006
Maier-Hein L, Mountney P, Bartoli A, Elhawary H, Elson D, Groch A, Kolb A, Rodrigues M, Sorger J, Speidel S, Stoyanov D (2013) Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery. Med Image Anal 17(8):974–996. doi:10.1016/j.media.2013.04.003
King AP, Edwards PJ, Maurer CR, Cunha DAd, Gaston RP, Clarkson M, Hill DLG, Hawkes DJ, Fenlon MR, Strong AJ, Cox TCS, Gleeson MJ (2000) Stereo augmented reality in the surgical microscope. Presence Teleoper Virtual Environ 9(4):360–368. doi:10.1162/105474600566862
Geng J (2011) Structured-light 3D surface imaging: a tutorial. Adv Opt Photon 3(2):128–160. doi:10.1364/AOP.3.000128
Ackerman JD, Keller K, Fuchs H (2002) Surface reconstruction of abdominal organs using laparoscopic structured light for augmented reality. Society of photo-optical instrumentation engineers, vol 4661. INTERNATIONAL, Bellingham
Chan M, Lin W, Zhou C, Qu JY (2003) Miniaturized three-dimensional endoscopic imaging system based on active stereovision. Appl Opt 42(10):1888–1898. doi:10.1364/AO.42.001888
Wu TT, Qu JY (2007) Optical imaging for medical diagnosis based on active stereo vision and motion tracking. Opt Express 15(16):10421–10426. doi:10.1364/OE.15.010421
Maurice X, Albitar C, Doignon C, de Mathelin M (2012) A structured light-based laparoscope with real-time organs’ surface reconstruction for minimally invasive surgery. In:Annual international conference of the IEEE engineering in medicine and biology society (EMBC), Aug. 28 2012-Sept. 1. pp 5769–5772. doi:10.1109/EMBC.2012.6347305
Schmalz C, Forster F, Schick A, Angelopoulou E (2012) An endoscopic 3D scanner based on structured light. Med Image Anal 16(5):1063–1072. doi:10.1016/j.media.2012.04.001
Sorg BS, Moeller BJ, Donovan O, Cao Y, Dewhirst MW (2005) Hyperspectral imaging of hemoglobin saturation in tumor microvasculature and tumor hypoxia development. BIOMEDO 10(4):044004-044004-044011. doi:10.1117/1.2003369
Tracy CR, Terrell JD, Francis RP, Wehner EF, Smith J, Litorja M, Hawkins DL, Pearle MS, Cadeddu JA, Zuzak KJ (2010) Characterization of renal ischemia using DLP hyperspectral imaging: a pilot study comparing artery-only occlusion versus artery and vein occlusion. J Endourol 24(3):321–325. doi:10.1089/end.2009.0184
Yang VXD, Muller PJ, Herman P, Wilson BC (2003) A multispectral fluorescence imaging system: design and initial clinical tests in intra-operative Photofrin-photodynamic therapy of brain tumors. Lasers Surg Med 32(3):224–232. doi:10.1002/lsm.10131
Xu H, Rice BW (2009) In-vivo fluorescence imaging with a multivariate curve resolution spectral unmixing technique. BIOMEDO 14(6):064011-064011-064019. doi:10.1117/1.3258838
Soille P (2003) Morphological image analysis: principles and applications. Springer, New York
Shi J, Malik J (2000) Normalized cuts and image segmentation. IEEE Trans Pattern Anal Mach Intell 22(8):888–905. doi:10.1109/34.868688
Heikkila J, Silven O (1997) A four-step camera calibration procedure with implicit image correction. In:Proceedings of IEEE computer society conference on computer vision and pattern recognition, 17–19 Jun 1997. pp 1106–1112. doi:10.1109/CVPR.1997.609468
Zhang Z (2000) A flexible new technique for camera calibration. IEEE Trans Pattern Anal Mach Intell 22(11):1330–1334. doi:10.1109/34.888718
Li Q, Dai C, Liu H, Liu J (2009) In:Leukemic cells segmentation algorithm based on molecular spectral imaging technology. pp 73830V-73830V-73837
Szeliski R (2010) Computer vision: algorithms and applications. Springer, New York
Ng A, Jordan M, Weiss Y (2001) On spectral clustering: analysis and an algorithm. Citeulike-article-id: 1158897
Von Luxburg U (2007) A tutorial on spectral clustering. Stat Comput 17(4):395–416. doi:10.1007/s11222-007-9033-z
Zhang Z (1994) Iterative point matching for registration of free-form curves and surfaces. Int J Comput Vis 13(2):119–152. doi:10.1007/BF01427149
Acknowledgments
This project is funded by ERC grant 242991, and the NIHR Imperial Biomedical Research Centre (BRC)/Imperial Innovations Therapeutic Primer Fund (Imperial Confidence in Concept). The authors thank Northwick Park Institute for Medical Research (NPIMR) and German Cancer Research Center (DKFZ) for collaboration in experimental data acquisition. Neil Clancy is supported by an Imperial College Junior Research Fellowship.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interest
None of the authors has a conflict of interest.
Rights and permissions
About this article
Cite this article
Lin, J., Clancy, N.T. & Elson, D.S. An endoscopic structured light system using multispectral detection. Int J CARS 10, 1941–1950 (2015). https://doi.org/10.1007/s11548-015-1264-4
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11548-015-1264-4