Abstract
This paper presents the contactless measuring of breathing using the MS Kinect depth sensor and compares the results obtained with records of breathing taken by polysomnography (PSG). We explore the methods of signal denoising, resampling, and spectral analysis of acquired data as well as feature extraction and their Bayesian classification. The proposed methodology was applied for analysis of the long-term monitoring of individuals who were observed simultaneously by PSG and MS Kinect in the sleep laboratory. After time synchronization of polysomnographic and MS Kinect video data, features were extracted from both signals and compared. The average error of the frequency while being evaluated by MS Kinect that was related to that obtained by PSG was 3.75 %. The mean accuracy of the Bayesian classification of features into two classes (i.e. wake or sleep) was 88.90 and 88.95 % for the PSG and MS Kinect measurements, respectively. The strong likeness of features supports the hypothesis that contactless techniques may represent a valid alternative to the present approach of sleep monitoring, thereby allowing data acquisition in the home environment as well.
Similar content being viewed by others
References
Alnowami, M., Alnwaimi, B., Tahavori, F., Copland, M., Wells, K.: A quantitative assessment of using the Kinect for Xbox360 for respiratory surface motion tracking. In: Proc. of SPIE 8316, Medical Imaging, pp. 1–10 (2012)
Aoki, H., Miyazaki, M., Nakamura, H., Furukawa, R., Sagawa, R., Kawasaki, H.: Non-contact respiration measurement using structured light 3-D sensor. In: Proc. of SICE Annual Conf., pp. 614–618 (2012)
Assefa, S., Diaz-Abad, M., Korotinsky, A., Tom, S., Scharf, S.M.: Comparison of a simple obstructive sleep apnea screening device with standard in-laboratory polysomnography. Sleep Breath. First online: Aug. 2015, 1–5 (2015)
Bernacchia, N., Scalise, L., Casacanditella, L., Ercoli, I., Marchionni, P., Tomasini, E.: Non contact measurement of heart and respiration rates based on Kinect. In: Int. Symp. on Medical Meas. and Appl., pp. 1–5 (2014)
Burba, N., Bolas, M., Krum, D., Suma, E.: Unobtrusive measurement of subtle nonverbal behaviors with the Microsoft Kinect. In: Proc.IEEE Virt.Reality, pp.1–4 (2012)
Carlson, B., Neelon, V., Hsiao, H.: Evaluation of a non-invasive respiratory monitoring system for sleeping subjects. Physiol. Meas. 20(1), 53–63 (1999)
Centonze, F.: Image processing and three-dimensional modeling using Microsoft Kinect v2 in analysis of sleep disorders. Thesis, Polytechnico Milano, Italy (2015)
Dafna, E., Tarasiuk, A., Zigel, Y.: Sleep-wake evaluation from whole-night non-contact audio recordings of breathing sounds. Plos One 10(2), e0117382 (2015)
Douglas, N.J., White, D.P., Pickett, C.K., Weil, J.V., Zwillich, C.W.: Respiration during sleep in normal man. Thorax 37(11), 840–844 (1982)
Erden, F., Velipasalar, S., Alkar, A.Z., Cetin, A.E.: Sensors in assisted living. IEEE Signal Process. Mag. 33(2), 36–44 (2016)
Hosťálková, E., Vyšata, O., Procházka, A.: Multi-dimensional biomedical image de-noising using Haar transform. In: Proc. of 15th Int. Conf. on Digital Signal Processing, pp. 175–178 (2007)
Kagawa, M., Ueki, K., Tojima, H., Matsui, T.: Noncontact screening system with two microwave radars for the diagnosis of sleep apnea-hypopnea syndrome. In: Proc. of the 35th Annual Int. Conf. of the IEEE: Engineering in Medicine and Biology Society, pp. 2052–2055 (2013)
Kolb, A., Barth, E., Koch, R., Larsen, R.: Time-of-Flight Sensors in Computer Graphics In: Eurographics 2009 - State of the Art Reports, pp. 119–134 (2009)
Krüger, B., Vögele, A., Lassiri, M., Herwartz, L., Terkatz, T., Weber, A., Garcia, C., Fietze, I., Penzel, T.: Sleep detection using de-identified depth data. J. Mob. Multimed. 10(3&4), 327–342 (2014)
Lee, Y.S., Pathirana, P.N., Steinfort, C.L., Caelli, T.: Monitoring and analysis of respiratory patterns using microwave Doppler radar. IEEE J. Eng. Health Med. 2, 1–12 (2014)
Lee, J., Hong, M., Ryu, S.: Sleep Monitoring System Using Kinect Sensor. Int J Distrib Sens Netw, Article ID 875371, (2015)
Long, X., Foussier, J., Fonseca, P., Haakma, R., Aarts, R.: Respiration amplitude analysis for REM and NREM sleep classification. In: Int. Conf. of the IEEE Engineering in Medicine and Biology Society, pp. 5017–5020 (2013)
Martinez, M., Stiefelhagen, R.: Breath rate monitoring during sleep using near-IR imagery and PCA. In: 21st Int. Conf. on Pattern Recognition (ICPR), vol. 48, pp. 3472–3475 (2012)
Metsis, V., Kosmopoulos, D., Athitsos, V., Makedon, F.: Non-invasive analysis of sleep patterns via multimodal sensor input. Pers Ubiquitous Comp 18, 19–26 (2014)
Penne, J., Schaller, C., Hornegger, J., Kuwert, T.: Robust real-time 3D respiratory motion detection using time-of-flight cameras. Int. J. Comput. Assist. Radiol. Surg. 3(5), 427–431 (2008)
Procházka, A., Vyšata, O.: ŤŤupa, O., Yadollahi, M., Vališ, M.: Discrimination of axonal neuropathy using sensitivity and specificity statistical measures. Neural Comput. Appl. 25(6), 1349–1358 (2015)
Procházka, A., Vyšata, O., Vališ, M., ŤŤupa, O., Schätz, M., Mařík, V.: Bayesian classification and analysis of gait disorders using image and depth sensors of Microsoft Kinect. Digit Signal Process. 47, 169–177 (2015)
Rai, R., Sontakke, T.: Implementation of image denoising using wavelet thresholding techniques. Int. J. Comput. Technol. Electron. Eng. 1(2), 6–10 (2011)
Rodríguez-Sotelo, J.L., Osorio-Forero, A., Jiménez-Rodríguez, A., Cuesta-Frau, D., Cirugeda-Roldán, E., Peluffo, D.: Automatic sleep stages classification using EEG entropy features and unsupervised pattern analysis techniques. Entropy 16, 6573–6589 (2014)
Redmond, S., Heneghan, C.: Cardiorespiratory-based sleep staging in subjects with obstructive sleep apnea. IEEE Trans. Biomed. Eng. 53(3), 485–496 (2006)
Schatz, M., Centonze, F., Kuchynka, J., Tupa, O., Vysata, O., Geman, O., Prochazka, A.: Statistical recognition of breathing by MS Kinect depth sensor. In: Int. Workshop on Computational Intelligence for Multimedia Understanding (IWCIM), pp. 1–4 (2015)
Sen, B., Peker, M., Cavusoglu, A., Celabi, F.V.: A comparative study on classification of sleep stage based on EEG signals using feature selection and classification algorithms. J. Med. Syst. 38(18), 1–21 (2014)
Stradling, J.R., Chadwick, G.A., Frew, A.J.: Changes in ventilation and its components in normal subjects during sleep. Thorax 40(5), 364–370 (1985)
Taheri, T., Anna, A.S.: Non-Invasive Breathing Rate Detection Using a Very Low Power Ultra-wide-band Radar. In: IEEE Int. Conf. on Bioinformatics and Biomedicine (BIBM), pp. 70–83 (2014)
Xia, J., Siochi, R.A.: A real-time respiratory motion monitoring system using kinect: proof of concept. Med. Phys. 39(5), 2682–2685 (2012)
Yu, M.C., Liou, J.L., Kuo, S.W., Lee, M.S., Hung, Y.P.: Noncontact respiratory measurement of volume change using depth camera. In: IEEE Int. Conf. Engineering in Medicine and Biology Society, pp. 2371–2374 (2012)
Zaffaroni, A., Kent, B., O’Hare, E., et al.: Assessment of sleep-disordered breathing using a non-contact bio-motion sensor. J. Sleep Res. 22(2), 231–236 (2014)
Acknowledgments
All measured data have been recorded in the University Hospital Hradec Králové.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Supplementary material 1 (mp4 1282 KB)
Rights and permissions
About this article
Cite this article
Procházka, A., Schätz, M., Centonze, F. et al. Extraction of breathing features using MS Kinect for sleep stage detection. SIViP 10, 1279–1286 (2016). https://doi.org/10.1007/s11760-016-0897-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-016-0897-2