Abstract
Human limb movement sensing is crucial in different areas of science. In this paper, a method for sensing human limb movement and the subsequent reconstruction in a 3-D plane is described. The sensors used in this task are four Microsoft Kinect, which has depth and RGB cameras. Depth images are processed by artificial vision algorithms to delimit an area where the movements will be performed. In the other hand, RGB images are processed by a Convolutional Neural Network to acquire a series of specific points which correspond to the human body’s joints. A comparison of the proposed algorithm performance is also described. The equations that relate the information in two dimensions are obtained by processing the four sensors are used to generate a skeleton in 3-D.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Tsai, A.-C., Luh, J.-J., Lin, T.-T.: A novel STFT-ranking feature of multi-channel EMG for motion pattern recognition. Expert Syst. Appl. 42, 3327–3341 (2015)
Brahmi, B., Saad, M., Ochoa-Luna, C., Rahman, M.H., Brahmi, A.: Adaptive tracking control of an exoskeleton robot with uncertain dynamics based on estimated time-delay control. IEEE/ASME Trans. Mechatron. 23, 575–585 (2018)
Yang, C., Chang, S., Liang, P., Li, Z., Su, C.Y.: Teleoperated robot writing using EMG signals. In: 2015 IEEE International Conference on Information and Automation, pp. 2264–2269 (2015)
Liu, L., Zhang, Y., Liu, G., Xu, W.: Variable motion mapping to enhance stiffness discrimination and identification in robot hand teleoperation. Robot. Comput.-Integr. Manuf. 51, 202–208 (2018)
Toyama, G., Hashida, T.: A detachable exoskeleton interface that duplicates the user’s hand posture and motions. In: Proceedings of the 9th Augmented Human International Conference, pp. 1–5. ACM, Seoul (2018)
Won, J.H., Yang, G.H., Jeon, J.W.: A novel wireless vibrotactile display device for representing 3DOF force feedback in teleoperation. In: 2015 IEEE International Conference on Advanced Intelligent Mechatronics (AIM), pp. 1749–1753 (2015)
Du, Y.-C., Shih, C.-B., Fan, S.-C., Lin, H.-T., Chen, P.-J.: An IMU-compensated skeletal tracking system using Kinect for the upper limb. Microsystem Technologies (2018)
Brüggemann, B., Röhling, T., Welle, J.: Coupled human-machine tele-manipulation. Procedia Manuf. 3, 998–1005 (2015)
Kim, M.K., Ryu, K., Oh, Y., Oh, S.R., Kim, K.: Implementation of real-time motion and force capturing system for tele-manipulation based on sEMG signals and IMU motion data. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 5658–5664 (2014)
Lu, Z., Cheng, D., Dong, H., Yu, Y.: Study of human motion capture and robot motion replay for lower limb exoskeleton. In: 2017 IEEE International Conference on Cyborg and Bionic Systems (CBS), pp. 206–210 (2017)
OpenKinect. https://openkinect.org/wiki/Main_Page. Accessed 14 Aug 2019
Microsoft Kinect Xbox 360. https://support.xbox.com/es-MX/xbox-360/accessories/kinect-sensor-components. Accessed 14 Aug 2019
Hafiane, S., Salih, Y., Malik, A.S.: 3D hand recognition for telerobotics. In: 2013 IEEE Symposium on Computers & Informatics (ISCI), pp. 132–137 (2013)
Zhao, W.: A concise tutorial on human motion tracking and recognition with Microsoft Kinect. Sci. China Inf. Sci. 59, 93101 (2016)
Wei, S., Ramakrishna, V., Kanade, T., Sheikh, Y.: Convolutional pose machines. In: 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 4724–4732 (2016)
Bogo, F., Black, M.J., Loper, M., Romero, J.: Detailed full-body reconstructions of moving people from monocular RGB-D sequences. In: Proceedings of the 2015 IEEE International Conference on Computer Vision (ICCV), pp. 2300–2308. IEEE Computer Society (2015)
Ajili, I., Mallem, M., Didier, J.Y.: Gesture recognition for humanoid robot teleoperation. In: 2017 26th IEEE International Symposium on Robot and Human Interactive Communication (RO-MAN), pp. 1115–1120 (2017)
Guanglong, D.U., Zhang, P.: Human–manipulator interface using hybrid sensors with Kalman filters and adaptive multi-space transformation. Measurement 55, 413–422 (2014)
Munoz, J.M., Avalos, J., Ramos, O.E.: Image-driven drawing system by a NAO robot. In: 2017 Electronic Congress (E-CON UNI), pp. 1–4 (2017)
Saini, S., Zakaria, N., Rambli, D.R.A., Sulaiman, S.: Markerless human motion tracking using hierarchical multi-swarm cooperative particle swarm optimization. PLoS ONE 10, e0127833 (2015)
Kim, Y., Baek, S., Bae, B.-C.: Motion capture of the human body using multiple depth sensors. ETRI J. 39, 181–190 (2017)
Moon, S., Park, Y., Ko, D.W., Suh, I.H.: Multiple Kinect sensor fusion for human skeleton tracking using Kalman filtering. Int. J. Adv. Rob. Syst. 13, 65 (2016)
Córdova-Esparza, D.-M., Terven, J.R., Jiménez-Hernández, H., Vázquez-Cervantes, A., Herrera-Navarro, A.-M., Ramírez-Pedraza, A.: Multiple Kinect V2 calibration. Automatika 57, 810–821 (2016)
Núñez, J.C., Cabido, R., Montemayor, A.S., Pantrigo, J.J.: Real-time human body tracking based on data fusion from multiple RGB-D sensors. Multimed. Tools Appl. 76, 4249–4271 (2017)
Cao, Z., Simon, T., Wei, S., Sheikh, Y.: Realtime multi-person 2D pose estimation using part affinity fields. In: 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), pp. 1302–1310 (2017)
Zuher, F., Romero, R.: Recognition of human motions for imitation and control of a humanoid robot. In: 2012 Brazilian Robotics Symposium and Latin American Robotics Symposium, pp. 190–195 (2012)
Xu, Y., Yang, C., Zhong, J., Wang, N., Zhao, L.: Robot teaching by teleoperation based on visual interaction and extreme learning machine. Neurocomputing 275, 2093–2103 (2018)
Ajili, I., Mallem, M., Didier, J.-Y.: Robust human action recognition system using Laban Movement Analysis. Procedia Comput. Sci. 112, 554–563 (2017)
Cerulo, I., Ficuciello, F., Lippiello, V., Siciliano, B.: Teleoperation of the SCHUNK S5FH under-actuated anthropomorphic hand using human hand motion tracking. Robot. Auton. Syst. 89, 75–84 (2017)
Ali, S.K., Firdaus, A.R., Tokhi, M.O., Al-Rezage, G.: Tracking human upper-limb movements with sliding mode control type-II fuzzy logic. In: 2016 21st International Conference on Methods and Models in Automation and Robotics (MMAR), pp. 426–431 (2016)
Caffe. https://caffe.berkeleyvision.org/. Accessed 14 Aug 2019
MPII Human Pose Dataset. http://human-pose.mpi-inf.mpg.de/. Accessed 14 Aug 2019
Orbbec. https://orbbec3d.com/. Accessed 14 Aug 2019
Acknowledgements
The authors would like to thank the Instituto Politécnico Nacional for the support to carry out this research. H. Sossa appreciates the economic support received from the SIP-IPN and CONACYT under grants 20190007 and 65 (Frontiers of Science), respectively, to conduct this investigation. J. Cruz appreciates the economic support received from the SIP-IPN under grants 20195940 to conduct this investigation.
Author information
Authors and Affiliations
Corresponding authors
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Cruz-Silva, J.E., Montiel-Pérez, J.Y., Sossa-Azuela, H. (2019). 3-D Human Body Posture Reconstruction by Computer Vision. In: Martínez-Villaseñor, L., Batyrshin, I., Marín-Hernández, A. (eds) Advances in Soft Computing. MICAI 2019. Lecture Notes in Computer Science(), vol 11835. Springer, Cham. https://doi.org/10.1007/978-3-030-33749-0_46
Download citation
DOI: https://doi.org/10.1007/978-3-030-33749-0_46
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-33748-3
Online ISBN: 978-3-030-33749-0
eBook Packages: Computer ScienceComputer Science (R0)