Abstract
This study is aimed at evaluating the impact of different technical solutions of a virtual reality simulator to support the assessment of advanced human–machine interfaces for hydraulic excavator based on a new coordinated control paradigm and haptic feedbacks. By mimicking the end-effector movements, the control is conceived to speed up the learning process for novice operators and to reduce the mental overload on those already trained. The design of the device can fail if ergonomics, usability and performance are not grounded on realistic simulations where the combination of visual, auditory and haptic feedbacks make the users feel like being in a real environment rather than a computer-generated one. For this reason, a testing campaign involving 10 subjects was designed to discriminate the optimal set-up for the hardware to ensure a higher immersion into the VR experience. Both the audio–video configurations of the simulator (head-mounted display and surround system vs. monitor and embedded speakers) and the two types of haptic feedback for the soil–bucket interaction (contact vs. shaker) are compared in three different scenarios. The performance of both the users and simulator are evaluated by processing subjective and objective data. The results show how the immersive set-up improves the users’ efficiency and ergonomics without putting any extra mental or physical effort on them, while the preferred haptic feedback (contact) is not the more efficient one (shaker).
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Akyeampong J, Udoka S, Caruso G, Bordegoni M (2014) Evaluation of hydraulic excavator Human-Machine Interface concepts using NASA TLX. Int J Ind Ergon 44(3):374–382. https://doi.org/10.1016/j.ergon.2013.12.002
Andreano J, Liang K, Kong L, Hubbard D, Wiederhold BK, Wiederhold MD (2009) Auditory cues increase the hippocampal response to unimodal virtual reality. Cyberpsychol Behav 12(3):309–313. https://doi.org/10.1089/cpb.2009.0104
Bachman P, Milecki A (2019) Safety improvement of industrial drives manual control by application of haptic joystick. In: Proceedings of the international conference on intelligent systems in production engineering and maintenance, pp 563–573. https://doi.org/10.1007/978-3-319-97490-3_54
Bommer SC, Fendley M (2018) A theoretical framework for evaluating mental workload resources in human systems design for manufacturing operations. Int J Ind Ergon 63:7–17. https://doi.org/10.1016/j.ergon.2016.10.007
Burdea GC (1996) Force and touch feedback for virtual reality. Wiley, New York
CS M, Sairam PS, Veeramalla V, Kumar A, Gupta MK, (2020) Design and analysis of new haptic joysticks for enhancing operational skills in excavator control. J Mech Des 142(12):1–8. https://doi.org/10.1115/1.4048462
Chacko V, Yu H, Cang S, Vladareanu L (2014) State of the art in excavators. In: Proceedings of the international conference on advanced mechatronic systems. IEEE, pp 481–488. https://doi.org/10.1109/ICAMechS.2014.6911593
Cohen J (2013) Statistical power analysis for the behavioral sciences. Academic Press, Cambridge
Dadhich S, Bodin U, Andersson U (2016) Key challenges in automation of earth-moving machines. Autom Constr 68:212–222. https://doi.org/10.1016/j.autcon.2016.05.009
Dopico D, Luaces A, Gonzalez M (2010) A soil model for a hydraulic simulator excavator based on real-time multibody dynamics. In: Proceedings of the 5th Asian conference on multibody dynamics 2010 (ACMD ’10) 1(c):325–333. https://doi.org/10.1299/jsmeacmd.2010.5._63245-1_
Fang Y, Teizer J, Marks E (2014) A framework for developing an as-built virtual environment to advance training of crane operators. In: Construction research congress, Reston, VA, pp 31–40. https://doi.org/10.1061/9780784413517.004
Feng H, Yin C, Li R, Ma W, Yu H, Cao D, Zhou J (2019) Flexible virtual fixtures for human-excavator cooperative system. Autom Constr 106(2018):11. https://doi.org/10.1016/j.autcon.2019.102897
Frankel JG (2004) Development of a haptic backhoe testbed. Master Thesis, Georgia Institute of Technology. https://smartech.gatech.edu/handle/1853/4980
Freina L, Ott M (2015) A literature review on immersive virtual reality in education: state of the art and perspective. In: The international scientific conference elearning and software for education, 01, pp 133–141. https://doi.org/10.12753/2066-026X-15-020
Gonzalez M, Luaces A, Dopico D, Cuadrado J (2009) A 3D physics-based hydraulic excavator simulator. In: ASME-AFM 2009 world conference on innovative virtual reality, ASMEDC, pp 75–80. https://doi.org/10.1115/WINVR2009-734
Hart SG, Staveland LE (1988) Development of NASA-TLX (task load index): results of empirical and theoretical research. Adv Psychol 52:139–183. https://doi.org/10.1016/S0166-4115(08)62386-9
Hayn H, Schwarzm D (2010) A haptically enhanced operational concept for a hydraulic excavator. In: Advances in haptics. InTech, pp 199–221. https://doi.org/10.5772/8714
Hughes K, Jiang X (2010) Using discrete event simulation to model excavator operator performance. Hum Factors Ergonomics Manuf Serv Ind 20(5):408–423. https://doi.org/10.1002/hfm.20191
Kim D, Kim J, Lee K, Park C, Song J, Kang D (2009) Excavator tele-operation system using a human arm. Autom Constr 18(2):173–182. https://doi.org/10.1016/j.autcon.2008.07.002
Kot T, Novák P (2014) Utilization of the oculus rift HMD in mobile robot teleoperation. Appl Mech Mater 555:199–208. https://doi.org/10.4028/www.scientific.net/AMM.555.199
Kuijt-Evers L, Krause F, Vink P (2003) Aspects to improve cabin comfort of wheel loaders and excavators according to operators. Appl Ergon 34(3):265–271. https://doi.org/10.1016/S0003-6870(03)00032-2
Le QH, Bae JI, Jeong YM, Nguyen CT, Yang SY (2015) Development of the flexible observation system for a virtual reality excavator using the head tracking system. In: 2015 15th international conference on control, automation and systems (ICCAS). IEEE, Iccas, pp 841–846. https://doi.org/10.1109/ICCAS.2015.7364738
Leap Motion (2012) Ultraleap. https://www.ultraleap.com/product/leap-motion-controller/. Accessed 10 July 2021
Lee J, Kim B, Sun D, Han C, Ahn Y (2019) Development of unmanned excavator vehicle system for performing dangerous construction work. Sensors 19(22):4853. https://doi.org/10.3390/s19224853
Morosi F, Rossoni M, Caruso G (2019) Coordinated control paradigm for hydraulic excavator with haptic device. Autom Constr 105:102848. https://doi.org/10.1016/j.autcon.2019.102848
Nezami EG, Hashash YMA, Zhao D, Ghaboussi J (2007) Simulation of front end loader bucket-soil interaction using discrete element method. Int J Numer Anal Meth Geomech 31(9):1147–1162. https://doi.org/10.1002/nag.594
Ni T, Zhang H, Yu C, Zhao D, Liu S (2013) Design of highly realistic virtual environment for excavator simulator. Comput Electric Eng 39(7):2112–2123. https://doi.org/10.1016/j.compeleceng.2013.06.010
Oculus Rift (2016) Oculus vr. https://www.oculus.com/rift. Accessed 10 July 2021
Pla-Castells M, Garcáa-Fernàndez I, Gamón M, Martínez R (2009) Interactive earthmoving simulation in real-time. In: Congreso Espanol de Informatica Grafica (CEIG 09), p 4, https://www.researchgate.net/publication/235223042_ Interactive_earthmoving_simulation_in_real-time
Potyondy DO (2015) The bonded-particle model as a tool for rock mechanics research and application: current trends and future directions. Geosyst Eng 18(1):1–28. https://doi.org/10.1080/12269328.2014.998346
Sekizuka R, Ito M, Saiki S, Yamazaki Y, Kurita Y (2019) Evaluation system for hydraulic excavator operation skill using remote controlled excavator and virtual reality. In: 2019 IEEE/RSJ international conference on intelligent robots and systems (IROS). IEEE, pp 3229–3234. https://doi.org/10.1109/IROS40897.2019.8968213
Slater M, Usoh M, Chrysanthou Y (1995) The influence of dynamic shadows on presence in immersive virtual environments. In: Proceedings of the virtual environments conference. Springer, Vienna, Eurographics, pp 8–21. https://doi.org/10.1007/978-3-7091-9433-1_2
So JCY, Proctor RW, Dunston PS, Wang X (2013) Better retention of skill operating a simulated hydraulic excavator after part-task than after whole-task training. J Hum Factors Ergonomics Soc 55(2):449–460. https://doi.org/10.1177/0018720812454292
SPSS Statistics (2021) IBM. https://www.ibm.com/products/spss-statistics. Accessed 10 July 2021
Stahl M, Konietzky H (2011) Discrete element simulation of ballast and gravel under special consideration of grain-shape, grain-size and relative density. Granul Matter 13(4):417–428. https://doi.org/10.1007/s10035-010-0239-y
Torres-Rodriguez H, Parra-Vega V, Ruiz-Sanchez F (2005) Dynamic haptic training system for the operation of an excavator. In: 1st International conference on electrical and electronics engineering, 2004. IEEE, pp 350–355. https://doi.org/10.1109/ICEEE.2004.1433907
Ueda Y, Yamano I, Maeno T (2004) Development of a grounded haptic device and a 5-fingered robot hand for dexterous teleoperation. In: Proceedings of the international symposium on robotics and automation. IEEE, Queretaro, Mexico, p 8, https://pdfs.semanticscholar.org/0ffe/a32b062ac23f5860c6e 0842cee7519f1ca27.pdf
Unity 3D (2019) Unity technologies. https://www.unity.com. Accessed 10 July 2021
Van Der Linde R, Lammertse P, Frederiksen E, De Ruiter B (2002) The HapticMaster, a new high-performance haptic interface. In: Procedings of EuroHaptic, Edinburgh, UK, pp 1–5. http://www.eurohaptics.vision.ee.ethz.ch/2002.shtml
Wang X, Dunston PS, Proctor R, Hou L, So JCY (2011) Reflections on using a game engine to develop a virtual training system for construction excavator operators. In: Proceedings of the 28th international symposium on automation and robotics in construction (ISARC ’11), pp 631–636. https://doi.org/10.22260/ISARC2011/0118
Winck RC, Elton M, Book WJ (2015) A practical interface for coordinated position control of an excavator arm. Autom Constr 51(C):46–58. https://doi.org/10.1016/j.autcon.2014.12.012
Witmer BG, Singer MJ (1998) Measuring presence in virtual environments: a presence questionnaire. Presence Teleoper Virtual Environ 7(3):225–240. https://doi.org/10.1162/105474698565686
Zhang Y, Liu H, Kang SC, Al-Hussein M (2020) Virtual reality applications for the built environment: research trends and opportunities. Autom Constr 118:103311. https://doi.org/10.1016/j.autcon.2020.103311
Acknowledgements
The authors would like to thank Jithin Varghese and Ardalan Amiri for the contribution provided in the development of this research as well as all the participants involved in the testing activities.
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher's Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Rights and permissions
About this article
Cite this article
Morosi, F., Caruso, G. Configuring a VR simulator for the evaluation of advanced human–machine interfaces for hydraulic excavators. Virtual Reality 26, 801–816 (2022). https://doi.org/10.1007/s10055-021-00598-z
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10055-021-00598-z