Abstract
Gesture-based systems allow users to interact with a virtual reality application in a natural way. Visual feedback for the gesture-based interaction technique has an impact on the performance and the hand instability making the manipulation of the object less precise. This paper investigated two new interaction techniques in a virtual environment. It describes the influence of natural and non-natural virtual feedback in the selection process using the GITDVR-G interaction technique, which consists of a grasping visual feedback. The GITDVR-G was evaluated in a virtual knee surgery training system. The results showed that it was effective in terms of the task completion time, and that the participants preferred the natural grasping visual feedback. Besides that, the precise manipulation in a newly-designed interaction technique (Precise GITDVR-G) was evaluated. The Precise GITDVR-G includes a normal manipulation mode and a precise manipulation mode that can be triggered by hand gestures. During the precise manipulation mode, an inset view will appear and move with the selected object to provide a better view to users, while the movements of the virtual hand are scaled down to improve the precision. Four different configurations of the precise manipulation technique were evaluated, and the results showed that the unimanual control method with an inset view performed better in terms of the task performance time and the subjective feedback. The finding suggested that the realistic virtual grasping visual feedback can be applied in a virtual hand interaction technique, and that the inset view feature is helpful in the precise manipulation.
Similar content being viewed by others
References
Argelaguet F, Andujar C (2009) Visual feedback techniques for virtual pointing on stereoscopic displays. In: Proceedings of the 16th ACM Symposium on Virtual Reality Software and Technology, Kyoto, Japan, 2009. ACM, 1643966, pp 163-170. https://doi.org/10.1145/1643928.1643966
Ascension Technology Corporation (2002) Flock of birds: installation and operation guide. Burlington. http://grva.lamce.coppe.ufrj.br/hardware/hw_drivers/flockofbirds/Flock_of_Birds_Manual-RevB.pdf. Accessed 12 Jun 2011
Bowman DA (1996) Conceptual Design Space: Beyond Walk-through to Immersive Design. In: Designing Digital Space: An Architect's Guide to Virtual Reality. John Wiley & Sons, New York, pp 225–236
Bowman DA, Hodges LF (1997) An evaluation of techniques for grabbing and manipulating remote objects in immersive virtual environments. In: Proceedings of the 1997 symposium on Interactive 3D graphics, Providence, Rhode Island, 1997. ACM, 253301, pp 35-38. https://doi.org/10.1145/253284.253301
Bowman DA, Kruijff E, LaViola JJ, Poupyrev I (2001) An Introduction to 3-D User Interface Design. Presence: Teleoper Virtual Environ 10(1):96–108. https://doi.org/10.1162/105474601750182342
Bowman DA, Kruijff E, LaViola JJ, Poupyrev I (2004) 3D User Interfaces: Theory and Practice. Addison Wesley Longman Publishing. http://www.amazon.com/3D-User-Interfaces-Practice-paperback/dp/0321980042
Brooks FP, Ouh-Young M, Batter JJ, Kilpatrick PJ (1990) Project GROPEHaptic displays for scientific visualization. In: Proceedings of the 17th annual conference on Computer graphics and interactive techniques, Dallas, TX, USA, 1990. ACM, 97899, pp 177-185. https://doi.org/10.1145/97879.97899
Burdea G, Coiffet P (1994) Virtual reality technology. John Wiley & Son, New York
Fabiani L, Burdea G, Langrana N, Gomez D (1996) Human interface using the Rutgers Master II force feedback interface. In: Proceedings of the IEEE Virtual Reality Annual International Symposium, 1996, 30 Mar-3 Apr 1996. pp 54-59. https://doi.org/10.1109/VRAIS.1996.490510
Faieza AAP, Pham DT, Sulaiman S, Napsiah I, Ariffin MK, BTHT B (2008) Visual Feedback and Pseudo-Haptic Feedback Improve Manual Lifting Performance. Jurnal Teknologi 49(D):157–171
Fifth Dimension Technologies (2004) 5DT data glove ultra series: user’s manual. India. http://5dt.com/downloads/dataglove/ultra/5DT%20Data%20Glove%20Ultra%20-%20Manual.pdf. Accessed 8 April 2011
Forsberg A, Herndon K, Zeleznik R (1996) Aperture based selection for immersive virtual environments. In: Proceedings of the 9th annual ACM symposium on User interface software and technology, Seattle, Washington, United States, 1996. ACM, 237105, pp 95-96. https://doi.org/10.1145/237091.237105
Frees S, Kessler GD, Kay E (2007) PRISM interaction for enhancing control in immersive virtual environments. ACM Trans Comput-Hum Interact 14(1):2. https://doi.org/10.1145/1229855.1229857
Grossman T, Balakrishnan R (2005) The bubble cursor: enhancing target acquisition by dynamic resizing of the cursor's activation area. In: Proceedings of the SIGCHI conference on Human factors in computing systems, Portland, Oregon, USA, 2005. ACM, 1055012, pp 281-290. https://doi.org/10.1145/1054972.1055012
Grossman T, Balakrishnan R (2006) The design and evaluation of selection techniques for 3D volumetric displays. In: Proceedings of the 19th annual ACM symposium on User interface software and technology, Montreux, Switzerland, 2006. ACM, 1166257, pp 3-12. https://doi.org/10.1145/1166253.1166257
Hilliges O, Izadi S, Wilson AD, Hodges S, Garcia-Mendoza A, Butz A (2009) Interactions in the air: adding further depth to interactive tabletops. In: Proceedings of the 22nd annual ACM symposium on User interface software and technology, Victoria, BC, Canada, 2009. ACM, 1622203, pp 139-148. https://doi.org/10.1145/1622176.1622203
Hou W (2008) The Three-Dimensional User Interface. In: Pinder S (ed) Advances in Human Computer Interaction. InTech, Croatia, pp 543–574. https://doi.org/10.5772/5910
Intan SR, Haslina A, Yahaya NHM, Sulong AB (2010) Development of Visualization Application (VJBK) for Newly Designed Jig and Fixture for Computer-Assisted Knee Replacement Surgery. In: International Business Information Management Conference (14th IBIMA), Istanbul Turkey, 2010. pp 882-890. http://www.ibima.org/TR2010/papers/int.html
Jacoby RH, Ferneau M, Humphries J (1994) Gestural interaction in a virtual environment. Proceedings of SPIE 1994:355–364. https://doi.org/10.1117/12.173892
Jauregui DAG, Argelaguet F, Lecuyer A (2012) Design and evaluation of 3D cursors and motion parallax for the exploration of desktop virtual environments. In: 2012 I.E. Symposium on 3D User Interfaces (3DUI), 4-5 March 2012 2012. pp 69-76. https://doi.org/10.1109/3dui.2012.6184186
Jiang H, Kessler GD, Nonnemaker J (2002) DEMIS: a dynamic event model for interactive systems. In: Proceedings of the ACM symposium on Virtual reality software and technology, Hong Kong, China, 2002. ACM, 585757, pp 97-104. https://doi.org/10.1145/585740.585757
Kadri A, Lecuyer A, Burkhardt JM (2007) The Visual Appearance of User's Avatar Can Influence the Manipulation of Both Real Devices and Virtual Objects. In: 3DUI '07. IEEE Symposium on 3D User Interfaces, 2007., 10-11 March 2007 2007. pp -. https://doi.org/10.1109/3dui.2007.340767
Kavakli M, Jayarathna D (2005) Virtual Hand: An Interface for Interactive Sketching in Virtual Reality. In: International Conference on Computational Intelligence for Modelling, Control and Automation, International Conference on Intelligent Agents, Web Technologies and Internet Commerce., 28-30 Nov. 2005 2005. pp 613–618. https://doi.org/10.1109/CIMCA.2005.1631331
Köpsel A, Majaranta P, Isokoski P, Huckauf A (2016) Effects of auditory, haptic and visual feedback on performing gestures by gaze or by hand. Behav Inform Technol 35(12):1044–1062. https://doi.org/10.1080/0144929X.2016.1194477
Kosch T, Kettner R, Funk M, Schmidt A (2016) Comparing Tactile, Auditory, and Visual Assembly Error-Feedback for Workers with Cognitive Impairments. In: Proceedings of the 18th International ACM SIGACCESS Conference on Computers and Accessibility, Reno, Nevada, USA, 2016. ACM, 2982157, pp 53-60. https://doi.org/10.1145/2982142.2982157
Lam MC, Arshad H (2011) Virtual Hand Modeling and Simulation Based on Unity 3D. International Review on Computers and Software 6 (6):1044-1049. http://connection.ebscohost.com/c/articles/89747281/virtual-hand-modeling-simulation-based-unity-3d
Liang J, Green M (1994) JDCAD: A highly interactive 3D modeling system. Computers & Graphics - CG 18(4):499–506. https://doi.org/10.1016/0097-8493(94)90062-0
Mine M (1995) Virtual Environment Interaction Techniques (trans: Science DoC). Mine95virtualenvironment. University of North Carolina at Chapel Hill Chapel Hill, NC, USA. http://dl.acm.org/citation.cfm?id=897820
Nguyen TTH, Duval T, Pontonnier C (2014) A new direct manipulation technique for immersive 3d virtual environments. In: ICAT-EGVE 2014: the 24th International Conference on Artificial Reality and Telexistence and the 19th Eurographics Symposium on Virtual Environments, 2014. p 8
Olwal A, Feiner S (2003) The Flexible Pointer: An Interaction Technique for Augmented and Virtual Reality. In: Proceedings of ACM Symposium on User Interface Software and Technology, 2003. pp 81-82. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.214.629
Ortega M (2013) 3D object position using automatic viewpoint transitions. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, Paris, France, 2013. ACM, 2470681, pp 193-196. https://doi.org/10.1145/2470654.2470681
Osawa N (2006) Automatic adjustments for efficient and precise positioning and release of virtual objects. In: Proceedings of the 2006 ACM international conference on Virtual reality continuum and its applications, Hong Kong, China, 2006. ACM, 1128943, pp 121-128. https://doi.org/10.1145/1128923.1128943
Osawa N, Asai K (2010) Adjustment and control methods for precise rotation and positioning of virtual object by hand. In: Proceedings of the 9th ACM SIGGRAPH Conference on Virtual-Reality Continuum and its Applications in Industry, Seoul, South Korea, 2010. ACM, 1900205, pp 131-138. https://doi.org/10.1145/1900179.1900205
Petzold B, Zaeh MF, Faerber B, Deml B, Egermeier H, Schilp J, Clarke S (2004) A Study on Visual, Auditory, and Haptic Feedback for Assembly Tasks. Presence: Teleoperators and Virtual Environments 13(1):16–21. https://doi.org/10.1162/105474604774048207
Pierce JS, Forsberg AS, Conway MJ, Hong S, Zeleznik RC, Mine MR (1997) Image plane interaction techniques in 3D immersive environments. In: Proceedings of the 1997 symposium on Interactive 3D graphics, Providence, Rhode Island, United States, 1997. ACM, 253303, pp 39-ff. https://doi.org/10.1145/253284.253303
Poupyrev I, Billinghurst M, Weghorst S, Ichikawa T (1996) The go-go interaction technique: non-linear mapping for direct manipulation in VR. In: Proceedings of the 9th annual ACM symposium on User interface software and technology, Seattle, Washington, United States, 1996. ACM, 237102, pp 79-80. https://doi.org/10.1145/237091.237102
Poupyrev I, Weghorst S, Billinghurst M, Ichikawa T (1998) A study of techniques for selecting and positioning objects in immersive VEs: effects of distance, size, and visual feedback. In: Proceedings of ACM CHt'98, 1998. pp -. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.34.1169
Prachyabrued M, Borst CW (2012) Visual interpenetration tradeoffs in whole-hand virtual grasping. In: 3D User Interfaces (3DUI), 2012 I.E. Symposium on, 2012. IEEE, pp 39-42
Prachyabrued M, Borst CW (2012) Virtual grasp release method and evaluation. International Journal of Human-Computer Studies 70(11):828–848. https://doi.org/10.1016/j.ijhcs.2012.06.002
Prachyabrued M, Borst CW (2014) Visual feedback for virtual grasping. In: 3D User Interfaces (3DUI), 2014 I.E. Symposium on, 2014. IEEE, pp 19-26
Smith G, Salzman T, Stuerzlinger W (2001) 3D scene manipulation with 2D devices and constraints. In: Graphics interface 2001, Ottawa, Ontario, Canada, 2001. Canadian Information Processing Society, 781003, pp 135-142. http://dl.acm.org/citation.cfm?id=781003
Stuerzlinger W, Wingrave C (2011) The Value of Constraints for 3D User Interfaces. In: Brunnett G, Coquillart S, Welch G (eds) Virtual Realities. Springer Vienna, pp 203-223. https://doi.org/10.1007/978-3-211-99178-7_11
Teather RJ, Stuerzlinger W (2014) Visual aids in 3D point selection experiments. In: Proceedings of the 2nd ACM symposium on Spatial user interaction, 2014. ACM, pp 127-136
Vanacken L, Grossman T, Coninx K (2009) Multimodal selection techniques for dense and occluded 3D virtual environments. International Journal of Human-Computer Studies 67(3):237–255. https://doi.org/10.1016/j.ijhcs.2008.09.001
Viciana-Abad R, Reyes-Lecuona A, Rosa-Pujazón A, Pérez-Lorenzo JM (2014) The influence of different sensory cues as selection feedback and co-location in presence and task performance. Multimedia tools and applications 68(3):623–639
Wan Hg, Chen Ff, Han Xx (2009) A 4-layer flexible virtual hand model for haptic interaction. In: VECIMS '09. IEEE International Conference on Virtual Environments, Human-Computer Interfaces and Measurements Systems, 2009, 11-13 May 2009 2009. pp 185-190. https://doi.org/10.1109/VECIMS.2009.5068890
Wilkes C, Bowman DA (2008) Advantages of velocity-based scaling for distant 3D manipulation. In: Proceedings of the 2008 ACM symposium on Virtual reality software and technology, Bordeaux, France, 2008. ACM, 1450585, pp 23-29. https://doi.org/10.1145/1450579.1450585
Wu C-H, Chen W-L, Lin CH (2015) Depth-based hand gesture recognition. Multimedia Tools and Applications 75(12):7065–7086. https://doi.org/10.1007/s11042-015-2632-3
Wyss HP, Blach R, Bues M (2006) ISith - Intersection-based spatial interaction for two hands. In: IEEE Symposium on 3D User Interfaces, 2006. pp 59-61. https://doi.org/10.1109/VR.2006.93
Acknowledgements
This research work was supported by the AP-2013-011 and the GGPM-2015-023 from Universiti Kebangsaan Malaysia (UKM) Grant Scheme. Other than that, the authors would like to thank the members of MyXLab, the Faculty of Information Science & Technology, Universiti Kebangsaan Malaysia for their support in this research.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Conflict of interests
The author declares that there is no conflict of interests regarding the publication of this paper.
Rights and permissions
About this article
Cite this article
Lam, M.C., Arshad, H., Prabuwono, A.S. et al. Interaction techniques in desktop virtual environment: the study of visual feedback and precise manipulation method. Multimed Tools Appl 77, 16367–16398 (2018). https://doi.org/10.1007/s11042-017-5205-9
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11042-017-5205-9