Abstract
In this work, we develop a multi-modal rendering framework comprising of hapto-visual and auditory data. The prime focus is to haptically render point cloud data representing virtual 3-D models of cultural significance and also to handle their affine transformations. Cultural heritage objects could potentially be very large and one may be required to render the object at various scales of details. Further, surface effects such as texture and friction are incorporated in order to provide a realistic haptic perception to the users. Moreover, the proposed framework includes an appropriate sound synthesis to bring out the acoustic properties of the object. It also includes a graphical user interface with varied options such as choosing the desired orientation of 3-D objects and selecting the desired level of spatial resolution adaptively at runtime. A fast, point proxy-based haptic rendering technique is proposed with proxy update loop running \(100\) times faster than the required haptic update frequency of \(1\) kHz. The surface properties are integrated in the system by applying a bilateral filter on the depth data of the virtual 3-D models. Position dependent sound synthesis is incorporated with the incorporation of appropriate audio clips.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Adí, W., Sulaiman, S.: Haptic texture rendering based on visual texture information: a study to achieve realistic haptic texture rendering. In: Zaman, H.B., Robinson, P., Maria, M.P., Olivier, P., Schröder, H., Shih, T.K. (eds.) IVIC 2009. LNCS, vol. 5857, pp. 279–287. Springer, Heidelberg (2009)
Choi, S., Tan, H.Z.: An analysis of perceptual instability during haptic texture rendering. In: Proceedings of the 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Orlando, Florida, U.S.A., pp. 129–136 (2002)
Gaver, W.W.: What in the world do we hear? an ecological approach to auditory event perception. Ecol. Psychol. J. 5(1), 1–29 (1993)
Gluckman, J.: Scale variant image pyramids. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, New York, U.S.A., pp. 1069–1075 (2006)
Harwin, W.S., Melder, N.: Improved haptic rendering for multi-finger manipulation using friction cone. In: Proceedings of the 2nd Eurohaptics Conference, Edinburgh, UK (2002)
Hayward, V., Armstrong, B.: A new computational model of friction applied to haptic rendering. In: Corke, P., Trevelyan, J. (eds.) Experimental Robotics VI. LNICST, vol. 250, pp. 403–412. Springer, Heidelberg (2000)
Laycock, S.D., Day, A.M.: A survey of haptic rendering techniques. Comput. Graph. Forum 26, 50–65 (2007). Blackwell Publishing
Lederman, S.J., Morgan, T., Hamilton, C., Klatzky, R.L.: Integrating multimodal information about surface texture via a probe: relative contributions of haptic and touch-produced sound sources. In: Proceedings of the 10th Symposium on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Washington, DC, U.S.A., pp. 97–105 (2002)
Lee, J.K., Kim, Y.J.: Haptic rendering of point set surfaces. In: World Haptics Conference, Tsukuba, Japan, pp. 513–518 (2007)
Leeper, A., Chan, S., Salisbury, K.: Constraint based 3-DoF haptic rendering of arbitrary point cloud data. In: RSS Workshop on RGB-D Cameras, University of Southern California, Los Angeles, U.S.A., June 2011
Melder, N., Harwin, W.S.: Extending the friction cone algorithm for arbitrary polygon based haptic objects. In: Proceedings of 12th International Conference on Haptic Interfaces for Virtual Environment and Teleoperator Systems, Washington, DC, U.S.A., pp. 234–241 (2004)
El-Far, N.R., Georganas, N.D., Saddik, A.E.: An algorithm for haptically rendering objects described by point clouds. In: Proceedings of the 21st Canadian Conference on Electrical and Computer Engineering, Ontario, Canada (2008)
Patil, H.A., Gajbhar, S.S.: Acoustical analysis of musical pillar of great stage of Vitthala temple at Hampi, India. In: International Conference on Signal Processing and Communications (SPCOM), pp. 1–5 (2012)
Richard, C., Cutkosky, M.R.: Friction modeling, display in haptic applications involving user performance. In: IEEE International Conference on Robotics and Automation, pp. 605–611 (2002)
Romano, J.M., Kuchenbecker, K.J.: Creating realistic virtual textures from contact acceleration data. IEEE Trans. Haptics 5, 109–119 (2012)
Ruspini, D.C., Kolarov, K., Khatib, O.: The haptic display of complex graphical environments. In: Proceedings of ACM SIGGRAPH, Los Angeles, U.S.A., pp. 345–352 (1997)
Salisbury, K., Conti, F., Barbagli, F.: Haptic rendering: introductory concepts. IEEE Comput. Graph. Appl. Mag., special issue on Haptic Rendering 24, 24–32 (2004)
Sreeni, K.G., Priyadarshini, K., Praseedha, A.K., Chaudhuri, S.: Haptic rendering of cultural heritage objects at different scales. In: Proceedings of the Eurohaptics, Finland, pp. 505–516 (2012)
Sreeni, K.G., Chaudhuri, S.: Haptic rendering of dense 3-D point cloud data. In: IEEE Haptics Symposium, Vancouver, BC, Canada, 4–7 March 2012
Tomasi, C., Manduchi, R.: Bilateral filtering for gray and color images. In: Proceedings of the Sixth International Conference on Computer Vision, Mumbai, India, pp. 839–846 (1998)
Wall, S.: An investigation of temporal and spatial limitation of haptic interfaces. Ph.D. Thesis, Department of Cybernetics, University of Reading (2004)
Weisstein, E.: Mean curvature, June 2013. http://mathworld.wolfram.com/MeanCurvature.html
Zilles, C.B., Salisbury, J.K.: A constraint-based god-object method for haptic display. In: IEEE/RSJ International Conference on Intelligent Robots and Systems, Pittsburgh, U.S.A., vol. 3, pp. 3146–3151 (1995)
Acknowledgement
The authors would like to thank DST for the grant provided on the Indian Digital Heritage Project and MCIT for the grant on perception engineering. The authors would also like to thank Prof. Manjunath Joshi and his team for providing us with the audio signals of musical pillars at Hampi which is the input to our proposed rendering framework.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2015 Springer International Publishing Switzerland
About this paper
Cite this paper
Aniyath, P.K., Gopalan, S.K., Kumari, P., Chaudhuri, S. (2015). Combined Hapto-visual and Auditory Rendering of Cultural Heritage Objects. In: Jawahar, C., Shan, S. (eds) Computer Vision - ACCV 2014 Workshops. ACCV 2014. Lecture Notes in Computer Science(), vol 9009. Springer, Cham. https://doi.org/10.1007/978-3-319-16631-5_36
Download citation
DOI: https://doi.org/10.1007/978-3-319-16631-5_36
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-16630-8
Online ISBN: 978-3-319-16631-5
eBook Packages: Computer ScienceComputer Science (R0)