Abstract
When ordering a product the options to personalize or customize the items have increased over the last years. This flexibility has lead to an increasing number of variants in manufactured products. As storing produced items is expensive, companies tend to produce their products lean, i.e. in smaller lot sizes just when they are needed. This lean manufacturing creates more flexible production environments. In our work, we investigate how human workers can be assisted to work in such demanding environments. Therefore, Augmented Reality systems can be used to provide work instructions. First, in this chapter we provide a comprehensive overview about Augmented Reality approaches to support workers directly at the workplace and introduce an assistive system for providing in-situ instructions. Through three user studies, we evaluate the general impact of in-situ instructions, evaluate three instruction creation strategies, and finally evaluate the created instructions using a real product assembly task.
The majority of the work has been conducted while he was a researcher at the University of Stuttgart.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
http://memex-academy.eu/ (last access 03-18-2016).
- 2.
http://www.ops-solutions.com/ (last access 03-18-2016).
- 3.
http://www.emgu.com/ (last access 03-18-2016).
- 4.
http://www.lego.com/en-us/duplo (last access 03-18-2016).
References
Aleotti J, Caselli S (2006) Robust trajectory learning and approximation for robot programming by demonstration. Robot Auton Syst 54(5):409–413
Antifakos S, Michahelles F, Schiele B (2002) Proactive instructions for furniture assembly. In: UbiComp 2002: ubiquitous computing. Springer, pp 351–360
Bannat A, Gast J, Rigoll G, Wallhoff F (2008) Event analysis and interpretation of human activity for augmented reality-based assistant systems. In: 4th international conference on intelligent computer communication and processing, 2008. ICCP 2008. IEEE, pp 1–8
Barna J, NovakovaMarcincinova L, Novak-Marcincin J, Fecova V, Janak M, Torok J (2012) Open source tools in assembling process enriched with elements of augmented reality. In: Proceedings of the 2012 virtual reality international conference. ACM, p 2
Bay H, Tuytelaars T, Van Gool L (2006) Surf: speeded up robust features. In: Computer vision-ECCV 2006. Springer, pp 404–417
Billard A, Calinon S, Dillmann R, Schaal S (2008) Robot programming by demonstration. In: Springer handbook of robotics. Springer, pp 1371–1394
Blanke U, Schiele B, Kreil M, Lukowicz P, Sick B, Gruber T (2010) All for one or one for all? Combining heterogeneous features for activity spotting. In: 2010 8th IEEE international conference on Pervasive computing and communications workshops (PERCOM Workshops). IEEE, pp 18–24
Büttner S, Sand O, Röcker C (2015) Extending the design space in industrial manufacturing through mobile projection. In: Proceedings of the 17th international conference on human-computer interaction with mobile devices and services adjunct. ACM, pp 1130–1133
Büttner S, Mucha H, Funk M, Kosch T, Aehnelt M, Robert S, Röcker C (2017) The design space of augmented and virtual reality applications for assistive environments in manufacturing: a visual approach. In: Proceedings of the 10th international conference on pervasive technologies related to assistive environments. ACM, pp 433–440
Büttner S, Funk M, Sand P, Röcker C (2016) Using head-mounted displays and in-situ projection for assistive systems: a comparison. In: Proceedings of the 9th ACM international conference on pervasive technologies related to assistive environments. ACM, p 44
Caudell TP, Mizell DW (1992) Augmented reality: an application of heads-up display technology to manual manufacturing processes. In: Proceedings of the twenty-fifth hawaii international conference on system sciences, vol 2. IEEE, pp 659–669
Collett T, MacDonald BA (2006) Developer oriented visualisation of a robot program. In: Proceedings of the 1st ACM SIGCHI/SIGART conference on human-robot interaction. ACM, pp 49–56
Fiorentino M, de Amicis R, Monno G, Stork A (2002) Spacedesign: a mixed reality workspace for aesthetic industrial design. In: Proceedings of the 1st international symposium on mixed and augmented reality. IEEE Computer Society, p 86
Funk M, Korn O, Schmidt A (2015) Enabling end users to program for smart environments. In: Proceedings of the CHI 2015—workshop on end user development in the internet of things era 12.2, pp 9–14
Funk M, Kosch T, Schmidt A (2016) Interactive worker assistance: comparing the effects of in-situ projection, head-mounted displays, tablet, and paper instructions. In: Proceedings of the 2016 ACM international joint conference on pervasive and ubiquitous computing. ACM, pp 934–939
Funk M, Mayer S, Schmidt A (2015) Using in-situ projection to support cognitively impaired workers at the workplace. In: Proceedings of the 17th international ACM SIGACCESS conference on computers and accessibility
Funk M, Schmidt A (2015) Cognitive assistance in the workplace. Pervasive Comput IEEE 14(3):53–55
Funk M, Kosch T, Greenwald SW, Schmidt A (2015) A benchmark for interactive augmented reality instructions for assembly tasks. In: Proceedings of the 14th international conference on mobile and ubiquitous multimedia. ACM, pp 253–257
Funk M, Bächler A, Bächler L, Korn O, Krieger C, Heidenreich T, Schmidt A (2015) Comparing projected in-situ feedback at the manual assembly workplace with impaired workers. In: Proceedings of the 8th ACM international conference on pervasive technologies related to assistive environments. ACM, p 1
Funk M, Kosch T, Kettner R, Korn O, Schmidt A (2016) Motioneap: an overview of 4 years of combining industrial assembly with augmented reality for industry 4.0. In: Proceedings of the 16th international conference on knowledge technologies and datadriven business
Funk M, Shirazi AS, Mayer S, Lischke L, Schmidt A (2015) Pick from here!: an interactive mobile cart using in-situ projection for order picking. In: Proceedings of the 2015 ACM international joint conference on pervasive and ubiquitous computing. ACM, pp 601–609
Guo A, Raghu S, Xie X, Ismail S, Luo X, Simoneau J, Gilliland S, Baumann H, Southern C, Starner T (2014) A comparison of order picking assisted by head-up display (HUD), cart-mounted display (CMD), light, and paper pick list. In: Proceedings of the 2014 ACM international symposium on wearable computers. ACM, pp 71–78
Hahn J, Ludwig B, Wolff C (2015) Augmented reality-based training of the PCB assembly process. In: Proceedings of the 14th international conference on mobile and ubiquitous multimedia. ACM, pp 395–399
Hardy J, Alexander J (2012) Toolkit support for interactive projected displays. In: Proceedings of the 11th international conference on mobile and ubiquitous multimedia. ACM, p 42
Hart SG, Staveland LE (1988) Development of NASA-TLX (task load index): results of empirical and theoretical research. Adv Psychol 52:139–183
Henderson S, Feiner S (2011) Exploring the benefits of augmented reality documentation for maintenance and repair. IEEE Trans Visual Comput Graph 17(10):1355–1368
Hermann M, Pentek T, Otto B. Design principles for industrie 4.0 scenarios: a literature review
Klompmaker F, Nebe K, Fast A (2012) dSensingNI: a framework for advanced tangible interaction using a depth camera. In: Proceedings of the sixth international conference on tangible, embedded and embodied interaction. ACM, pp 217–224
Korn O, Schmidt A, Hörz T (2013) Augmented manufacturing: a study with impaired persons on assistive systems using in-situ projection. In: Proceedings of the 6th international conference on pervasive technologies related to assistive environments. ACM, p 21
Korn O, Funk M, Abele S, Hörz T, Schmidt A (2014) Context-aware assistive systems at the workplace: analyzing the effects of projection and gamification. In: Proceedings of the 7th international conference on pervasive technologies related to assistive environments. ACM, p 8
Kubitza T, Schmidt A (2015) Towards a toolkit for the rapid creation of smart environments. In: End-user development. Springer, pp 230–235
Lee C-H, Bonnani L, Selker T (2005) Augmented reality kitchen: enhancing human sensibility in domestic life. In: ACM SIGGRAPH 2005 posters. ACM, p 60
Lieberman H (2001) Your wish is my command: programming by example. Morgan Kaufmann
Linder N, Maes P (2010) LuminAR: portable robotic augmented reality interface design and prototype. In: Adjunct proceedings of the 23nd annual ACM symposium on user interface software and technology. ACM, pp 395–396
Lucke D, Constantinescu C, Westkämper E (2008) Smart factory-a step towards the next generation of manufacturing. In: Manufacturing systems and technologies for the new frontier. Springer, pp 115–118
Marinos D, Wöldecke B, Geiger C (2013) Prototyping natural interactions in virtual studio environments by demonstration: combining spatial mapping with gesture following. In: Proceedings of the virtual reality international conference: laval virtual. ACM, p 2
Myers BA (1986) Creating dynamic interaction techniques by demonstration. In: ACM SIGCHI bulletin 17.SI, pp 271–278 (1986)
Pinhanez C (2001) The everywhere displays projector: a device to create ubiquitous graphical interfaces. In: Ubicomp 2001: ubiquitous computing. Springer, pp 315–331
Rüther S, Hermann T, Mracek M, Kopp S, Steil J (2013) An assistance system for guiding workers in central sterilization supply departments. In: Proceedings of the 6th international conference on pervasive technologies related to assistive environments. ACM, p 3
Salonen T, Sääski J, Hakkarainen M, Kannetis T, Perakakis M, Siltanen S, Potamianos A, Korkalo O, Woodward C (2007) Demonstration of assembly work using augmented reality. In: Proceedings of the 6th ACM international conference on image and video retrieval. ACM, pp 120–123
Schmidt A (2000) Implicit human computer interaction through context. Pers Technol 4(2–3):191–199
Tang A, Owen C, Biocca F, Mou W (2003) Comparative effectiveness of augmented reality in object assembly. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, pp 73–80
Ward JA, Lukowicz P, Troster G, Starner TE (2006) Activity recognition of assembly tasks using body-worn microphones and accelerometers. IEEE Trans Pattern Anal Mach Intell 28(10):1553–1567
Wilson AD (2004) TouchLight: an imaging touch screen and display for gesture-based interaction. In: Proceedings of the 6th international conference on Multimodal interfaces. ACM, pp 69–76
Wilson AD (2010) Using a depth camera as a touch sensor. In: ACM international conference on interactive tabletops and surfaces. ACM, pp 69–72
Wohlgemuth W, Triebfürst G (2000) ARVIKA: augmented reality for development, production and service. In: Proceedings of DARE 2000 on designing augmented reality environments. ACM, pp 151–152
Zauner J, Haller M, Brandl A, Hartman W (2003) Authoring of a mixed reality assembly instructor for hierarchical structures. In: The second IEEE and ACM international symposium on mixed and augmented reality, 2003. Proceedings. IEEE, pp 237–246
Zollner R, Rogalla O, Dillmann R, Zollner M (2002) Understanding users intention: programming fine manipulation tasks by demonstration. In: IEEE/RSJ international conference on intelligent robots and systems, 2002, vol 2. IEEE, pp 1114–1119
Acknowledgements
This work is funded by the German Federal Ministry for Economic Affairs and Energy in the project motionEAPÂ [20], grant no. 01MT12021E. We thank Mathias Hoppe for his work in helping to implement the software and conducting the user study. We further thank Klaus Klein, Michael Spreng, and Johann Hegel from Audi AG.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Singapore Pte Ltd.
About this chapter
Cite this chapter
Funk, M., Lischke, L., Mayer, S., Shirazi, A.S., Schmidt, A. (2018). Teach Me How! Interactive Assembly Instructions Using Demonstration and In-Situ Projection. In: Huber, J., Shilkrot, R., Maes, P., Nanayakkara, S. (eds) Assistive Augmentation. Cognitive Science and Technology. Springer, Singapore. https://doi.org/10.1007/978-981-10-6404-3_4
Download citation
DOI: https://doi.org/10.1007/978-981-10-6404-3_4
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-6402-9
Online ISBN: 978-981-10-6404-3
eBook Packages: EngineeringEngineering (R0)