Abstract
In the real world, where there is a large amount of information, humans recognize an object efficiently by moving their sensors, and if it is supported by context information, a better result could be produced. In this paper, the emergence of sensor motion and a context-based recognition function are expected. The sensor-equipped recognition learning system has a very simple and general architecture that is consisted of one recurrent neural network and trained by reinforcement learning. The proposed learning system learns to move a visual sensor intentionally and to classify a word from the series of partial information simultaneously only based on the reward and punishment generated from the recognition result. After learning, it was verified that the context-based word recognition could be achieved. All words were correctly recognized at the appropriate timing by actively moving the sensors not depending on the initial sensor location.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Whitehead, S.D., Ballard, D.H.: Active Perception and Reinforcement Learning. Neural Computation 2(4), 409–419 (1990)
Darrell, T.: Reinforcement Learning of Active Recognition Behavior. Interval Research Technical Report, 1997-045 (1998)
Paletta, L., Pinz, A.: Active Object Recognition by View Integration and Reinforcement Learning. Robotics and Automation Systems 31, 71–86 (2000)
Lopez, M.T., et al.: Dynamic Visual Attention Model in Images Sequences. Image and Vision Computing 25, 597–613 (2007)
Larochelle, H., Hinton, G.: Learning to combine foveal glimpses with a third-order Boltzmann machine. In: Advances in Neural Information Processing, vol. 23, pp. 1243–1251 (2010)
Denil, M., Bazzani, L., de Freitas, N.: Learning Where to Attend with Deep Architecture for Image Tracking. Neural Computation 24(8), 2151–2184 (2012)
Shibata, K.: Emergence of Intelligence through Reinforcement Learning with a Neural Network. In: Mellouk, A. (ed.) Advances in Reinforcement Learning, pp. 99–120. InTech (2011)
Shibata, K., Nishino, T., Okabe, Y.: Active Perception and Recognition Learning System Based on Actor-Q Architecture. System and Computers in Japan 33, 12–22 (2002)
Faudzi, A.A.M., Shibata, K.: Acquisition of active perception and recognition through Actor-Q learning using a movable camera. In: Proc. of SICE Annual Conf., FB03-2.pdf (2010)
Utsunomiya, H., Shibata, K.: Contextual Behaviors and Internal Representations Acquired by Reinforcement Learning with a Recurrent Neural Network in a Continuous State and Action Space Task. In: Köppen, M., Kasabov, N., Coghill, G. (eds.) ICONIP 2008, Part II. LNCS, vol. 5507, pp. 970–978. Springer, Heidelberg (2009)
Shibata, K., Utsunomiya, H.: Discovery of Pattern Meaning from Delayed Rewards by Reinforcement Learning with a Recurrent Neural Network. In: Proc. of IJCNN (Int’l Joint Conf. on Neural Networks), pp. 1445–1452 (2011)
Goto, K., Shibata, K.: Emergence of prediction by reinforcement learning using a recurrent neural network. Journal of Robotics, Article ID 437654 (2010)
Faudzi, A.A.M., Shibata, K.: Context-based Word Recognition through a Coupling of Q-Learning and Recurrent Neural Network. In: Proc. of SICE Annual Conf. of the Kyushu Branch, pp. 155–158 (2011)
Rumelhart, D.E., Hinton, G.E., Williams, R.J.: Learning Internal Representations by Error Propagation. In: Parallel Distributed Processing, pp. 318–362. The MIT Press (1986)
Watkins, C.J.C.H., Dayan, P.: Q-learning. Machine Learning 8, 279–292 (1992)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2014 Springer International Publishing Switzerland
About this chapter
Cite this chapter
Faudzi, A.A.M., Shibata, K. (2014). Acquisition of Context-Based Active Word Recognition by Q-Learning Using a Recurrent Neural Network. In: Kim, JH., Matson, E., Myung, H., Xu, P., Karray, F. (eds) Robot Intelligence Technology and Applications 2. Advances in Intelligent Systems and Computing, vol 274. Springer, Cham. https://doi.org/10.1007/978-3-319-05582-4_17
Download citation
DOI: https://doi.org/10.1007/978-3-319-05582-4_17
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-05581-7
Online ISBN: 978-3-319-05582-4
eBook Packages: EngineeringEngineering (R0)