Abstract
Deep Transfer Learning or DTS has proven successful with deep neural networks and deep belief networks. However, there has been limited research on to using deep autoencoder (DAE)-based network to implement DTS. This paper for the first time attempts to identify transferable features in the form of learning and transfer them to another network implementing a simple DTS mechanism. In this paper, a transfer of knowledge process is proposed where in knowledge is transferred from one Deep autoencoder network to another. This knowledge transfer has helped to improve the classification accuracy of the receiving autoencoder, particularly when experimented using corrupted dataset. The experiments are carried out on a texa based hierarchical dataset. Firstly, a DAE is trained with regular undamaged dataset to achieve maximum accuracy. Then, a distorted dataset was used to train second DAEN for classification with which only 56.7% of the data is correctly classified. Then a set of weights are transferred from from first DAEN to the second DAEN which resulted in an an improvement of classification accuracy by about 22%. The key contribution of this paper is highlighting importance of knowledge transfer between two deep autoencoder networks which is proposed for the first time.
This is a preview of subscription content, log in via an institution to check access.
References
D. K. Milligan and M. J. D. Wilson, Fundamental Structure/Behaviour Relationships in Synchronous Boolean Neural Networks, 1990, pp. 997–1000.
Z. Waszczyszyn, Fundamentals of artificial neural networks. Springer, 1999, pp. 1–51.
A. V. Terekhov, G. Montone, and J. K. OâĂŹRegan, Knowledge Transfer in Deep Block-Modular Neural Networks, 2015, pp. 268–279.
Y. Bengio, P. Lamblin, D. Popovici, and H. Larochelle, “Greedy layer-wise training of deep networks,” in Advances in Neural Information Processing Systems 19. MIT Press, 2007, pp. 153–160.
Y. Bengio and Y. LeCun, “Scaling learning algorithms towards AI,” in Large Scale Kernel Machines. MIT Press, 2007.
K. He, X. Zhang, S. Ren, and J. Sun, “Spatial pyramid pooling in deep convolutional networks for visual recognition,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. PP, no. 99, pp. 1–1, 2015.
C. Xiong, L. Liu, X. Zhao, S. Yan, and T. Kim, “Convolutional fusion network for face verification in the wild,” Circuits and Systems for Video Technology, IEEE Transactions on, vol. PP, no. 99, pp. 1–1, 2015.
D. Hingu, D. Shah, and S. S. Udmale, “Automatic text summarization of wikipedia articles,” in Communication, Information Computing Technology (ICCICT), 2015 International Conference on, Jan 2015, pp. 1–4.
A. Graves and J. Schmidhuber, “Offline handwriting recognition with multidimensional recurrent neural networks,” pp. 545–552, 2009.
S. S. Tirumala, “Implementation of evolutionary algorithms for deep architectures,” in Proceedings of the 2nd International Workshop on Artificial Intelligence and Cognition (AIC), Torino, Italy, November, 2014, pp. 164–171.
S. S. Tirumala and A. Narayanan, “Hierarchical data classification using deep neural networks,” in Neural Information Processing. Springer International Publishing, 2015, pp. 492–500.
E. Y. Li, “Artificial neural networks and their business applications,” Information & Management, vol. 27, no. 5, pp. 303–313, 1994.
Y. Bengio, A. Courville, and P. Vincent, “Representation learning: A review and new perspectives,” Pattern Analysis and Machine Intelligence, IEEE Transactions on, vol. 35, no. 8, pp. 1798–1828, 2013.
J. Yosinski, J. Clune, Y. Bengio, and H. Lipson, “How transferable are features in deep neural networks?” in Advances in Neural Information Processing Systems, 2014, pp. 3320–3328.
S. Gutstein, O. Fuentes, and E. Freudenthal, “Knowledge transfer in deep convolutional neural nets,” International Journal on Artificial Intelligence Tools, vol. 17, no. 03, pp. 555–567, 2008.
D. C. Cireşan, U. Meier, and J. Schmidhuber, “Transfer learning for latin and chinese characters with deep neural networks,” in Neural Networks (IJCNN), The 2012 International Joint Conference on. IEEE, 2012, pp. 1–6.
C. Kandaswamy, L. M. Silva, L. A. Alexandre, J. M. Santos, and J. M. Sá, Artificial Neural Networks and Machine Learning - ICANN 2014: 24th International Conference on Artificial Neural Networks, Hamburg, Germany, September 15–19, 2014. Proceedings. Cham: Springer International Publishing, 2014, ch. Improving Deep Neural Network Performance by Reusing Features Trained with Transductive Transference, pp. 265–272.
M. Long, Y. Cao, J. Wang, and M. Jordan, “Learning transferable features with deep adaptation networks,” in Proceedings of the 32nd International Conference on Machine Learning (ICML-15), D. Blei and F. Bach, Eds. JMLR Workshop and Conference Proceedings, 2015, pp. 97–105. [Online]. Available: http://jmlr.org/proceedings/papers/v37/long15.pdf.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Singapore Pte Ltd.
About this paper
Cite this paper
Tirumala, S.S. (2018). A Deep Autoencoder-Based Knowledge Transfer Approach. In: Chaki, N., Cortesi, A., Devarakonda, N. (eds) Proceedings of International Conference on Computational Intelligence and Data Engineering. Lecture Notes on Data Engineering and Communications Technologies, vol 9. Springer, Singapore. https://doi.org/10.1007/978-981-10-6319-0_23
Download citation
DOI: https://doi.org/10.1007/978-981-10-6319-0_23
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-10-6318-3
Online ISBN: 978-981-10-6319-0
eBook Packages: EngineeringEngineering (R0)