Abstract
The existing residual attention network (RAN) method mainly utilizes the deeper network layer for the image objects which are to be classified. However, when the network depth is simply increased, it will lead to gradient dispersion (or explosion) effect. To address the problem, we propose a new improvement method of residual attention network for image classification, which applies several upsampling schemes to the RAN process, i.e., the stacked network structure extraction, and the bottom-up and top-down feedforward attention for residual learning. In the experiments, we have given comparisons of different network structures and different upsampling methods that are applied to the RAN. The proposed improvement method achieves state-of-the-art image classification performance on two benchmark datasets including CIFAR-10 (4.23\(\%\) error) and CIFAR-100 (21.15\(\%\) error). Compared with the traditional RAN method, the proposed improvement method can improve the accuracy of image classification to some extent.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Mnih, V., Heess, N., Graves, A .: Recurrent models of visual attention. In: Advances in Neural Information Processing Systems, pp. 2204–2212 (2014)
Bahdanau, D., Cho, K., Bengio, Y.: Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv:1409.0473 (2014)
Fan, R., Zhou, P., Chen, W.: An online attention-based model for speech recognition. arXiv preprint arXiv:1811.05247 (2018)
Rocktschel, T., Grefenstette, E., Hermann, K.M., et al.: Reasoning about entailment with neural attention. arXiv preprint arXiv:1509.06664 (2015)
Xu, K., Ba, J., Kiros, R., et al.: Show, attend and tell: neural image caption generation with visual attention. In: International Conference on Machine Learning, pp. 2048–2057 (2015)
He, K., Zhang, X., Ren, S., et al.: Deep residual learning for image recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 770–778 (2016)
Wang, F., Jiang, M., Qian, C., et al.: Residual attention network for image classification. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3156–3164 (2017)
Vaswani, A., Shazeer, N., Parmar, N., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008 (2017)
Chen, L., Zhang, H., Xiao, J., et al.: SCA-CNN: spatial and channel-wise attention in convolutional networks for image captioning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5659–5667 (2017)
Parmar, N., Vaswani, A., Uszkoreit, J., et al.: Image transformer. arXiv preprint arXiv:1802.05751 (2018)
Shi, W., Caballero, J., Huszár, F., et al.: Real-time single image and video super-resolution using an efficient sub-pixel convolutional neural network. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1874–1883 (2016)
Anderson, P., He, X., Buehler, C., et al.: Bottom-up and top-down attention for image captioning and visual question answering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 6077–6086 (2018)
Liu, S., Huang, D., Wang, Y., et al.: Receptive field block net for accurate and fast object detection. arXiv preprint arXiv:1711.07767 (2017)
Szegedy, C., Liu, W., Jia, Y., et al.: Going deeper with convolutions. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–9 (2015)
Xie, S., Girshick, R., Dollár, P., et al.: Aggregated residual transformations for deep neural networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1492–1500 (2017)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 Springer Nature Switzerland AG
About this paper
Cite this paper
Liang, L., Cao, J., Li, X., You, J. (2019). Improvement of Residual Attention Network for Image Classification. In: Cui, Z., Pan, J., Zhang, S., Xiao, L., Yang, J. (eds) Intelligence Science and Big Data Engineering. Visual Data Engineering. IScIDE 2019. Lecture Notes in Computer Science(), vol 11935. Springer, Cham. https://doi.org/10.1007/978-3-030-36189-1_44
Download citation
DOI: https://doi.org/10.1007/978-3-030-36189-1_44
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-36188-4
Online ISBN: 978-3-030-36189-1
eBook Packages: Computer ScienceComputer Science (R0)