Abstract
We propose nonlinearized relevance propagation (NRP), an improved method of exploring deep neural networks (DNN). This method derives from well-known layer-wise relevance propagation (LRP), which employs a linear process to explain a DNN model’s outputs ordinarily. Although the nonlinear functions are widely used by most of the neural network models, to the best of our knowledge, they have not been employed in the LRP for DNN models. In this paper, we apply NRP to the attentive pooling answer selection model and compare the performance of NRP to sensitivity analysis (SA) and LRP of the linear setting. The result shows exploiting nonlinear functions in LRP can help inputs retain more information of importance than SA. The contribution of this work is extending the use of relevance propagation in understanding inner workings of complicated DNN models.
Supported by JST CREST JPMJCR1304, JSPS Kakenhi JP 16H01836, JP 16K1242.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Arras, L., Horn, F., Montavon, G., Müller, K.R., Samek, W.: What is relevant in a text document?: an interpretable machine learning approach. PLoS ONE 12(8), e0181142 (2017). https://doi.org/10.1371/journal.pone.0181142
Arras, L., Montavon, G., Müller, K.R., Samek, W.: Explaining recurrent neural network predictions in sentiment analysis, pp. 159–168 (2017). http://arxiv.org/abs/1706.07206
Bach, S., Binder, A., Montavon, G., Klauschen, F., Müller, K.R., Samek, W.: On pixel-wise explanations for non-linear classifier decisions by layer-wise relevance propagation. PLoS ONE 10(7), 1–46 (2015). https://doi.org/10.1371/journal.pone.0130140
Bachrach, Y., et al.: An attention mechanism for answer selection using a combined global and local view, pp. 1–8, July 2017. http://arxiv.org/abs/1707.01378
Ding, Y., Liu, Y., Luan, H., Sun, M.: Visualizing and understanding neural machine translation. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1150–1159 (2017). https://doi.org/10.18653/v1/P17-1106, http://aclweb.org/anthology/P17-1106
Feng, M., Xiang, B., Glass, M.R., Wang, L., Zhou, B.: Applying deep learning to answer selection: a study and an open task. In: ASRU 2015 - Proceedings 2015 IEEE Workshop on Automatic Speech Recognition and Understanding, pp. 813–820 (2016). https://doi.org/10.1109/ASRU.2015.7404872
Horn, F., Arras, L., Montavon, G., Müller, K.R., Samek, W.: Discovering topics in text datasets by visualizing relevant words, pp. 1–10 (2017). http://arxiv.org/abs/1707.06100
Kanter, J.M., Veeramachaneni, K.: Deep feature synthesis: towards automating data science endeavors. In: Proceedings of the 2015 IEEE International Conference on Data Science and Advanced Analytics, DSAA 2015 (2015). https://doi.org/10.1109/DSAA.2015.7344858, http://arxiv.org/abs/1708.08296
Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. In: 2017 IEEE International Conference on Consumer Electronics ICCE 2017, pp. 434–435, December 2014. https://doi.org/10.1109/ICCE.2017.7889386, http://arxiv.org/abs/1412.6980
Montavon, G., Samek, W., Müller, K.R.: Methods for interpreting and understanding deep neural networks. Digit. Sig. Process. A Rev. J. 73, 1–15 (2018). https://doi.org/10.1016/j.dsp.2017.10.011
Pennington, J., Socher, R., Manning, C.: Glove: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014). https://doi.org/10.3115/v1/D14-1162, http://aclweb.org/anthology/D14-1162
Rücklé, A., Gurevych, I.: End-to-end non-factoid question answering with an interactive visualization of neural attention weights. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics-System Demonstrations (ACL), pp. 19–24 (2017). https://doi.org/10.18653/v1/P17-4004, https://www.informatik.tu-darmstadt.de/fileadmin/user_upload/Group_UKP/publikationen/2017/2017_ACL_AR_End2End_QA.pdf
Rücklé, A., Gurevych, I.: Representation learning for answer selection with LSTM-based importance weighting. In: Proceedings of the 12th International Conference on Computational Semantics (IWCS 2017) (2017). (to appear). http://www.aclweb.org/anthology/W17-6935
dos Santos, C., Tan, M., Xiang, B., Zhou, B.: Attentive Pooling Networks (CV) (2016). http://arxiv.org/abs/1602.03609
Shrikumar, A., Greenside, P., Kundaje, A.: Learning Important Features Through Propagating Activation Differences (2017). http://arxiv.org/abs/1704.02685
Tan, M., dos Santos, C., Xiang, B., Zhou, B.: LSTM-based deep learning models for non-factoid answer selection, vol. 1, pp. 1–11 (2015). http://arxiv.org/abs/1511.04108
Tan, M., dos Santos, C., Xiang, B., Zhou, B.: Improved representation learning for question answer matching. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 464–473 (2016). https://doi.org/10.18653/v1/P16-1044, http://aclweb.org/anthology/P16-1044
Vinyals, O., Kaiser, L., Koo, T., Petrov, S., Sutskever, I., Hinton, G.: Grammar as a foreign language, pp. 1–15 (2014). https://doi.org/10.1146/annurev.neuro.26.041002.131047, http://arxiv.org/abs/1412.7449
Wang, B., Liu, K., Zhao, J.: Inner attention based recurrent neural networks for answer selection. In: Proceedings of the 54th Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers), pp. 1288–1297 (2016). https://doi.org/10.18653/v1/P16-1122, http://aclweb.org/anthology/P16-1122
Wu, W., Wang, H., Li, S.: Bi-directional gated memory networks for answer selection. In: Sun, M., Wang, X., Chang, B., Xiong, D. (eds.) CCL/NLP-NABD -2017. LNCS (LNAI), vol. 10565, pp. 251–262. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-69005-6_21
Zurada, J., Malinowski, A., Cloete, I.: Sensitivity analysis for minimization of input data dimension for feedforward neural network. In: Proceedings of IEEE International Symposium on Circuits and Systems - ISCAS 1994 6, pp. 447–450. https://doi.org/10.1109/ISCAS.1994.409622, http://ieeexplore.ieee.org/document/409622/
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer Nature Switzerland AG
About this paper
Cite this paper
Zhang, Q., Ohsawa, Y. (2018). Nonlinearized Relevance Propagation. In: Geng, X., Kang, BH. (eds) PRICAI 2018: Trends in Artificial Intelligence. PRICAI 2018. Lecture Notes in Computer Science(), vol 11012. Springer, Cham. https://doi.org/10.1007/978-3-319-97304-3_69
Download citation
DOI: https://doi.org/10.1007/978-3-319-97304-3_69
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-97303-6
Online ISBN: 978-3-319-97304-3
eBook Packages: Computer ScienceComputer Science (R0)