Skip to main content
SpringerLink
Log in

Exploring the Incorporation of Opinion Polarity for Abstractive Multi-document Summarisation

  • Conference paper
  • First Online:
Advances in Information Retrieval (ECIR 2021)

Part of the book series: Lecture Notes in Computer Science ((LNISA,volume 12657))

Included in the following conference series:

Abstract

Abstractive multi-document summarisation (MDS) remains a challenging task. Part of the problem is the question as to how to preserve a document’s polarity in the summary. We propose an opinion polarity attention model for MDS, which incorporates a polarity estimator based on a BERT-GRU sentiment analysis network. It captures the impact of opinions expressed in the source documents and integrates it in the attention mechanism. Experimental results using a state-of-the-art MDS approach and a common benchmark test collection demonstrate that this model has a measurable positive effect using a range of metrics.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 139.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 179.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    All code, model files and outputs are available at https://github.com/dramsauer/Summarizing-Opinions.

  2. 2.

    A conceivable way to penalise opinionated content might be to choose \(1-|tanh(v{_i})|\).

  3. 3.

    https://www.newser.com/.

  4. 4.

    https://github.com/OpenNMT/OpenNMT-py/blob/master/docs/source/examples/Summarization.md.

  5. 5.

    The basic PG-BRRN approach was also reproduced with similarly lower results, which raises the question of reproducibility in general. There have also been attempts to reproduce MGSum and GraphSum, but without success.

References

  1. Amplayo, R.K., Lapata, M.: Unsupervised opinion summarization with noising and denoising. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1934–1945. Association for Computational Linguistics, Online, July 2020 (2020). https://doi.org/10.18653/v1/2020.acl-main.175

  2. Bahdanau, D., Cho, K.H., Bengio, Y.: Neural machine translation by jointly learning to align and translate. In: 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, pp. 1–15. International Conference on Learning Representations, ICLR, San Diego, CA, USA (2015). https://arxiv.org/abs/1409.0473v7

  3. Barnes, J., Øvrelid, L., Velldal, E.: Sentiment analysis is not solved! Assessing and probing sentiment classification. In: Proceedings of the Second BlackboxNLP Workshop on Analyzing and Interpreting Neural Networks for NLP, pp. 12–23. Association for Computational Linguistics, Florence, Italy (2019). https://doi.org/10.18653/v1/W19-4802

  4. Carbonell, J., Goldstein, J.: Use of MMR, diversity-based reranking for reordering documents and producing summaries. In: Proceedings of the 21st Annual International ACM SIGIR Conference on Research and Development in Information Retrieval (SIGIR 1998), pp. 335–336. Association for Computing Machinery, Melbourne, Australia (1998). https://doi.org/10.1145/290941.291025

  5. Carenini, G., Cheung, J.C.K., Pauls, A.: Multi-document summarization of evaluative text. Comput. Intell. 29(4), 545–576 (2013). https://doi.org/10.1111/j.1467-8640.2012.00417.x

  6. Cho, K., et al.: Learning phrase representations using RNN encoder-decoder for statistical machine translation. In: EMNLP 2014–2014 Conference on Empirical Methods in Natural Language Processing, Proceedings of the Conference, pp. 1724–1734. Association for Computational Linguistics (ACL), Doha, Qatar (2014). https://doi.org/10.3115/v1/d14-1179

  7. Choubey, P.K., Lee, A., Huang, R., Wang, L.: Discourse as a function of event: profiling discourse structure in news articles around the main event. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5374–5386. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.acl-main.478

  8. Deng, L., Wiebe, J.: MPQA 3.0: an entity/event-level sentiment corpus. In: Proceedings of the 2015 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, pp. 1323–1328. Association for Computational Linguistics, Denver, Colorado (2015). https://doi.org/10.3115/v1/N15-1146

  9. Devlin, J., Chang, M.W., Lee, K., Google, K.T., Language, A.I.: BERT: pre-training of deep bidirectional transformers for language understanding. In: Proceedings of NAACL-HLT, pp. 4171–4186. Association for Computational Linguistics, Minneapolis, Minnesota (2019). https://doi.org/10.18653/v1/N19-1423, https://github.com/google-research/bertwww.aclweb.org/anthology/N19-1423

  10. Duchi, J., Hazan, E., Singer, Y.: Adaptive subgradient methods for online learning and stochastic optimization * Elad Hazan. J. Mach. Learn. Res. 12(61), 2121–2159 (2011). www.jmlr.org/papers/volume12/duchi11a/duchi11a.pdf?source=post_page

  11. Erkan, G., Radev, D.R.: LexRank: graph-based lexical centrality as salience in text summarization. J. Artif. Intell. Res. 22, 457–479 (2011). https://doi.org/10.1613/jair.1523, http://arxiv.org/abs/1109.2128

  12. Fabbri, A.R., Li, I., She, T., Li, S., Radev, D.R.: Multi-News: a large-scale multi-document summarization dataset and abstractive hierarchical model. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, pp. 1074–1084. Association for Computational Linguistics, Florence, Italy (2019). https://doi.org/10.18653/v1/P19-1102, https://www.aclweb.org/anthology/P19-1102.pdf

  13. Gehrmann, S., Deng, Y., Rush, A.M.: Bottom-up abstractive summarization. In: Proceedings of the 2018 Conference on Empirical Methods in Natural Language Processing, EMNLP 2018, pp. 4098–4109. Association for Computational Linguistics, Brussels, Belgium (2018). http://arxiv.org/abs/1808.10792

  14. Huang, L., Wu, L., Wang, L.: Knowledge graph-augmented abstractive summarization with semantic-driven cloze reward. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5094–5107. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.acl-main.457

  15. Jin, H., Wang, T., Wan, X.: Multi-granularity interaction network for extractive and abstractive multi-document summarization. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6244–6254. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.acl-main.556

  16. Kingma, D.P., Ba, J.L.: Adam: a method for stochastic optimization. In: 3rd International Conference on Learning Representations, ICLR 2015 - Conference Track Proceedings, pp. 1–15. International Conference on Learning Representations, ICLR, San Diego, CA, USA (2015). https://arxiv.org/abs/1412.6980v9

  17. Li, W., Xiao, X., Liu, J., Wu, H., Wang, H., Du, J.: Leveraging graph to improve abstractive multi-document summarization. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 6232–6243. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.acl-main.555

  18. Lin, C.Y.: ROUGE: a package for automatic evaluation of summaries. In: Text Summarization Branches Out, pp. 74–81. Association for Computational Linguistics, Barcelona, Spain (2004). https://www.aclweb.org/anthology/W04-1013

  19. Liu, B.: Sentiment Analysis and Opinion Mining. Synthesis Lectures Hum. Lang. Technol. 5(1), 1–167 (2012). http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.244.9480&rep=rep1&type=pdf

  20. Liu, Y., Lapata, M.: Hierarchical transformers for multi-document summarization. In: Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, vol. 5070, pp. 5070–5081. Association for Computational Linguistics (ACL), Florence, Italy (2019). https://doi.org/10.18653/v1/p19-1500, https://github.com/nlpyang/hiersumm

  21. Mihalcea, R., Tarau, P.: TextRank: bringing order into texts. In: Proceedings of the 2004 Conference on Empirical Methods in Natural Language Processing, pp. 404–411. Association for Computational Linguistics, Barcelona, Spain (2004). https://www.aclweb.org/anthology/W04-3252

  22. See, A., Liu Google Brain, P.J., Manning, C.D.: Get to the point: summarization with pointer-generator networks. In: Proceedings of the 55th Annual Meeting of the Association for Computational Linguistic, pp. 1073–1083. Association for Computational Linguistics, Vancouver, Canada (2017). https://doi.org/10.18653/v1/P17-1099

  23. Su, Y., Kuo, C.C.J.: On extended long short-term memory and dependent bidirectional recurrent neural network. Neurocomputing 356, 151–161 (2018). https://doi.org/10.1016/j.neucom.2019.04.044, http://arxiv.org/abs/1803.01686 dx.doi.org/10.1016/j.neucom.2019.04.044

  24. Suhara, Y., Wang, X., Angelidis, S., Tan, W.C.: OpinionDigest: a simple framework for opinion summarization. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 5789–5798. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.acl-main.513, https://www.aclweb.org/anthology/2020.acl-main.513

  25. Tan, X., Cai, Y., Zhu, C.: Recognizing conflict opinions in aspect-level sentiment classification with dual attention networks. In: Proceedings of the 2019 Conference on Empirical Methods in Natural Language Processing and the 9th International Joint Conference on Natural Language Processing, pp. 3426–3431. Association for Computational Linguistics, Hong Kong, China (2019). https://doi.org/10.18653/v1/D19-1342

  26. Vaswani, A., et al.: Attention is all you need. In: Advances in Neural Information Processing Systems, pp. 5998–6008. Curran Associates Inc, Long Beach, California (2017). http://papers.nips.cc/paper/7181-attention-is-all-you-need.pdf

  27. Xu, S., Li, H., Yuan, P., Wu, Y., He, X., Zhou, B.: Self-attention guided copy mechanism for abstractive summarization. In: Proceedings of the 58th Annual Meeting of the Association for Computational Linguistics, pp. 1355–1362. Association for Computational Linguistics, Online (2020). https://doi.org/10.18653/v1/2020.acl-main.125

Download references

Author information

Authors and Affiliations

  1. University of Regensburg, Regensburg, Germany

    Dominik Ramsauer & Udo Kruschwitz

Authors
  1. Dominik Ramsauer
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Udo Kruschwitz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Dominik Ramsauer .

Editor information

Editors and Affiliations

  1. Radboud University Nijmegen, Nijmegen, The Netherlands

    Djoerd Hiemstra

  2. Department of Computer Science, Katholieke Universiteit Leuven, Heverlee, Belgium

    Marie-Francine Moens

  3. Toulouse, Toulouse Institute of Computer Science Research, Toulouse, France

    Josiane Mothe

  4. Istituto di Scienza e Tecnologie dell’Informazione, Consiglio Nazionale delle Ricerche, Pisa, Italy

    Raffaele Perego

  5. Leipzig University, Leipzig, Germany

    Martin Potthast

  6. Istituto di Scienza e Tecnologie dell’Informazione, Consiglio Nazionale delle Ricerche, Pisa, Italy

    Fabrizio Sebastiani

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ramsauer, D., Kruschwitz, U. (2021). Exploring the Incorporation of Opinion Polarity for Abstractive Multi-document Summarisation. In: Hiemstra, D., Moens, MF., Mothe, J., Perego, R., Potthast, M., Sebastiani, F. (eds) Advances in Information Retrieval. ECIR 2021. Lecture Notes in Computer Science(), vol 12657. Springer, Cham. https://doi.org/10.1007/978-3-030-72240-1_35

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-72240-1_35

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-72239-5

  • Online ISBN: 978-3-030-72240-1

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation