Skip to main content
SpringerLink
Log in

Improving Results on Russian Sentiment Datasets

  • Conference paper
  • First Online:
Artificial Intelligence and Natural Language (AINL 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1292))

Included in the following conference series:

Abstract

In this study, we test standard neural network architectures (CNN, LSTM, BiLSTM) and recently appeared BERT architectures on previous Russian sentiment evaluation datasets. We compare two variants of Russian BERT and show that for all sentiment tasks in this study the conversational variant of Russian BERT performs better. The best results were achieved by BERT-NLI model, which treats sentiment classification tasks as a natural language inference task. On one of the datasets, this model practically achieves the human level .

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    http://docs.deeppavlov.ai/en/master/features/pretrained_vectors.html.

  2. 2.

    https://rusvectores.org/ru/models/.

  3. 3.

    http://www.cs.cmu.edu/~afm/projects/multilingual_embeddings.html.

  4. 4.

    https://scikit-learn.org/stable/.

  5. 5.

    http://docs.deeppavlov.ai/en/master/features/models/bert.html.

  6. 6.

    https://github.com/antongolubev5/Targeted-SA-for-Russian-Datasets.

  7. 7.

    https://github.com/LAIR-RCC/Russian-Sentiment-Analysis-Evaluation-Datasets.

References

  1. Amigó, E., et al.: Overview of RepLab 2013: evaluating online reputation monitoring systems. In: Forner, P., Müller, H., Paredes, R., Rosso, P., Stein, B. (eds.) CLEF 2013. LNCS, vol. 8138, pp. 333–352. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40802-1_31

    Chapter  Google Scholar 

  2. Arkhipenko, K., Kozlov, I., Trofimovich, J., Skorniakov, K., Gomzin, A., Turdakov, D.: Comparison of neural network architectures for sentiment analysis of Russian tweets. In: Computational Linguistics and Intellectual Technologies: Proceedings of the International Conference Dialogue, pp. 50–59 (2016)

    Google Scholar 

  3. Burtsev, M.: DeepPavlov: open-source library for dialogue systems. In: Proceedings of ACL 2018, System Demonstrations, pp. 122–127 (2018)

    Google Scholar 

  4. Chetviorkin, I., Loukachevitch, N.: Evaluating sentiment analysis systems in Russian. In: Proceedings of the 4th Biennial International Workshop on Balto-Slavic Natural Language Processing, pp. 12–17 (2013)

    Google Scholar 

  5. Cliche, M.: BB twtr at SemEval-2017 task 4: twitter sentiment analysis with CNNs and LSTMs. In: Proceedings of the 11th International Workshop on Semantic Evaluation (SemEval-2017), pp. 573–580 (2017)

    Google Scholar 

  6. Devlin, J., Chang, M.W., Lee, K., Toutanova, K.: BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding. arXiv preprint arXiv:1810.04805 (2018)

  7. Kuratov, Y., Arkhipov, M.: Adaptation of deep bidirectional multilingual transformers for Russian language (2019)

    Google Scholar 

  8. Kuznetsova, E., Loukachevitch, N., Chetviorkin, I.: Testing rules for a sentiment analysis system. In: Proceedings of International Conference Dialog, pp. 71–80 (2013)

    Google Scholar 

  9. Loukachevitch, N., Rubtsova, Y.: Entity-oriented sentiment analysis of tweets: results and problems. In: Král, P., Matoušek, V. (eds.) TSD 2015. LNCS (LNAI), vol. 9302, pp. 551–559. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24033-6_62

    Chapter  Google Scholar 

  10. Loukachevitch, N., Rubtsova, Y.: SentiRuEval-2016: overcoming time gap and data sparsity in tweet sentiment analysis. In: Proceedings of International Conference Dialog-2016 (2016)

    Google Scholar 

  11. Loukachevitch, N., Rusnachenko, N.: Extracting sentiment attitudes from analytical texts. In: Proceedings of Computational Linguistics and Intellectual Technologies, Papers from the Annual Conference Dialog-2018, pp. 459–468 (2018)

    Google Scholar 

  12. Maas, A., Daly, R., Pham, P., Huang, D., Ng, A.Y., Potts, C.: Learning word vectors for sentiment analysis. In: Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, vol. 1, pp. 142–150 (2011)

    Google Scholar 

  13. Nakov, P., Ritter, A., Rosenthal, S., Sebastiani, F., Stoyanov, V.: Semeval-2016 task 4: sentiment analysis in twitter. In: Proceedings of the 10th International Workshop on Semantic Evaluations, SemEval-2016, pp. 502–518 (2016)

    Google Scholar 

  14. Rogers, A., Romanov, A., Rumshisky, A., Volkova, S., Gronas, M., Gribov, A.: RuSentiment: an enriched sentiment analysis dataset for social media in Russian. In: Proceedings of the 27th International Conference on Computational Linguistics, pp. 755–763 (2018)

    Google Scholar 

  15. Rosenthal, S., Farra, N., Nakov, P.: Semeval-2017 task 4: sentiment analysis in twitter. In: Proceedings of the 11th International Workshop on Semantic Evaluations (SemEval-2017) (2017)

    Google Scholar 

  16. Rubtsova, Y.: Constructing a corpus for sentiment classification training. Softw. Syst. 109, 72–78 (2015)

    Article  Google Scholar 

  17. Rubtsova, Y.: Reducing the deterioration of sentiment analysis results due to the time impact. Information 9(8), 184 (2018)

    Article  Google Scholar 

  18. Rusnachenko, N., Loukachevitch, N., Tutubalina, E.: Distant supervision for sentiment attitude extraction. In: Proceedings of the International Conference on Recent Advances in Natural Language Processing (RANLP 2019), pp. 1022–1030 (2019)

    Google Scholar 

  19. Socher, R., et al.: Recursive deep models for semantic compositionality over a sentiment treebank. In: Proceedings of the 2013 Conference on Empirical Methods in Natural Language Processing, pp. 1631–1642 (2013)

    Google Scholar 

  20. Sun, C., Huang, L., Qiu, X.: Utilizing BERT for aspect-based sentiment analysis via constructing auxiliary sentence. In: Proceedings of the 2019 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies, vol. 1, pp. 380–385 (2019)

    Google Scholar 

  21. Svetlov, K., Platonov, K.: Sentiment analysis of posts and comments in the accounts of Russian politicians on the social network. In: 2019 25th Conference of Open Innovations Association (FRUCT), pp. 299–305. IEEE (2019)

    Google Scholar 

  22. Zhang, Y., Wallace, B.: A Sensitivity Analysis of (and Practitioners’ Guide to) Convolutional Neural Networks for Sentence Classification. arXiv preprint arXiv:1510.03820, pp. 573–580 (2015)

  23. Zvonarev, A., Bilyi, A.: A comparison of machine learning methods of sentiment analysis based on Russian language twitter data. In: The 11th Majorov International Conference on Software Engineering and Computer Systems (2019)

    Google Scholar 

Download references

Acknowledgments

The reported study was funded by RFBR according to the research project № 20-07-01059.

Author information

Authors and Affiliations

  1. Bauman Moscow State Technical University, Moscow, Russia

    Anton Golubev

  2. Lomonosov Moscow State University, Moscow, Russia

    Natalia Loukachevitch

Authors
  1. Anton Golubev
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Natalia Loukachevitch
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Anton Golubev or Natalia Loukachevitch .

Editor information

Editors and Affiliations

  1. ITMO University, St. Petersburg, Russia

    Andrey Filchenkov

  2. Haaga-Helia University of Applied Sciences, Helsinki, Finland

    Janne Kauttonen

  3. University of Helsinki, Helsinki, Finland

    Lidia Pivovarova

Rights and permissions

Reprints and permissions

Copyright information

© 2020 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Golubev, A., Loukachevitch, N. (2020). Improving Results on Russian Sentiment Datasets. In: Filchenkov, A., Kauttonen, J., Pivovarova, L. (eds) Artificial Intelligence and Natural Language. AINL 2020. Communications in Computer and Information Science, vol 1292. Springer, Cham. https://doi.org/10.1007/978-3-030-59082-6_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-59082-6_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-59081-9

  • Online ISBN: 978-3-030-59082-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation