Skip to main content
SpringerLink
Log in

A POS Tagging Model for Vietnamese Social Media Text Using BiLSTM-CRF with Rich Features

  • Conference paper
  • First Online:
PRICAI 2019: Trends in Artificial Intelligence (PRICAI 2019)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 11672))

Included in the following conference series:

Abstract

This paper deals with the task of part-of-speech (POS) tagging for Vietnamese social media text, which poses several challenges compared with tagging for conventional text. We introduce a POS tagging model that takes advantages of deep learning and manually engineered features to overcome the challenges of the task. The main part of the model consists of several bidirectional long short-term memory (BiLSTM) layers that are used to learn intermediate representations of sentences from features extracted at both the character and the word levels. Conditional random field (CRF) is then used on top of those BiLSTM layers, at the inference layer, to predict the most suitable POS tags. We leverage various types of manually engineered features in addition to automatically learned features to capture the characteristics of Vietnamese social media data and therefore improve the performance of the model. Experimental results on a public POS tagging corpus for Vietnamese social media text show that our model outperforms previous work [4] by a large margin, reaching 91.9% accuracy with 27% error rate reduction. The results also reveal the effectiveness of using both automatically learned and manually designed features in a deep learning framework when only a limited amount of training data is available.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 84.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 109.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    Goldberg [11] shows that by stacking several BiLSTM layers we can produce better representations for sentences.

  2. 2.

    http://commons.apache.org/codec/.

  3. 3.

    The division into training and test sets is the same as the work of Bach et al. [4].

  4. 4.

    Software available at https://github.com/jiesutd/NCRFpp.

References

  1. Albogamy, F., Ramsay A.: POS tagging for Arabic tweets. In: Proceedings of RANLP, pp. 1–8 (2015)

    Google Scholar 

  2. Albogamy, F., Ramsay, A.: Fast and robust POS tagger for Arabic tweets using agreement-based bootstrapping. In: Proceedings of LREC, pp. 1500–1506 (2016)

    Google Scholar 

  3. Bach, N.X., Hiraishi, K., Minh, N.L., Shimazu, A.: Dual decomposition for Vietnamese part-of-speech tagging. In: Proceedings of KES, pp. 123–131 (2013)

    Article  Google Scholar 

  4. Bach, N.X., Linh, N.D., Phuong, T.M.: An empirical study on POS tagging for Vietnamese social media text. Comput. Speech Lang. 50, 1–15 (2018)

    Article  Google Scholar 

  5. Brants, T.: TnT: a statistical part-of-speech tagger. In: Proceedings of the Sixth Conference on Applied Natural Language Processing, pp. 224–231 (2000)

    Google Scholar 

  6. Brown, P.F., Desouza, P.V., Mercer, R.L., Pietra, V.D., Lai, J.: Class-based n-gram models of natural language. Comput. Linguist. 18(4), 467–479 (1992)

    Google Scholar 

  7. Derczynski, L., Ritter, A., Clark, S., Bontcheva, K.: Twitter part-of-speech tagging for all: overcoming sparse and noisy data. In: Proceedings of the RANLP, pp. 198–206 (2013)

    Google Scholar 

  8. Elman, J.L.: Finding structure in time. Cogn. Sci. 14(2), 179–211 (1990)

    Article  Google Scholar 

  9. Gimenez, J., Marquez, L.: SVM-tool: a general POS tagger generator based on support vector machines. In: Proceedings of LREC, pp. 43–46 (2004)

    Google Scholar 

  10. Gimpel, K. et al.: Part-of-speech tagging for twitter: annotation, features, and experiments. In: Proceedings of ACL, pp. 42–47 (2011)

    Google Scholar 

  11. Goldberg, Y.: Neural Network Methods for Natural Language Processing. Morgan & Claypool, San Rafael (2017)

    Book  Google Scholar 

  12. Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. 18(5–6), 602–610 (2005)

    Article  Google Scholar 

  13. He, K., Zhang, X., Ren, S., Sun, J.: Deep residual learning for image recognition. arXiv:1512.03385 (2015). https://arxiv.org/abs/1512.03385

  14. Heigold, G., Neumann, G., Genabith, J.V.: An extensive empirical evaluation of character-based morphological tagging for 14 languages. In: Proceedings of EACL, pp. 505–513 (2017)

    Google Scholar 

  15. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  16. Huang, Z., Xu, W., Yu, K.: Bidirectional LSTM-CRF models for sequence tagging. arXiv:1508.01991. https://arxiv.org/abs/1508.01991 (2015)

  17. Labeau, M., Loser, K., Allauzen, A.: Non-lexical neural architecture for fine-grained POS tagging. In: Proceedings of EMNLP, pp. 232–237 (2015)

    Google Scholar 

  18. Lafferty, J., McCallum, A., Pereira, F.: Conditional random fields: probabilistic models for segmenting and labeling sequence data. In: Proceedings of ICML, pp. 282–289 (2001)

    Google Scholar 

  19. Le, H.P., Roussanaly, A., Nguyen, T.M.H., Rossignol, M.: An empirical study of maximum entropy approach for part-of-speech tagging of Vietnamese texts. In: Proceedings of the TALN (2010)

    Google Scholar 

  20. LeCun, Y., Bottou, L., Bengio, Y., Haffner, P.: Gradient-based learning applied to document recognition. Proc. IEEE 86(110), 2278–2324 (1998)

    Article  Google Scholar 

  21. LeCun, Y., Bengio, Y., Hinton, G.: Deep Learning. Nature 521, 436–444 (2015)

    Article  Google Scholar 

  22. Ma, X., Hovy, E.: End-to-end sequence labeling via bi-directional LSTM-CNNs-CRF. In: Proceedings of ACL, pp. 1064–1074 (2016)

    Google Scholar 

  23. Neunerdt, M., Trevisan, B., Reyer, M., Mathar, R.: Part-of-speech tagging for social media texts. In: Gurevych, I., Biemann, C., Zesch, T. (eds.) GSCL 2013. LNCS (LNAI), vol. 8105, pp. 139–150. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40722-2_15

    Chapter  Google Scholar 

  24. Nghiem, M., Dinh, D., Nguyen, M.: Improving Vietnamese POS tagging by integrating a rich feature set and support vector machines. In: Proceedings of RIVF, pp. 128–133 (2008)

    Google Scholar 

  25. Nguyen, L.M., Ngo, X.B., Nguyen, V.C., Pham, Q.N.M., Shimazu, A.: A semi-supervised learning method for Vietnamese part-of-speech tagging. In: Proceedings of KSE, pp. 141–146 (2010)

    Google Scholar 

  26. Nguyen, D.Q., Vu, T., Nguyen, D.Q., Dras, M., Johnson, M.: From word segmentation to POS tagging for Vietnamese. In: Proceedings of ALTA, pp. 108–113 (2017)

    Google Scholar 

  27. Nooralahzadeh, F., Brun, C., Roux, C.: Part of speech tagging for French social media data. In: Proceedings of COLING, pp. 1764–1772 (2014)

    Google Scholar 

  28. Owoputi, O., O’Connor, B., Dyer, C., Gimpel, K., Schneider, N., Smith, N.A.: Improved part-of-speech tagging for online conversational text with word clusters. In: Proceedings of NAACL, pp. 380–390 (2013)

    Google Scholar 

  29. Plank, B., Sogaard, A., Goldberg, Y.: Multilingual part-of-speech tagging with bidirectional long short-term memory models and auxiliary loss. In: Proceedings of ACL, pp. 412–418 (2016)

    Google Scholar 

  30. Ratnaparkhi, A.: A maximum entropy model for part-of-speech tagging. In: Proceedings of EMNLP, pp. 133–142 (1996)

    Google Scholar 

  31. Rehbein, I.: Fine-grained POS tagging of German tweets. In: Gurevych, I., Biemann, C., Zesch, T. (eds.) GSCL 2013. LNCS (LNAI), vol. 8105, pp. 162–175. Springer, Heidelberg (2013). https://doi.org/10.1007/978-3-642-40722-2_17

    Chapter  Google Scholar 

  32. Shao, Y., Hardmeier, C., Tiedemann, J., Nivre, J.: Character-based joint segmentation and POS tagging for Chinese using bidirectional RNN-CRF. In: Proceedings of IJCNLP, pp. 173–183 (2017)

    Google Scholar 

  33. Stratos, K., Kim, D., Collins, M., Hsu, D.: A spectral algorithm for learning class-based n-gram models of natural language. In: Proceedings of UAI, pp. 762–771 (2014)

    Google Scholar 

  34. Stratos, K., Collins, M.: Simple semi-supervised POS tagging. In: Proceedings of NAACL-HLT, pp. 79–87 (2015)

    Google Scholar 

  35. Toutanova, K., Manning, C.: Enriching the knowledge sources used in a maximum entropy part-of-speech tagger. In: Proceedings of EMNLP, pp. 63–70 (2000)

    Google Scholar 

  36. Tran, T.O., Le, A.C., Ha, Q.T., Le, H.Q.: An experimental study on Vietnamese POS tagging. In: Proceedings of IALP, pp. 23–27 (2009)

    Google Scholar 

  37. Wang, P., Qian, Y., Soong, F.K., He, L., Zhao, H.: Part-of-speech tagging with bidirectional long short-term memory recurrent neural network. arXiv:1510.06168 (2015). https://arxiv.org/abs/1510.06168

  38. Yang, J., Zhang, Y.: NCRF++: an open-source neural sequence labeling toolkit. In: Proceedings of ACL-System Demonstrations, pp. 74–79 (2018)

    Google Scholar 

  39. Young, T., Hazarika, D., Poria, S., Cambria, E.: Recent trends in deep learning based natural language processing. arXiv:1708.02709v8 (2018). https://arxiv.org/abs/1708.02709

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Posts and Telecommunications Institute of Technology, Hanoi, Vietnam

    Ngo Xuan Bach, Trieu Khuong Duy & Tu Minh Phuong

Authors
  1. Ngo Xuan Bach
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Trieu Khuong Duy
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Tu Minh Phuong
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Ngo Xuan Bach .

Editor information

Editors and Affiliations

  1. Department of Computing, Macquarie University, Sydney, NSW, Australia

    Abhaya C. Nayak

  2. RIKEN Center for Integrative Medical Sciences, Yokohama, Japan

    Alok Sharma

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Xuan Bach, N., Khuong Duy, T., Minh Phuong, T. (2019). A POS Tagging Model for Vietnamese Social Media Text Using BiLSTM-CRF with Rich Features. In: Nayak, A., Sharma, A. (eds) PRICAI 2019: Trends in Artificial Intelligence. PRICAI 2019. Lecture Notes in Computer Science(), vol 11672. Springer, Cham. https://doi.org/10.1007/978-3-030-29894-4_16

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-29894-4_16

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-29893-7

  • Online ISBN: 978-3-030-29894-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation