Skip to main content
SpringerLink
Log in

One Single Deep Bidirectional LSTM Network for Word Sense Disambiguation of Text Data

  • Conference paper
  • First Online:
Advances in Artificial Intelligence (Canadian AI 2018)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 10832))

Included in the following conference series:

Abstract

Due to recent technical and scientific advances, we have a wealth of information hidden in unstructured text data such as offline/online narratives, research articles, and clinical reports. To mine these data properly, attributable to their innate ambiguity, a Word Sense Disambiguation (WSD) algorithm can avoid numbers of difficulties in Natural Language Processing (NLP) pipeline. However, considering a large number of ambiguous words in one language or technical domain, we may encounter limiting constraints for proper deployment of existing WSD models. This paper attempts to address the problem of one-classifier-per-one-word WSD algorithms by proposing a single Bidirectional Long Short-Term Memory (BLSTM) network which by considering senses and context sequences works on all ambiguous words collectively. Evaluated on SensEval-3 benchmark, we show the result of our model is comparable with top-performing WSD algorithms. We also discuss how applying additional modifications alleviates the model fault and the need for more training data.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://www.nlm.nih.gov/research/umls/.

  2. 2.

    http://www.senseval.org/senseval3.

  3. 3.

    http://www.wordsmyth.net/.

  4. 4.

    https://wsd.nlm.nih.gov/collaboration.shtml.

  5. 5.

    https://www.nlm.nih.gov/.

References

  1. Vickrey, D., Biewald, L., Teyssier, M., Koller, D.: Word-sense disambiguation for machine translation. In: Proceedings of Human Language Technology Conference and Conference on Empirical Methods in Natural Language Processing (2005)

    Google Scholar 

  2. Hung, J.C., Wang, C.S., Yang, C.Y., Chiu, M.S., Yee, G.: Applying word sense disambiguation to question answering system for e-learning. In: 19th International Conference on Advanced Information Networking and Applications, AINA 2005, vol. 1, pp. 157–162. IEEE (2005)

    Google Scholar 

  3. Pennington, J., Socher, R., Manning, C.: GloVe: global vectors for word representation. In: Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), pp. 1532–1543 (2014)

    Google Scholar 

  4. Mihalcea, R., Chklovski, T., Kilgarriff, A.: The Senseval-3 English lexical sample task. In: Proceedings of SENSEVAL-3, The Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text (2004)

    Google Scholar 

  5. Zhong, Z., Ng, H.T.: It makes sense: a wide-coverage word sense disambiguation system for free text. In: Proceedings of the ACL 2010 System Demonstrations, pp. 78–83. Association for Computational Linguistics (2010)

    Google Scholar 

  6. Miller, G.A.: Wordnet: a lexical database for English. Commun. ACM 38(11), 39–41 (1995)

    Article  Google Scholar 

  7. Pesaranghader, A., Pesaranghader, A., Mustapha, N.: Word sense disambiguation for biomedical text mining using definition-based semantic relatedness and similarity measures. Int. J. Biosci. Biochem. Bioinform. 4(4), 280 (2014)

    Google Scholar 

  8. Kim, S., Yoon, J.: Link-topic model for biomedical abbreviation disambiguation. J. Biomed. Inf. 53, 367–380 (2015)

    Article  Google Scholar 

  9. Navigli, R.: Word sense disambiguation: a survey. ACM Comput. Surv. (CSUR) 41(2), 10 (2009)

    Article  Google Scholar 

  10. Collobert, R., Weston, J.: A unified architecture for natural language processing: deep neural networks with multitask learning. In: Proceedings of the 25th International Conference on Machine Learning, pp. 160–167. ACM (2008)

    Google Scholar 

  11. Mikolov, T., Chen, K., Corrado, G., Dean, J.: Efficient estimation of word representations in vector space. arXiv preprint arXiv:1301.3781 (2013)

  12. Chen, X., Liu, Z., Sun, M.: A unified model for word sense representation and disambiguation. In: EMNLP, pp. 1025–1035 (2014)

    Google Scholar 

  13. Navigli, R., Litkowski, K.C., Hargraves, O.: SemEval-2007 task 07: coarse-grained English all-words task. In: Proceedings of the 4th International Workshop on Semantic Evaluations, pp. 30–35. Association for Computational Linguistics (2007)

    Google Scholar 

  14. Hochreiter, S., Schmidhuber, J.: Long short-term memory. Neural Comput. 9(8), 1735–1780 (1997)

    Article  Google Scholar 

  15. Graves, A., Schmidhuber, J.: Framewise phoneme classification with bidirectional LSTM and other neural network architectures. Neural Netw. 18(5), 602–610 (2005)

    Article  Google Scholar 

  16. Kågebäck, M., Salomonsson, H.: Word sense disambiguation using a bidirectional LSTM. arXiv preprint arXiv:1606.03568 (2016)

  17. Taghipour, K., Ng, H.T.: Semi-supervised word sense disambiguation using word embeddings in general and specific domains. In: HLT-NAACL, pp. 314–323 (2015)

    Google Scholar 

  18. McInnes, B.T., Pedersen, T.: Evaluating measures of semantic similarity and relatedness to disambiguate terms in biomedical text. J. Biomed. Inform. 46(6), 1116–1124 (2013)

    Article  Google Scholar 

  19. Pedersen, T., Kolhatkar, V.: Wordnet::Senserelate::Allwords: a broad coverage word sense tagger that maximizes semantic relatedness. In: Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, Companion Volume: Demonstration Session. Association for Computational Linguistics, pp. 17–20 (2009)

    Google Scholar 

  20. Hinton, G., Srivastava, N., Swersky, K.: RMSprop: divide the gradient by a running average of its recent magnitude. Neural Networks for Machine Learning, COURSERA Lecture 6e (2012)

    Google Scholar 

  21. Srivastava, N., Hinton, G.E., Krizhevsky, A., Sutskever, I., Salakhutdinov, R.: Dropout: a simple way to prevent neural networks from overfitting. J. Mach. Learn. Res. 15(1), 1929–1958 (2014)

    MathSciNet  MATH  Google Scholar 

  22. Iyyer, M., Manjunatha, V., Boyd-Graber, J., Daumé III, H.: Deep unordered composition rivals syntactic methods for text classification. In: Proceedings of the 53rd Annual Meeting of the Association for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing (Volume 1: Long Papers), vol. 1, pp. 1681–1691 (2015)

    Google Scholar 

  23. Grozea, C.: Finding optimal parameter settings for high performance word sense disambiguation. In: Proceedings of SensEval-3 Workshop (2004)

    Google Scholar 

  24. Strapparava, C., Gliozzo, A., Giuliano, C.: Pattern abstraction and term similarity for word sense disambiguation: IRST at SensEval-3. In: Proceedings of SENSEVAL-3 Third International Workshop on Evaluation of Systems for the Semantic Analysis of Text, pp. 229–234 (2004)

    Google Scholar 

  25. Lee, Y.K., Ng, H.T., Chia, T.K.: Supervised word sense disambiguation with support vector machines and multiple knowledge sources. In: SensEval-3: Third International Workshop on the Evaluation of Systems for the Semantic Analysis of Text, pp. 137–140 (2004)

    Google Scholar 

  26. Pesaranghader, A., Matwin, S., Sokolova, M., Beiko, R.G.: simDEF: definition-based semantic similarity measure of gene ontology terms for functional similarity analysis of genes. Bioinformatics 32(9), 1380–1387 (2015)

    Article  Google Scholar 

  27. Pesaranghader, A., Rezaei, A., Pesaranghader, A.: Adapting gloss vector semantic relatedness measure for semantic similarity estimation: an evaluation in the biomedical domain. In: Kim, W., Ding, Y., Kim, H.-G. (eds.) JIST 2013. LNCS, vol. 8388, pp. 129–145. Springer, Cham (2014). https://doi.org/10.1007/978-3-319-06826-8_11

    Chapter  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Institute for Big Data Analytics, Dalhousie University, Halifax, Canada

    Ahmad Pesaranghader & Stan Matwin

  2. School of Computer Science, University of Ottawa, Ottawa, Canada

    Ali Pesaranghader & Marina Sokolova

  3. School of Epidemiology and Public Health, University of Ottawa, Ottawa, Canada

    Marina Sokolova

Authors
  1. Ahmad Pesaranghader
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ali Pesaranghader
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Stan Matwin
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Marina Sokolova
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding authors

Correspondence to Ahmad Pesaranghader or Ali Pesaranghader .

Editor information

Editors and Affiliations

  1. Ryerson University, Toronto, Ontario, Canada

    Ebrahim Bagheri

  2. McGill University, Montréal, Québec, Canada

    Jackie C.K. Cheung

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG, part of Springer Nature

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Pesaranghader, A., Pesaranghader, A., Matwin, S., Sokolova, M. (2018). One Single Deep Bidirectional LSTM Network for Word Sense Disambiguation of Text Data. In: Bagheri, E., Cheung, J. (eds) Advances in Artificial Intelligence. Canadian AI 2018. Lecture Notes in Computer Science(), vol 10832. Springer, Cham. https://doi.org/10.1007/978-3-319-89656-4_8

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-89656-4_8

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-89655-7

  • Online ISBN: 978-3-319-89656-4

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation