Abstract
Traders often rely on financial news to come up with predictions for stock price changes. Dealing with vast amount of news data makes it essential to use an automated methodology to identify the relevant news items for a given criteria. In this study we use Latent Dirichlet Allocation (LDA) to model the correlation of news items with stock price time series data. LDA model is trained with news items from a time window in the past and then the trained model is used to measure the similarity between the current news items and the news items used for training. Calculated similarity measure can be used as a predictor for switching points in the future. We tested our methodology using a collection of about 1,700,000 financial news items published between 2015-01-01 and 2015-12-31, and compared the results with various standard classification techniques. Our results indicate that use of LDA instead of standard classification techniques makes it possible to achieve the same level of performance by using a much smaller feature space.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
With document similarity threshold = 0.99, 0.98, number of topics = 100, 100, doc-topic prior = 0.01, 0.001, topic-words prior = 0.01, 0.005, max-iter = 200, 200, learning-offset = 64, - and learning-decay = 0.5, - for online and offline learning respectively.
References
Avellaneda, M., Stoikov, S.: High-frequency trading in a limit order book. Quant. Financ. 8(3), 217–224 (2008)
Mittermayer, M.A.: Forecasting intraday stock price trends with text mining techniques. In: Proceedings of the 37th Annual Hawaii International Conference on System Sciences, 10-pp. IEEE (2004)
Boudoukh, J., Feldman, R., Kogan, S., Richardson, M.: Which news moves stock prices? A textual analysis. National Bureau of Economic Research (2013)
Xie, B., Passonneau, R.J., Wu, L., Creamer, G.G.: Semantic frames to predict stock price movement (2013)
Li, X., Xie, H., Chen, L., Wang, J., Deng, X.: News impact on stock price return via sentiment analysis. Knowl.-Based Syst. 69, 14–23 (2014)
Thomas, J.D., Sycara, K.: GP and the predictive power of internet message traffic. In: Chen, S.H. (ed.) Genetic Algorithms and Genetic Programming in Computational Finance, pp. 81–102. Springer, Boston (2002). https://doi.org/10.1007/978-1-4615-0835-9_4
Nguyen, T.H., Shirai, K.: Topic modeling based sentiment analysis on social media for stock market prediction. In: ACL (1), pp. 1354–1364 (2015)
Papadimitriou, C.H., Tamaki, H., Raghavan, P.: Latent semantic indexing: a probabilistic analysis. J. Comput. Syst. Sci. 61, 217–235 (2000)
Hofmann, T.: Probabilistic latent semantic indexing. In: Proceedings of the Fifteenth Conference on Uncertainty in Artificial Intelligence, pp. 289–296 (1999)
Hoffman, M., Bach, F.R., Blei, D.M.: Online learning for latent Dirichlet allocation. In: Advances in Neural Information Processing Systems, pp. 856–864 (2010)
Blei, D.M., Ng, A.Y., Jordan, M.I.: Latent Dirichlet allocation. J. Mach. Learn. Res. 3, 993–1022 (2003)
Canini, K., Shi, L., Griffiths, T.: Online inference of topics with latent Dirichlet allocation. In: Artificial Intelligence and Statistics, pp. 65–72 (2009)
Habert, B., Adda, G., Adda-Decker, M., de Marëuil, P.B., Ferrari, S., Ferret, O.: Towards tokenization evaluation. In: Proceedings of LREC, pp. 427–431 (1998)
Sarkar, D.: Text analytics with Python (2016)
Alpaydin, E.: Introduction to Machine Learning. MIT Press, Cambridge (2014)
Acknowledgments
This research is supported by TMX financial services company, NSERC CRDPJ under the grant number 499983-16, and OCE VIPII under the grant number 26280.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG, part of Springer Nature
About this paper
Cite this paper
Asadi Kakhki, S.S., Kavaklioglu, C., Bener, A. (2018). Topic Detection and Document Similarity on Financial News. In: Bagheri, E., Cheung, J. (eds) Advances in Artificial Intelligence. Canadian AI 2018. Lecture Notes in Computer Science(), vol 10832. Springer, Cham. https://doi.org/10.1007/978-3-319-89656-4_34
Download citation
DOI: https://doi.org/10.1007/978-3-319-89656-4_34
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-89655-7
Online ISBN: 978-3-319-89656-4
eBook Packages: Computer ScienceComputer Science (R0)