Abstract
The adoption of computer-aided stock trading methods is gaining popularity in recent years, mainly because of their ability to process efficiently past information through machine learning to predict future market behavior. Several approaches have been proposed to this task, with the most effective ones using fusion of a pile of classifiers decisions to predict future stock values. However, using prices information in single supervised classifiers has proven to lead to poor results, mainly because market history is not enough to be an indicative of future market behavior. In this paper, we propose to tackle this issue by proposing a multi-layer and multi-ensemble stock trader. Our method starts by pre-processing data with hundreds of deep neural networks. Then, a reward-based classifier acts as a meta-learner to maximize profit and generate stock signals through different iterations. Finally, several metalearner trading decisions are fused in order to get a more robust trading strategy, using several trading agents to take a final decision. We validate the effectiveness of the approach in a real-world trading scenario, by extensively testing it on the Standard & Poor’s 500 future market and the J.P. Morgan and Microsoft stocks. Experimental results show that the proposed method clearly outperforms all the considered baselines (which still performs very well in the analysed period), and even the conventional Buy-and-Hold strategy, which replicates the market behaviour.
Similar content being viewed by others
References
Ahmadian S, Khanteymoori AR (2015) Training back propagation neural networks using asexual reproduction optimization. In: Conference on information and knowledge technology (IKT), pp. 1–6
An N, Ding H, Yang J, Au R, Ang TF (2020) Deep ensemble learning for alzheimer’s disease classification. J Biomed Inform 105:103411. https://doi.org/10.1016/j.jbi.2020.103411
Asad M (2015) Optimized stock market prediction using ensemble learning. In: 2015 9Th international conference on application of information and communication technologies (AICT), pp 263–268
Barra S, Carta SM, Corriga A, Podda AS, Recupero DR (2020) Deep learning and time series-to-image encoding for financial forecasting. IEEE/CAA J Autom Sin 73:683–692
Calvi GG, Lucic V, Mandic DP (2019) Support tensor machine for financial forecasting. In: ICASSP 2019 - 2019 IEEE International conference on acoustics, speech and signal processing (ICASSP), pp 8152–8156. https://doi.org/10.1109/ICASSP.2019.8683383
Chun SH, Park YJ (2005) Dynamic adaptive ensemble case-based reasoning: application to stock market prediction. Expert Syst Appl 28(3):435–443
Fenghua W, Jihong X, Zhifang H, Xu G (2014) Stock price prediction based on ssa and svm. Procedia Comput Sci 31:625–631. 2nd International Conference on Information Technology and Quantitative Management ITQM
Fu TC, Lee KK, Sze D, Chung FL, Ng CM (2008) Discovering the correlation between stock time series and financial news. In: Proceedings of the 2008 IEEE/WIC/ACM International Conference on Web Intelligence and Intelligent Agent Technology - Volume 01, WI-IAT ’08. IEEE Computer Society, pp 880–883. https://doi.org/10.1109/WIIAT.2008.228
Gao X, Hongkong S, Chan L (2000) An algorithm for trading and portfolio management using q-learning and sharpe ratio maximization. In: International conference on neural information processing, pp 832–837
Gyamerah SA, Ngare P, Ikpe D (2019) On stock market movement prediction via stacking ensemble learning method. In: IEEE Conference on computational intelligence for financial engineering economics (CIFEr), pp 1–8
Han J, Kamber M, Pei J (2011) Data Transformation and Data Discretization, chap. 3. Elsevier, pp 111–118
Hasselt H (2010) Double q-learning. In: Lafferty JD, Williams CKI, Shawe-Taylor J, Zemel RS, Culotta A (eds) Advances in neural information processing systems, vol 23, Curran Associates, Inc, pp 2613–2621
Jalali SMJ, Ahmadian S, Kebria PM, Khosravi A, Lim CP, Nahavandi S (2019) Evolving artificial neural networks using butterfly optimization algorithm for data classification. In: Neural information processing. Springer International Publishing, Cham, pp 596–607
Jalali SMJ, Ahmadian S, Khosravi A, Mirjalili S, Mahmoudi MR, Nahavandi S (2020) Neuroevolution-based autonomous robot navigation: a comparative study. Cogn Syst Res 62:35–43
Kamijo KI, Tanigawa T (1990) Stock price pattern recognition-a recurrent neural network approach. In: 1990 IJCNN International joint conference on neural networks. IEEE, pp 215–221
Kang Q, Zhou H, Kang Y (2018) An asynchronous advantage actor-critic reinforcement learning method for stock selection and portfolio management. In: Proceedings of the 2nd International Conference on Big Data Research, ICBDR 2018. Association for Computing Machinery, New York, pp 141–145. https://doi.org/10.1145/3291801.3291831
Khairi TWA, Zaki R.M, Mahmood WA (2019) Stock price prediction using technical, fundamental and news based approach. In: 2019 2Nd scientific conference of computer sciences (SCCS), pp 177–181
Khan W, Ghazanfar MA, Azam MA, Karami A, Alyoubi KH, Alfakeeh AS (2020) Stock market prediction using machine learning classifiers and social media, news. Journal of Ambient Intelligence and Humanized Computing
Kim T, Kim HY (2019) Forecasting stock prices with a feature fusion lstm-cnn model using different representations of the same data. PLOS One 14(2):1–23. https://doi.org/10.1371/journal.pone.0212320
Kimoto T, Asakawa K, Yoda M, Takeoka M (1990) Stock market prediction system with modular neural networks. In: 1990 IJCNN International joint conference on neural networks. IEEE, pp 1–6
Lee CH, Park KC (1992) Prediction of monthly transition of the composition stock price index using recurrent back-propagation. In: Artificial neural networks. Elsevier, pp 1629–1632
Lee J, Park J, Jangmin O, Lee J, Hong E (2007) A multiagent approach to $q$-learning for daily stock trading. IEEE Trans Syst Man Cybern Part A Syst Hum 37:864–877
Lei K, Zhang B, Li Y, Yang M, Shen Y (2020) Time-driven feature-aware jointly deep reinforcement learning for financial signal representation and algorithmic trading. Expert Syst Appl 140:112872. https://doi.org/10.1016/j.eswa.2019.112872, http://www.sciencedirect.com/science/article/pii/S0957417419305822
Lin Y, Huang T, Chung W, Ueng Y (2020) Forecasting fluctuations in the financial index using a recurrent neural network based on price features. IEEE Transactions on Emerging Topics in Computational Intelligence, pp 1–12
Magdon-Ismail M, Atiya AF (2004) Maximum drawdown. Risk Mag 17(10):99–102
Mihatsch O, Neuneier R (1999) Risk-sensitive reinforcement learning. In: Advances in neural information processing systems. MIT press, pp 1031–1037
Moody J, Wu L, Liao Y, Saffell M (1998) Performance functions and reinforcement learning for trading systems and portfolios. J Forecast 17(5-6):441–470
Patil P, Wu CSM, Potika K, Orang M (2020) Stock market prediction using ensemble of graph theory, machine learning and deep learning models. In: Proceedings of the 3rd International Conference on Software Engineering and Information Management, ICSIM ’20. Association for Computing Machinery, New York, pp 85–92
Plappert M (2016) keras-rl. https://github.com/keras-rl/keras-rl
Puterman M (2014) Markov decision processes: discrete stochastic dynamic programming. Wiley, New York
Sezer OB, Ozbayoglu AM (2018) Algorithmic financial trading with deep convolutional neural networks: Time series to image conversion approach. Appl Soft Comput 70:525–538. https://doi.org/10.1016/j.asoc.2018.04.024. http://www.sciencedirect.com/science/article/pii/S1568494618302151
Si W, Li J, Ding P, Rao R (2017) A multi-objective deep reinforcement learning approach for stock index future’s intraday trading. In: International symposium on computational intelligence and design (ISCID), vol 2, pp 431–436
Sim HS, Kim HI, Ahn JJ (2019) Is deep learning for image recognition applicable to stock market prediction? Complexity
Sun T, Wang J, Ni J, Cao Y, Liu B (2019) Predicting futures market movement using deep neural networks. In: 18Th IEEE international conference on machine learning and applications (ICMLA), pp 118–125
Surton R, Barto A (1998) Reinforcement Learning: an introduction, vol 1. MIT press, Cambridge
Tan TZ, Quek C, Ng GS (2005) Brain-inspired genetic complementary learning for stock market prediction. In: 2005 IEEE Congress on evolutionary computation, vol 3. IEEE, pp 2653–2660
Tan Z, Yan Z, Zhu G (2019) Stock selection with random forest: An exploitation of excess return in the chinese stock market. Heliyon 5(8):e02310. https://doi.org/10.1016/j.heliyon.2019.e02310. http://www.sciencedirect.com/science/article/pii/S2405844019359705
Wang Z, Oates T (2015) Encoding time series as images for visual inspection and classification using tiled convolutional neural networks. In: Workshops at the twenty-ninth AAAI conference on artificial intelligence
Wang Z, Schaul T, Hessel M, Hasselt H, Lanctot M, Freitas N (2016) Dueling network architectures for deep reinforcement learning. In: International Conference on Machine Learning, Proceedings of Machine Learning Research, vol 48. PMLR, New York, pp 1995–2003
Wolpert DH (1992) Stacked generalization. Neural Netw 5:241–259
Wolpert DH, Macready WG (1997) No free lunch theorems for optimization. Trans Evol Comp 1(1):67–82. https://doi.org/10.1109/4235.585893
Wu Y, Mao J, Li W (2018) Predication of futures market by using boosting algorithm. In: International conference on wireless communications, signal processing and networking (wiSPNET), pp 1–4
Ye C, Ma H, Zhang X, Zhang K, You S (2017) Survival-oriented reinforcement learning model: an effcient and robust deep reinforcement learning algorithm for autonomous driving problem. In: Zhao Y, Kong X, Taubman D (eds) Image and graphics. Springer International Publishing, Cham, pp 417–429
Zhang Y, Wu L (2009) Stock market prediction of s&p 500 via combination of improved bco approach and bp neural network. Expert Syst Appl 36(5):8849–8854
Zhou Z, Gao M, Liu Q, Xiao H (2020) Forecasting stock price movements with multiple data sources: Evidence from stock market in china. Physica A Stat Mech Appl 542:123389. https://doi.org/10.1016/j.physa.2019.123389. http://www.sciencedirect.com/science/article/pii/S0378437119318941
Author information
Authors and Affiliations
Corresponding author
Additional information
Publisher’s note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
The research performed in this paper has been supported by the “Bando ”Aiuti per progetti di Ricerca e Sviluppo“-POR FESR 2014-2020—Asse 1, Azione 1.1.3, Strategy 2- Program 3, Project AlmostAnOracle - AI and Big Data Algorithms for Financial Time Series Forecasting”
Rights and permissions
About this article
Cite this article
Carta, S., Corriga, A., Ferreira, A. et al. A multi-layer and multi-ensemble stock trader using deep learning and deep reinforcement learning. Appl Intell 51, 889–905 (2021). https://doi.org/10.1007/s10489-020-01839-5
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10489-020-01839-5