Abstract
In this paper we present and implement different Reinforcement Learning (RL) algorithms in financial trading systems. RL-based approaches aim to find an optimal policy, that is an optimal mapping between the variables describing an environment state and the actions available to an agent, by interacting with the environment itself in order to maximize a cumulative return. In particular, we compare the results obtained considering different on-policy (SARSA) and off-policy (Q-Learning, Greedy-GQ) RL algorithms applied to daily trading in the Italian stock market. We both consider computational issues and investigate practical solutions applications, in an effort to improve previous results while keeping a simple and understandable structure of the used models.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Barto, A.G., Sutton, R.S.: Reinforcement Learning: An Introduction. The MIT Press, Boston (2018)
Bellman, R.E.: Dynamic Programming. Princeton University Press, Princeton (1957)
Bloomberg Finance L.P.: https://www.bloomberg.com/professional/product/market-data/
Corazza, M., Sangalli, A.: Q-learning and SARSA: a comparison between two intelligent stochastic control approaches for financial trading. Working Papers, Department of Economics, Ca’ Foscari University of Venice, 15 (2015)
Geramifard, A., Dann, C., How, J.P.: Off-policy learning combined with automatic feature expansion for solving large MDPs. In: Proceedings of the 1st Multidisciplinary Conference on Reinforcement Learning and Decision Making, pp. 29–33. Princeton University Press, Princeton (2013)
Gosavi, A.: Simulation-Based Optimization. Parametric Optimization Techniques and Reinforcement Learning. Springer, Berlin (2015)
Lo, A.W.: Adaptive Markets. Financial Evolution at the Speed of Thought. Princeton University Press, Princeton (2017)
Maei, H.R., Szepesvári, C., Bhatnagar, S., Sutton, R.S.: Toward off-policy learning control with function approximation. In: International Conference on Machine Learning (ICML), pp. 719–726. Omnipress, Madison (2010)
Rummery, G.A., Niranjan, M.: On-line Q-Learning using connectionist systems. Technical Report CUED/F-INFENG/TR, 166, Engineering Department, Cambridge University (1994)
Watkins, C.J.C.H., Dayan, P.: Q-learning. Mach. Learn. 8, 279–292 (1992)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 The Author(s), under exclusive license to Springer Nature Switzerland AG
About this paper
Cite this paper
Corazza, M., Fasano, G., Gusso, R., Pesenti, R. (2021). Comparing RL Approaches for Applications to Financial Trading Systems. In: Corazza, M., Gilli, M., Perna, C., Pizzi, C., Sibillo, M. (eds) Mathematical and Statistical Methods for Actuarial Sciences and Finance. Springer, Cham. https://doi.org/10.1007/978-3-030-78965-7_22
Download citation
DOI: https://doi.org/10.1007/978-3-030-78965-7_22
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-78964-0
Online ISBN: 978-3-030-78965-7
eBook Packages: Economics and FinanceEconomics and Finance (R0)