Reinforcement Learning for Financial Portfolio Management: A study of Neural Networks for Reinforcement Learning on currency exchange market

Abstract

Portfolio management is the process of continually reallocating funds into financial instruments, aiming to maximize the return. This paper presents a Reinforcement Learning framework where an agent interacts with the trading environment to learn a strategy for decision making. Forex exchange data for five currencies are used in this study. We briefly introduce RL methods and advance towards what practically can be implemented in this study; then, model the agent’s behavior using Artificial Neural Networks, Convolutional Neural Networks, and devise an Actor-Critic approach. A variety of network topologies for each of the three approaches are studied. Furthermore, several hyperparameters, optimizers, and activation and loss functions have been applied to the models. The aim of the agent is to buy, sell, or hold the currencies to maximize the expected return of the portfolio. The performance and the profit yields of all models are evaluated. Several models from various classes of approaches, make over 10% profit with 0.1% commission rate. On the other hand, several other models result in loss or do not converge. A2C approaches are more likely to converge than CNNs and ANNs are the least likely ones to converge. We propose what modifications can be made to the framework to study to make improvements.