High Frequency Trading with Deep Reinforcement Learning Agents Under a Directional Changes Sampling Framework

High frequency trading strategies in the foreign exchange (FX) market often attempt to extract the latent signals in extremely noisy price moves to help inform trading decisions. Due to the fast-paced environments within which these decisions are made, intelligent trading is an impossible task for the human mind. Deep reinforcement learning (DRL) offers human-like intelligence and high speed computation but, due to the noisy nature of the tick data, can be prone to learning sub-optimal policies as a result of misleading feature and reward signals. In this work we use an intrinsic time sampling method referred to as directional changes (DC), which reports information whenever there is a significant change in price. By sampling tick data from nine FX currency pairs for 2250 datasets, we were able to train reinforcement learning (RL) agents using the Proximal Policy Optimisation (PPO) algorithm to identify and trade profitable strategies in high frequency FX environments. The resultant models were compared to four benchmarks including buy and hold, moving average crossover, relative strength index and a rule-based DC strategy, across three different metrics (namely returns, maximum drawdown, and Calmar ratio), with the reinforcement learning models outperforming them all.