Challenge
The goal here is to predict the intraday movements of the SPY S&P500 ETF from historical data. There is an inherent paradox in the challenge to build a model to predict SPY making it almost impossible to to build such a model. However it is possible to build models where patterns, and broad trends, can be identified from seemingly random data. The challenge here is to identify those patterns, along with a confidence factor with respect to which the probability of the prediction will be correct.
Solution
We have employed deep learning for this tough problem. Specifically, we designed and developed stockNet, which is a combination of CNN and LSTM, i.e. DNN architectures. We employed causal dilated convolutions in order to increase the receptive field to capture short and long-term dependencies and to ensure causality, i.e. the model would not violate temporal ordering since the prediction cannot depend on future data. Finally an LSTM was employed in order to achieve sequence-to-sequence learning using neural networks.
Results
StockNet was able to capture intra-day patterns and broad trends using historical data and thus forecast the intra-day movements of the SPY. In addition on top of this forecasting we were able to learn and deploy intra-day trading strategies for revenue optimization.