Now we turn to our LSTM implementation. First of all, lets look at the cumulative returns:
Figure 48: Cumulative returns and vol adjustment Naive vs HRP - LSTM
The first thing to notice in Figure 48 is the admittedly subpar performance when measured solely on cumulative returns. The strategy employing the naive diversification does not manage to outperform the benchmark, not in raw cumulative returns, neither in the volatility matched cumulative returns.
The strategy employing HRP seems to match the benchmark in terms of raw cumulative returns but fails to match in the volatility matched cumulative returns. From this initial analysis, the LSTM strategy does not seem like a favorable method of investment.
The results do not get better when we examine the rolling volatility and drawdown periods in Figure 49:
Figure 49: Rolling volatility and Top 5 Drawdown Periods Naive vs HRP - LSTM
Notice how from start of backtest to ultimo 2017, the LSTM strategy employs a higher rolling volatil-ity than the benchmark. However, post 2017 it does seem to match and even dip slightly below that of the benchmark. We can also see this in Figure 48. The strategy has a big rally up until mid-2015 way above the index, and afterwards stays below it. The drawdown periods are equally bad. For 1/N we observe a drawdown period of almost the entire duration of the backtest, this does improve on HRP but the results are still not satisfactory. From appendix two, we can see that the strategies take 1103 and 693 days to recover, respectively.
Comparing the 1/N and HRP implementations, we get the following table of performance measures in Table 4:
Table 4: LSTM performance
Strategy Long Short-Term Memory
Asset Allocation Naive Diversification (1/N) Hierarchical Risk Parity
Cumulative Returns 51.5% 75.6%
Annual Returns 6.9% 9.5%
Annual Volatility 15.5% 16.0%
Sharpe Ratio 0.51 0.65
Sortino Ratio 0.74 0.92
Max. Drawdown -30.9% -26.5%
Daily VaR95% -1.9% -2.0%
Daily Turnover 14.7% 15.9%
α against S&P 500 -0.01 0.01
β against S&P 500 0.86 0.89
Start Date 07-01-2014
End Date 31-12-2019
Total Months 74
1. Returns: Cumulative- and Annual returns are significantly higher for HRP.
2. Variance: Annual volatility and Daily VaR95% are lower for 1/N while Max. Drawdown is better for HRP.
3. Performance Criteria: The Sortino- and Sharpe-ratios are both higher for HRP.
4. Cost: The daily turnover slightly favors 1/N.
5. CAPM: While neither exhibit an explicit α, HRP does have the edge.
As such, the partial conclusion for the LSTM strategy is that Hierarchical Risk Parity once again outperforms the naive 1/N diversification.
8.4 Comparison of Individual Strategies
In this section we will compare the individual strategies on their performance metrics, starting with their cumulative returns in Figure 50 with the best of each strategy (being HRP for all three):
Figure 50: Comparison of the best methods (HRP) for each individual strategy
The first thing we notice is the apparent out-performance of the XGBoost strategy compared to the other two. While this is not necessarily indicative of a ”better” strategy, it is a decent indicator of what is to come, namely that the XGBoost strategy does seem to outperform the others in several performance metrics, which we see in Table 5. There is only a short period of time around 2018, where momentum has higher return. Additionally, we also see that momentum is able to ”prevent” some losses by not trading in late 2018, where the two others have negative returns.
Table 5: Performance of individual strategies
Strategy Momentum XGBoost LSTM
Asset Allocation 1/N HRP 1/N HRP 1/N HRP
Cumulative Returns 90.7% 102.8% 109.9% 132.0% 51.5% 75.6%
Annual Returns 11.0% 12.1% 12.7% 14.5% 6.9% 9.5%
Annual Volatility 14.6% 14.2% 15.6% 14.7% 15.5% 16.0%
Sharpe Ratio 0.79 0.87 0.85 1.00 0.51 0.65
Sortino Ratio 1.10 1.24 1.21 1.43 0.74 0.92
Max. Drawdown -17.7% -17.5% -22.7% -21.9% -30.9% -26.5%
Daily VaR95% -1.8% -1.7% -1.9% -1.8% -1.9% -2.0%
Daily Turnover 9.6% 12.3% 19.7% 22.0% 14.7% 15.9%
α against S&P 500 0.05 0.07 0.04 0.07 -0.01 0.01 β against S&P 500 0.66 0.59 0.89 0.81 0.86 0.89
Start Date 07-01-2014
End Date 31-12-2019
Total Months 74
1. Returns: Cumulative- and Annual returns appears significantly higher for the XGBoost strategy with HRP.
2. Variance: Annual volatility, Max. Drawdown and Daily VaR95%are all lower for the Momentum strategy with HRP.
3. Performance Criteria: However, despite the lower variance observed in the Momentum strat-egy with HRP, the Sortino- and Sharpe-ratios are higher for the XGBoost stratstrat-egy with HRP, implying that the higher returns offset the higher volatility.
4. Cost: In this metric, the Momentum strategy wins out on a large margin.
5. CAPM: Here we observe a similarly high α between the Momentum and XGBoost strategies with HRP, though the Momentum strategy employs a lower β.
While these measurements suggest a close competition between XGBoost and Momentum, LSTM is left in the dust, losing to the two ”superior” strategies in every metric. This is an unfortunate result.
We would have liked to see the LSTM strategy perform much better. However, this is not the case.
With focus on returns, we see once again that XGBoost returns are much higher than the two other strategies. In addition, Sharpe- and Sortino ratio is also higher. This indicates that, even though XGBoost is more volatile than momentum, the greater returns offset this volatility.
Regarding the daily turnover, we observed that it is much lower for the momentum strategy, this is further confirmation of what we mentioned earlier, in that the momentum strategy has periods where it does not trade at all. In addition, the momentum strategy has strict buy and sell criteria; the fact that the S&P 500 must be above its 200-day MA and that the individual stocks must be above their 100-day MA definitely impacts the daily turnover by a great amount.
Momentum and XGBoost also generates the same excess return (orα) at 0.07 against the S&P 500, even though their approaches are completely different. Only the XGBoost is highlighted as being the highest, because if we consider more decimals, the α for momentum HRP is 0.0666 and for XGBoost HRP it is 0.0672, so they are not actually the same.
Another interesting observation is that both XGBoost and LSTM have betas which are approximately 0.81 and 0.89, whereas momentum has substantially lower betas of approximately 0.6. This could be partially explained by the fact that on multiple occasions momentum lies entirely uncorrelated with the market holding no assets at all, in comparison the XGBoost and LSTM strategies which are constantly exposed to the market.
Furthermore, common to all of the strategies is that they have non-zero positive betas, indicating that each strategy is correlated with the market. Since all strategies are long only, we are not overly surprised by the positive loadings.
To get a slightly more life-like depiction of the performance of these strategies, we set a starting portfolio value before backtesting to $10,000 and the portfolio value after backtesting for all individual strategies can be seen in Table 6 below:
Table 6: Portfolio end values
HRP 1/N
Strategy Momentum XGBoost LSTM Momentum XGBoost LSTM
Portfolio end value $20,278 $23,200 $17,559 $19,074 $20,990 $15,150 To little surprise, the XGBoost strategy has performed better with regards to the metrics included in this section. We will now explore if the three strategies are able to work together in ensemble and beat the XGBoost strategy.
8.5 Ensemble Strategies
In this section we will explore the results of our 3 ensemble strategies as defined by:
• 1/3: Stock will be bought if a buy signal is present from 1 or more of the individual strategies.
• 2/3: Stock will be bought if a buy signal is present from 2 or more of the individual strategies.
• 3/3: Stock will be bought if a buy signal is present from all of the individual strategies.
Each ensemble strategy has been evaluated with a naive (1/N) diversification as well as a HRP diversification like the previous strategies. Similarly to the individual strategies, we observed that the HRP diversification outperforms the naive on almost every metric, so we keep our focus on looking at results with this allocation method. The cumulative returns plot for the ensemble strategies using HRP can be seen below in Figure 51:
Figure 51: Comparison of the best methods (HRP) for each ensemble strategy
We notice how 1/3 and 2/3 seem to behave in a similar fashion, with 2/3 outperforming 1/3 in a purely returns perspective. However, 3/3 is noticed to differ significantly from the others. This is primarily due to the enforcement of a buy-signal being present from the Momentum strategy, which is
evident from the periods in Figure 51 where the cumulative return for3/3is flat, i.e. times where the momentum strategy is prevented from buying. Additionally, while 2/3 is observed having a higher cumulative return, it doesn’t seem to exhibit higher volatility when compared to1/3, which indicates a higher Sharpe and Sortino ratio. Although the 3/3 strategy looks worse than the others, we can see that it is able to reduce losses at some periods. Specifically in late 2018. However, it also ”misses”
the entire upside in late 2015 and the beginning of 2019.
In the same manner as the individual strategies, we analyse the rolling volatility as well as the top 5 drawdown periods. Though for the sake of simplicity, we only reference the plots for the2/3strategy using HRP in Figure 52.
Figure 52: Rolling vol and top 5 drawdown periods for ensemble strategy 2/3 with HRP
The first thing to notice is that the volatility resembles the volatility of the benchmark, though it is generally higher from the beginning up until 2018, where it continues to lie slightly below the bench-mark volatility. In our analysis of the top 5 drawdown periods, we notice the worst being from early 2015 to mid 2016 in which the max. drawdown reaches -18.3%. The second biggest drawdown for the 2/3 ensemble is similar to the drawdown period for XGBoost. They both range form late 2018 and re-cover in 2019. However, 2/3 ensemble has a lower max. drawdown and is able to rere-cover 36 days faster.
Next, let us consider the actual performance metrics of the 3 ensemble strategies in Table 7:
Table 7: Ensemble performance
Strategy 1/3 2/3 3/3
Asset Allocation 1/N HRP 1/N HRP 1/N HRP
Cumulative Returns 92.2% 106.0% 129.7% 147.5% 49.0% 62.7%
Annual Returns 11.1% 12.4% 14.3% 15.7% 6.6% 8.2%
Annual Volatility 16.3% 14.8% 14.3% 14.0% 11.3% 12.4%
Sharpe Ratio 0.73 0.86 1.01 1.12 0.63 0.69
Sortino Ratio 1.05 1.25 1.44 1.60 0.86 0.98
Max. Drawdown -25.5% -23.4% -16.8% -18.3% -18.9% -21.6%
Daily VaR95% -2.0% -1.8% -1.8% -1.7% -1.4% -1.5%
Daily Turnover 13.8% 17.0% 15.9% 18.1% 21.2% 21.9%
α against S&P 500 0.02 0.04 0.06 0.08 0.03 0.04 β against S&P 500 0.99 0.87 0.88 0.83 0.47 0.48
Start Date 07-01-2014
End Date 31-12-2019
Total Months 74
In line with the visual interpretation, we see that the HRP diversification dominates across all ensemble strategies in most metrics. Additionally, the 2/3 strategy seems to dominate the other strategies.
1. Returns: Cumulative- and Annual returns appears significantly higher for the 2/3 strategy with HRP.
2. Variance: Annual volatility and Daily VaR95% are all lower for HRP with the exception of the 3/3 strategy in which the 1/N diversification wins out. However, the lowest max. drawdown is for Naive 2/3.
3. Performance Criteria: Though as a result of greater returns and lower variance, the Sortino-and Sharpe-ratios are higher for HRP across the strategies, with 2/3 winning out.
4. Cost: Similarly to the individual strategies, the daily turnover is higher for HRP.
5. CAPM: Similarly to the individual strategies we observe a higherαcompared to the benchmark for the HRP enabled strategies.
The results from this table is similar to the individual strategies, where HRP generally outperforms with high returns and Sharpe/Sortino ratios.
In addition to finding the daily 95%-VaR for the 2/3 strategy, we have also found the expected short-fall. It is simply the mean of returns that exceed the 95%-VaR threshold. We find it to be -2.4%, indicating that the daily expected losses, when exceeding the 95%-VaR, is -2.4%. In comparison, the XGBoost expected shortfall is -2.6%. We then expect lower extreme losses in the 2/3 ensemble.
In an effort to gauge whether any 2-strategy combination of our individual strategies were solely re-sponsible for the performance of our2/3 ensemble strategy, we ran backtests on all combinations of
the three strategies; [”Momentum + XGBoost”], [”Momentum + LSTM”] and [”XGBoost + LSTM”], and found that none of the combinations performed as well as the ensemble strategy. Thus we can assume that all 3 of the individual strategies contribute in a significant manner to the overall perfor-mance of the ensemble strategy.
It is evident that the 2/3 strategy dominates the others. Lets see how it compares to the ”best” of the individual strategies; XGBoost in Figure 53:
Figure 53: Comparison of XGBoost and 2/3 Ensemble with HRP
We notice that the movement in the cumulative returns plot seems identical, though with the 2/3 ensemble outperforming XGBoost. Focusing on december 2017 - february 2018 we notice that XG-Boost decreases in value whereas the ensemble continues to rise. This is a great example of what ensemble learning brings to the table i.e. in that period of time the ensemble sets aside the predictions of XGBoost, to utilize the strengths of the other strategies. Furthermore, we can see from figure 50 that both LSTM and Momentum performs extremely well in that period.
To expand the analysis we have calculated a full performance metric comparison which can be seen in table 8:
Table 8: Performance of XGBoost and 2/3 Ensemble
Strategy XGBoost 2/3
Asset Allocation 1/N HRP 1/N HRP
Cumulative Returns 109.9% 132.0% 129.7% 147.5%
Annual Returns 12.7% 14.5% 14.3% 15.7%
Annual Volatility 15.6% 14.7% 14.3% 14.0%
Sharpe Ratio 0.85 1.00 1.01 1.12
Sortino Ratio 1.21 1.43 1.44 1.60
Max. Drawdown -22.7% -21.9% -16.8% -18.3%
Daily VaR95% -1.9% -1.8% -1.8% -1.7%
Daily Turnover 19.7% 22.0% 15.9% 18.1%
α against S&P 500 0.04 0.07 0.06 0.08 β against S&P 500 0.89 0.81 0.88 0.83
Start Date 07-01-2014
End Date 31-12-2019
Total Months 74
One might be tempted to say that the results speak for themselves but let us venture a little bit further:
1. Returns: Cumulative- and Annual returns are significantly higher for the 2/3 ensemble strategy.
2. Variance: Annual volatility, Max. Drawdown and Daily VaR95% are all lower for the 2/3 ensemble strategy.
3. Performance Criteria: The Sortino- and Sharpe-ratios are all higher for the 2/3 ensemble strategy, with even the 2/3 with 1/N diversification beating out the XGBoost with HRP strategy.
4. Cost: The daily turnover is also significantly lower for the 2/3 ensemble strategy.
5. CAPM: The 2/3 ensemble with HRP exhibits the highest α recorded compared to the bench-mark.
Looking at the CAPM results, we see that the 2/3 strategy exhibit higher α and higher β than XGBoost. This means that 2/3 ensemble actually generates more alpha, as its loading towards the market is higher. Furthermore, we notice that the drawdown for the ensemble strategy is substantially better than that of the XGBoost implementation, indicating a more stable strategy. In addition to being more stable, we also see that the ensemble produces substantially larger returns. Conclusively, the 2/3 ensemble outperforms the XGBoost strategy in all means.