Backtest a trading rule

Test a moving-average rule on real prices, then see how easy it is to fool yourself by overfitting.

Asset

Rule

Moving average10 mo

When out, cash earns

Cost per switch0.10%

Trading fees, spreads and taxes, simplified.

Yearly growth, rule8.9%Buy and hold 11.2%

Sharpe ratio, rule0.68Buy and hold 0.65

Worst fall, rule-25%Buy and hold -51%

Time invested80%32 switches

Distance below the previous peak

Holding S&P 500 only when its price above its 10-month average, $10,000 from Aug 2006 became $55,212, versus $84,514 for simply holding. It finished behind, with a worst fall of 25% instead of 51%. Trend rules usually trade some return for smaller crashes, and they get whipsawed in choppy markets.

The overfitting trap

Try every fast and slow average on the early years, pick the best, then run it on years it has never seen. The best backtest is usually the luckiest one, not the smartest.

In-sample Sharpe ratio, Aug 2006 to Sep 2016

Tap a cell to use it above

2468101214161820

Rows: fast average (1 means the price itself). Columns: slow average, in months.Outlined: best in-sample

In-sample endsSep 2016

Best in-sample: fast 1, slow 80.94Buy and hold in-sample: 0.40

Same rule, out of sample0.82Buy and hold out of sample: 0.90

Median rule, out of sample0.84A typical, unoptimized choice

Testing 145 rules and keeping the winner is a search for luck. If the winner does no better than an average rule on new data, the backtest was fitting noise.

Uses monthly prices with dividends reinvested. Past performance does not predict future results, and this is a teaching tool, not a recommendation to use any trading rule.

Try this

Run the 10-month rule on the S&P 500. How much of the 2008 crash did it sidestep, and what did it miss in the rebound?
Raise the cost per switch to 0.5% with a 2-month average. Watch whipsaw trades eat the edge.
Move the in-sample end date. Does the best combination stay the same, or jump around? Stable is a better sign.

The math

MA_n(t) = \frac{1}{n}\sum_{i=0}^{n-1} P_{t-i}, \qquad \text{Sharpe} = \frac{\bar{r} - \bar{r}_f}{\sigma_r}\sqrt{12}

$MA_n(t)$: n-month moving average of the price
$P_t$: price at the end of month t
$\bar{r}$: average monthly return of the rule
$\bar{r}_f$: average monthly cash return
$\sigma_r$: standard deviation of monthly returns

Ask the Loop about this Next: Time value of money