Are Backtested Results Effective?

One of the key factors influencing the success of raising capital for a new investment strategy is what is known as a “backtest.” When an investment strategy has little to no auditable track record (i.e. the strategy is new or hasn’t launched yet), firms looking to raise capital for an algorithmic or quantitative strategy tend to rely heavily on the backtested results, which compose the hypothetical performance history of the strategy in question. This article from Bloomberg highlights some of the challenges with relying solely on backtested results to determine the suitability of a strategy for an investment portfolio. Often times, backtested results look nothing like realized performance when the strategy is “live” in the real world. Because backtesting is a statistical process, a strategist or researcher can try a large enough combination of strategy configurations in which one is likely to fit the desired performance; this is known as “overfitting.”

While the article from Bloomberg focuses on how insurance companies use backtests to market insurance products with investment components to retail investors, the underlying concerns also apply to sophisticated asset allocators who are evaluating prospective investment managers. Though not exhaustive by any means, requiring that backtests be of a minimum length (enough to incorporate more than one market cycle) and include a sufficient out of sample testing period (data the model has not been trained or “overfit” on) are two necessities that will help separate the wheat from the chaff in the evaluation process.