Small Sample Size Bias

Home » Topics » FAQ » Small Sample Size Bias
Small Sample Size Bias 2017-03-03T15:26:55+00:00
  • Author
    Posts
  • broker
    Participant
    Post count: 1

    Although I appreciate your hard work, I am concerned with the limited time frame that your backtest cover.
    Going back to 2008 is not robust. You are using a small sample and arbitrarily projecting that into the future.

    I have seen several trend following studies with data going back to the 1960’s or 1970’s. That data is out there and I think you owe it to your paying membership to do the hard work of connecting ETF data to prior fund or index data that exists.

    If you do this, then you will provide strategies that will have a greater probability of success going forward.

  • Alexander Horn
    Keymaster
    Post count: 374

    Thanks for the open feedback!

    There are two perspectives when it comes to backtests. I generally agree that the underlying ‘base’ mechanism needs to be backtested during different economic environments, e.g. high/low inflation, interests, equity growth. This normally requires going back at least into the early 90s.

    However, and as we’ve argued in several places already, just trying to backtest everything since the 60’/70’s also holds two hickups in our opinion:

    – The financial world has undergone fundamental changes in the recent years, and some opportunities like the ‘volatility premium’ harvesting due to VIX contango just did not exist before 2008/2009. So any attempt to validate this in a longer backtest would lead to ignoring this opportunity, and by default invalidate the MYRS and GMRSE.

    – Some ETF we employ are based on relativly new indices (inflation protected or volatility based ETF), so backtests are just technically limited, as no adecuate proxies like indices or mutual funds are available. Again here you might miss the opportunity if seeking long backtests.

    We’re about to publish articles including 20 years backtests for the Bond Rotation and Global Market rotation, and assure you it is not the additional work that has hold us back from doing so before.

You must be logged in to reply to this topic.