Backtest vs Live Performance

Home Forums Logical Invest Forum Backtest vs Live Performance

Viewing 9 posts - 1 through 9 (of 9 total)
  • Author
    Posts
  • #78554

    Hi Chaps,

    How are you?

    Does anyone have feedback on live results vs the backtests? The models look excellent but I am assuming that everything shown is the backtested strat and:
    a) Does not account for execution fees or slippage (understand this is different for everyone but it can make a large difference).
    b) When a strategy is updated, like the Bond Rotation, the website shows the new strategy, and not how the strategy performed in its old format and hence how we would have traded it live at the time?

    Also interested in users feedback if anyone would be so kind to comment? I was to setup the Enhanced Perm Portfolio or Min Drawdown 10%.

    Thanks a lot. James

    #78638
    Adrian
    Participant

    Hi James.

    Even if I heard of LI a long time ago, it’s only recently that I start looking into it more carefully.
    Based on what I saw skimming different forum threads, the lack of skeptics, of people trying to find the biggest possible hammer to hit the strategies in order to crack them in their most vulnerable place, the multitude of strategies (which could be a positive for some people, but I am not in that camp; if you provide 20 strategies, of course at the end of each year you’ll have 3 which performed well), the constant re-modelling (some would say “improvement”, others would say “hindsight abuse”) of the strategies, the permanent pursuit of our fellow forum members to use some software to optimize the combination of multiple LI strategies (which is too much of a curve-fitting for me) – all these are troublesome in my view.
    You say that “models look excellent”. That is a bit debatable.
    Since the rules of strategies are sometimes changed, when you evaluate a strategy is not very obvious for you when the last changed was done and how much historical performance of that strategy happened before and after the last update. You can look at 15 years worth of strategy performance, but if last strategy update was 9 months ago, you only have 9 months of out-of-sample performance.
    Secondly, take a look at Dow 30 Strategy between march 2009 – mar 2020, US Sector Rotation Strategy between aug 2002 – feb 2020, World Top 4 Strategy between aug 2011 – feb 2020, NASDAQ 100 Balanced unhedged between dec 2008 – may 2020, NASDAQ 100 Low Volatility Strategy nov 2008 – may 2020 – neither of them outperformed (I mean in terms of returns, they may be done better in terms of volatility) their benchmark. You may say I cherry picked the intervals, which of course I did. But if a strategy has an edge over buy and hold, I would say there should be such a thing as a long enough period of time that no matter how selected from the entire history of the strategy would show an outperformance over the benchmark. Isn’t 18 years long enough for US Sector Rotation to beat S&P 500? Or 12 years for NASDAQ 100 Balanced to beat QQQ?

    Just to be clear, I am not someone eager to inform the world that LI strategies don’t work. I am just more circumspect in my investing decisions and I know that, assuming it exists, a market edge is very difficult to find.

    Happy to get some feedback from others.

    #78644

    Hi Adrian,

    A lot of valid points above. I especially agree with curve fitting using backtested data albeit I would disagree on performance and vol. Achieving equity returns with lower vol is a far superior return profile and does enable one to then leverage up the portfolio to match equity vol and theoretically achieve higher returns. a la one of the theories behind risk parity.

    Frank/ Vangelis – Would you care to shed some light? Again the product looks very interesting and has so much potential but I am trying to ascertain what performance has actually been like for the strategies LIVE. Where can we see such data? The historical data pages are confusing in that they only show data forward of early 2019.

    Keen to hear feedback. James

    #78646

    After the Oct.-Dec. 2018 correction we did quite an important change to the strategies. Instead of letting the strategies freely allocate in the best ETFs we in fact imposed a 40%-60% hedge composed of ETFs with a negative correlation to equity. We did this as more and more corrections came out of nothing during a 100% bull market, so that there was no time to wait for the next rebalance date to include a hedging ETF (Treasury or Gold). We did this for all strategies but not for BRS and BUG as they did do very well since years and we did not want to change a winning team. We should have also done it for these two, as they did not have time to switch early enough during the Corona crash.
    So if you want to know how a strategy would have really done in the past, then you can look back to Jan. 2019 but not longer. BRS and BUG are new updated strategies (see blog). Before 2019 it is just backtesting telling you how the actual strategy would have done during previous corrections. It would made no sense to have a lookback based on the previous versions of the strategies which could go 100% long. For the 3x leveraged UIS strategy we lowered volatility for 2020, which resulted in a 30% lower return for 2019 in the chart (real return was 80% now it shows 50%). This shows also that in fact pure return is for us less important as to reduce drawdowns.
    All modern “always hedged” strategies did very well during the 2019 bull market and also during the corona crash. This was really a sort of successfully passed general trial and it is also a good sign that also going forward the probability of a good return at reduced risk could be achieved. We all are personally invested in our strategies and we do everything to reduce risk while still getting a good return.

    #78650
    Vangelis
    Keymaster

    There are very valuable and correct points being made in this discussion. Let me try to show our point of view:

    The main purpose of developing these strategies was to cut down on risk and protect accounts from large drawdowns. LI was really born out of the 2008 crisis. Outperforming the SP500 was never our intention. But as James mentioned if someones wants to outperform they are welcome to create a portfolio using LI tools and then leverage it up, taking advantage of lower drawdown limits.

    We have 3 goals we try to achieve:
    a. Diversify: From individual stocks to ETFs, from ETFs to strategies-of-ETFs and from strategies of ETFs to a portfolio-of-different-strategies. What we could call a level-3 diversification.
    b. Limit risk: Cut down large drawdowns as to gain from long-term from growth. This means keep intelligent hedges that don’t cost money such as put options or (long) VIX ETFs and futures.
    c. Make it simple: Make complex rules-based models ‘investable’ and not just theoretically possible. This is a big deal since we do have complex option and futures-based strategies that we can (and have) build. But it makes no sense unless someone can actually realistically trade it, for the long run. So we tried to make everything come down to a handful of ETFs that one has to buy/sell once a month.

    Other than that we do not pretend to be very smart (except Frank who is actually a really a smart trader) or to create alpha out of nowhere. There is nothing wrong in just holding DIA or QQQ or a 60/40 SPY/TLT. The problem is that investors with no rules end up being aggravated by emotions and end up losing even though markets go up (for the most part) in the long term. So we provide some rules to help.

    You can see some actual historical returns by going back to our published end of year newsletters:

    https://logical-invest.com/the-logical-invest-newsletter-for-january-2020/
    https://logical-invest.com/the-logical-invest-mid-month-newsletter-for-january-2019/
    https://logical-invest.com/the-logical-invest-monthly-newsletter-for-january-2018/
    https://logical-invest.com/update-january-2017-401k-investments/
    https://logical-invest.com/the-logical-invest-monthly-newsletter-for-january-2016/

    #78659

    Hi chaps,

    Appreciate the response and transparency. I agree that equity index returns with lower volatility are a reasonable goal for all investors. It is hard to outperform in asset allocation without dialing up risk which is not my aim. Slow and steady please.

    James

    #78665
    Mark Vincent
    Participant

    Hello James here are my thoughts,

    Trading for about a year very pleased. Slippage trading costs are not an issue. I can’t comment on the portfolio you mentioned my favorite is the Top 3. The results are very close to what is published.

    As long as risk parity and uncorrelated assets hold up we are good. They did not work well in the 80’s will they work in the future do not know. Risk parity works now and probably to well. Here are a few thoughts:

    If you traded a model you should achieve close the results during that given year. Just check out the year end results they will be close. BRS was a disappointment but black swan events are part of investing. Everything else broke down but most of LI systems did not have as much drawdown as SPY.

    The models do change and it would be nice if they showed the real results with simulated and the point in time where the model changed other websites do this but there are pros and con’s to both. You could download previous version of LI and string them together but I don’t care.

    Time horizon I think is very important when evaluating a system. 5 years vs 20 years you would want very different risk models but maybe not different assets. The models are trying to give you an edge there is no free ride but I would be willing to bet over 5 years I will sleep better with LI than without.

    Understand that by controlling volatility you are increasing your chance of success. That is proven. You can read about it on Websites like Resolve asset management. They prove this methodology but they are much more conservative. Which killed their returns for the last 10 years. They are also globally diversified on every asset class which hurts them over the last 10 years since the US markets are so dominate.

    I also like LI since the only input is price. You can’t hide behind price. Other sites that use fundamentals they tend to break down quickly and you need to pivot all the time.

    Picking an LI model for the first time is confusing there is a lot of choice but after following a few for a couple months you will understand them better. Most models have a common theme. It would take you a couple months to learn them all and another couple of months to learn all the advanced boundaries.

    LI updates the models this is great I don’t have to. They are changing as the market changes and that is very important. The current trend is more hedging. Some would call this curve fitting but as long as the changes achieve better risk adjusted returns in the future I am good with it.

    Building your own models yes you need a good Hypothesis mine is technology rules the US Economy and now the world that would be FAANG +MS. Imagine if they disappeared went to zero or even 50% decline? I stated this back in 2009 on other forums and no event yet has proven me wrong. That being said if I am retiring next year I would not want a large tech component because of volatility. And LI is perfect for my hypothesis and Hedging you just need to figure out your time horizon and risk.

    Back testing is good but OS is all that matters. You need to have a hypothesis on what is going to happen to give you an edge. Be conservative only beat the market by 1% over 10 years you are better than 95% of mutual fund managers. LI can help you do that.

    Hope that gives you some perspective,
    Cheers,
    Mark V.

    #78763
    iantest1
    Participant

    Has the chart here been updated to reflect the recent changes made?

    https://logical-invest.com/app/portfolios

    Thanks!

    #78776
    Patrick
    Keymaster

    Yes, the chart is based on backtests for the past 5 years and does reflect the recent updates to the BRS and BUG strategies.

Viewing 9 posts - 1 through 9 (of 9 total)
  • You must be logged in to reply to this topic.