Any out-of-sample results for more than a year?

Home Forums Logical Invest Forum Any out-of-sample results for more than a year?

This topic contains 23 replies, has 12 voices, and was last updated by  Howard 6 days, 7 hours ago.

Viewing 15 posts - 1 through 15 (of 24 total)
  • Author
    Posts
  • #62848

    zhigulin
    Participant

    Dear Frank, Alex and co.

    I came across LI and initially it looked very impressive, but then I noticed that you keep tweaking and changing strategies. Having strategies that work well ex-post is not difficult at all. Ex-ante is a totally different ball game and I see no evidence you’re playing it well. Moreover, I see evidence to the contrary. I’ve installed QuantTrader323S in order to test Morpheus strategy. And I see that not only Morpheus didn’t work out-of-sample since 2017, but NONE of your strategies worked! They ALL had Sharpe of 0.5 or lower over the last 2 years. Of course you kept tweaking them since 2017, but who are you fooling? All your current backtests and strategies are ex-post and can’t be relied upon as predictors of future returns whatsoever.

    I’ll be happy to consider any evidence that my understanding so far is incorrect. Can you show any out-of-sample results with good performance for more than a year?

    Thanks!
    Valentin Zhigulin, PhD, CFA

    #62984

    jpildis
    Participant

    As a new subscriber, I’m interested in a reply on this.

    #62985

    Frank Grossmann
    Participant

    I am not sure about the Morpheus strategy as this is a strategy developed by one of our users, but we do out of sample backtesting of strategies if they have enough historical data. Here is a short article describing OOS with our UIS strategy: https://logical-invest.com/about/out-of-sample-walk-forward-optimization/

    #62986

    Frank Grossmann
    Participant

    By the way strategies are never a static thing which can be made once and then is valid forever. Going forward it is extremely important that you adapt to changes in the market environment while keeping the character of the strategy. All our main strategies still do the same like at the beginning. The biggest change was that we tried to give the strategies more possibilities to hedge. In the past we relied only on Treasuries what was a good choice for a long time, however with rising rates and more and more interventions of the Fed and the Governement we had to broaden our choice of hedging assets. You will always have the situation that something what worked for long times in the past does not work anymore. Then our approach is to do changes to the strategy so that it works as good as possible in the past and and also under this new market conditions. This way you have a high chance, that it will also work in the future. The goal is, that we try to learn and not to do investment errors again. A normal investor normally makes investment errors over and over again. They lost big percentages of their money during every severe market correction but still I guess they are mostly 100% long the stock market and have already totally forgotten the correction of end 2018.

    #65441

    lottifab
    Participant

    Hi

    as a side question of this thread.
    ie. let’s take a specific strategy, ie. NAS100

    Given the new nature of this service, could you kindly let me know exactly what is the OOS month that is intaken in the Performance report as “start of OOS period”?

    So, starting from which month have you started to produce “fresh new signals” from the model /algorythm, and what period instead is taken as “in-sample” backtest?

    This question is also necessary given no algorytmh detail is reported, therefore nothing can be infered about how the rotation mechanism really works at each month on the Nasdaq100…..
    thanks!

    #65457

    marcel
    Participant

    Hi Frank,

    I definitely agree that improving the algorithms as market conditions evolve is a good thing. I’d propose you publish two additional things:

    a) Per strategy change log: You’ve been publishing (some) changes in the blog but it would be good to have this in a complete and structured form, including creation date of strategy –> pre-creation date time periods would be in sample

    b) Realisable results: Aggregate result of different versions of strategy always forward looking post change, i.e. assuming strategy creation t=0 years, period of n=2 years, one change after m=1 years. Aggregate results would be time weighted return of initial strategy between 0 & 1 years and of changed strategy between 1 & 2 years. This would be the result an investor would have realised would he have followed the strategy from the beginning onwards.

    This would then also allow for a comparison between optimised and realisable results.

    Best,
    Marcel

    #66099

    rikder
    Participant

    Good suggestion Marcel. I would be very interested in this as well.

    #66100

    lottifab
    Participant

    Me too!! thx

    #66198

    JP
    Participant

    I’m interested in the reply to this post.

    #66440

    Howard
    Participant

    I am a 2+yr subscriber. I think the strategy has been effective and money making within expectation but I agree we should be able to see actual historical performance attributed by older versions of the strategies. I think what Marcel outlined is very sensible. —>

    b) Realisable results: Aggregate result of different versions of strategy always forward looking post change, i.e. assuming strategy creation t=0 years, period of n=2 years, one change after m=1 years. Aggregate results would be time weighted return of initial strategy between 0 & 1 years and of changed strategy between 1 & 2 years. This would be the result an investor would have realised would he have followed the strategy from the beginning onwards.

    #66680

    robininni
    Participant

    So, when we look at the historical performance of one of the investment strategies or portfolios involving a mixture of strategies, is the complete historical returns shown each month all from the latest tweaking of the strategies?

    So, for example, looking at the Moderate Risk Portfolio, it gives monthly returns back to May of 2000. If the underlying strategies that make up the Moderate Risk Portfolio have been tweaked one or more times in the past (sounds like many/all strategies have), is the monthly return table showing right now on the Moderate Risk Portfolio page going back to May of 2000 the returns I would have gotten from the most CURRENT version of the strategies making up the Moderate Portfolio? Or are some monthly returns shown from older versions and when the strategies are tweaked and changed all return information is not overwritten based on how the newest version of strategies would have performed? Of course, if it is the latter, that would not be very accurate for the return history if some of the return information is based off of previous strategy versions.

    #66681

    lottifab
    Participant

    thanks Robininni!!!!!
    great question

    #66691

    rikder
    Participant

    The data on this page might help with some of these questions: https://logical-invest.com/app/historical.php Of course if you trade a combination of strategies you won’t be able to track actual historical signals because each update will change how the strategies interact.

    #67150

    Vangelis
    Keymaster

    We have been around since 2014 and used to run our charts based on historical data. Last year (2018) we created the new site and added tools to enable subscribers to build custom portfolios online. In order to do that we switched to backtested data. Maybe it has been a poor choice… Here is an article explaining the reason we switched :https://logical-invest.com/backtested-data-vs-historical-data/

    The historical data is still being tracked here, manually, per strategy: https://logical-invest.com/backtested-data-vs-historical-data/

    #67160

    marcel
    Participant

    Hi Vangelis,

    I still think the two proposals I posted above are sensible, I think this would add transparency. It would make your improvements to the strategies and resulting higher returns also more transparent. E.g. if I’m running a factory and I’m getting some new, more efficient machinery, (a) I would not report past factory output as if the new machine would have been there the whole time and (b) I would take credit for reporting the uplift in output with the new machine. Hope that analogy makes sense :-)

    I think it is important to separate the strategy optimisation task and the reporting task. By all means, use as much historic data to optimise a strategy if it helps future performance.

    Maybe you could think about implementing this for future strategy changes. Here again:

    a) Per strategy change log: You’ve been publishing (some) changes in the blog but it would be good to have this in a complete and structured form, including creation date of strategy –> pre-creation date time periods would be in sample

    b) Realisable results: Aggregate result of different versions of strategy always forward looking post change, i.e. assuming strategy creation t=0 years, period of n=2 years, one change after m=1 years. Aggregate results would be time weighted return of initial strategy between 0 & 1 years and of changed strategy between 1 & 2 years. This would be the result an investor would have realised would he have followed the strategy from the beginning onwards.

    Regards
    Marcel

Viewing 15 posts - 1 through 15 (of 24 total)

You must be logged in to reply to this topic.