Using synthetic data or external data sources in QuantTrader

Home » Topics » QuantTrader Forum » Using synthetic data or external data sources in QuantTrader
Using synthetic data or external data sources in QuantTrader 2017-03-28T21:16:03+00:00
  • Author
    Posts
  • Alexander Horn
    Keymaster
    Post count: 374

    Currently there are discussions on how to use synthetic data to extend backtests, or how to use external datasources, while avoiding QuantTrader to overwrite it. To channel all these discussions, here a new topic and a short descriptions. The topic is a bit geeky and using synthetic data always has some risks, so use at your own precaution.

    In the example I want to extend the Maximum Yield Rotation Strategy beyond the inception date of the ZIV and TMF tickers.

    • Open this Dropbox link in your browser: https://www.dropbox.com/s/jk3ortdyru4sg4n/ZIVlong.TXT
    • Download as file into your QT folder and rename from “TXT” to “_ZIVSyn.csv”. Explanation of the file naming: “_” instructs QT that this is a symbol, not Strategy. The “Syn” is to prevent QT to overwrite the file with the next data download, as “ZIVSyn” is not a valid Symbol QT will ignore it. Next we need to manipulate the file a bit manually in Excel:
      1. Sort by descending date order, e.g. last date on Top.
      2. Add a column F named “Volume” and fill all rows with 1.
      3. Copy column E “Close”, paste into Column F and Rename “Adj Close”
    • For the synthetic TMF symbols we use the TLT returns and multiply them by 2.8, which is approximately the factor after time decay (See article about 3x UIS “Hell on fire”). Attached an Excel file so you can track the modifications I made to create the _TMFSyn.csv.
    • Open QT and create new symbols “ZIVSyn” and “TMFSyn” in the Symbol Manager. QT will now recognize the symbol and use the prices from the csv file, any update from Yahoo will fail, thus they are not overwritten – but also not updated..

    As an example we´ll now extend the MYRS backtest:

    • Open Portfolio Manager and select the “MYRS ZIV-TMF”
    • Copy / Duplicate the strategy and rename into “MYRS ZIV-TMF – Synthetic”
    • Delete ZIV and TMNF in the used symbols, and replace it by our synthetic “ZIVSyn” and “TMFSyn”
    • Save the strategy

    Et voila, here is our extended Backtest. I used the current parameters to avoid any curve-fitting discussion, you can test by your own.

    Btw, you can use the trick with the “non-valid Tickers” also for other purposes. For example if you have futures data or ETF/Stock data from other data sources, just create some artificial file names and the needed file structure and you can use external data sources.

    Extended Maximum Yield Rotation Strategy Backtest Investment Portfolio

    Extended Maximum Yield Rotation Strategy Backtest (Click to enlarge)

    Attached also the synthetic tickers which you can simply copy into your QT folder to eliminate some of above steps. The Excel file shows the calculations for the TLT manipulation.

    _ZIVSyn _TMFsyn _TMFsyn Excel

    Attachments:
  • Caleb Mock
    Participant
    Post count: 10

    Thanks Alexander! It’s not geeky at all.

  • Dan Yaklin
    Participant
    Post count: 1

    In looking at your spreadsheet, I would suggest using actual data as far back as possible and use synthetic data prior to the earliest actual data.

    • Alexander Horn
      Keymaster
      Post count: 374

      Yes, fully agree. It was just easier as an example to start at the earliest data as base. Actually best in this case would be to use actual TMF as far as possible, and then plug the 3x TLT below, just a bit more complex to implement.

  • Tom Gnade
    Participant
    Post count: 25

    I just tested these synthetic price files in the latest version of QT and they work fine. The ZIV data is from Alex’s file above, just merged with up-to-date actuals. The TMF data is computed from -2.8x daily TLT returns starting from 4/15/2009.

    https://drive.google.com/drive/folders/0B0pvnD9e2gFva1NNQzVrQzFEQ3c?usp=sharing

    1 user thanked author for this post.
    • Tom Gnade
      Participant
      Post count: 25

      I have now also added SPXL and TQQQ synthetic data, generated from SPY and QQQ, respectively. I looked at the relationship of the two over some time and they do seem to track pretty closely to 3x, so I used that value instead of 2.8x as in the TMF calculations. I’ll keep adding more synthetic data as I try to do longer-term analysis of other strategies. For now, I have:

      ZIV
      TMF
      SPXL
      TQQQ

      and these 4 enable analysis back to 2004 for the main growth driver strategies MYRS, Nasdaq 100, UIS 3x, and perhaps a few others.

      1 user thanked author for this post.
  • Derrick Scheid
    Participant
    Post count: 19

    I just downloaded the latest QuantTrader with updated leveraged strategies that now use UGLD. The 3x gold ETN only goes back to 2012 or so. Has anyone found or generated synthetic data to extend the backtests to 2004? Thanks.

    • Alexander Horn
      Keymaster
      Post count: 374

      Hi Derrick,

      you can probably use GLD data, multiply daily returns by 2.8 and then calculate prices backwards using the last available UGLD price, see TMF/TLT example I´ve posted in Excel. Have not done this for UGLD yet, so to be sure also calculate forward to see how good it matches. Bit of fiddling around in Excel, but works fine – we´re looking into how to automate this further.

      Pls share your findings with the crowd. 🙂

You must be logged in to reply to this topic.