Using synthetic data or external data sources in QuantTrader

[Alex @ Logical Invest]

Currently there are discussions on how to use synthetic data to extend backtests, or how to use external datasources, while avoiding QuantTrader to overwrite it. To channel all these discussions, here a new topic and a short descriptions. The topic is a bit geeky and using synthetic data always has some risks, so use at your own precaution.

In the example I want to extend the Maximum Yield Rotation Strategy beyond the inception date of the ZIV and TMF tickers.

• Open this Dropbox link in your browser: https://www.dropbox.com/s/jk3ortdyru4sg4n/ZIVlong.TXT
• Download as file into your QT folder and rename from “TXT” to “_ZIVSyn.csv”. Explanation of the file naming: “_” instructs QT that this is a symbol, not Strategy. The “Syn” is to prevent QT to overwrite the file with the next data download, as “ZIVSyn” is not a valid Symbol QT will ignore it. Next we need to manipulate the file a bit manually in Excel:
  1. Sort by descending date order, e.g. last date on Top.
  2. Add a column F named “Volume” and fill all rows with 1.
  3. Copy column E “Close”, paste into Column F and Rename “Adj Close”
• For the synthetic TMF symbols we use the TLT returns and multiply them by 2.8, which is approximately the factor after time decay (See article about 3x UIS “Hell on fire”). Attached an Excel file so you can track the modifications I made to create the _TMFSyn.csv.
• Open QT and create new symbols “ZIVSyn” and “TMFSyn” in the Symbol Manager. QT will now recognize the symbol and use the prices from the csv file, any update from Yahoo will fail, thus they are not overwritten – but also not updated..

As an example we´ll now extend the MYRS backtest:

• Open Portfolio Manager and select the “MYRS ZIV-TMF”
• Copy / Duplicate the strategy and rename into “MYRS ZIV-TMF – Synthetic”
• Delete ZIV and TMNF in the used symbols, and replace it by our synthetic “ZIVSyn” and “TMFSyn”
• Save the strategy

Et voila, here is our extended Backtest. I used the current parameters to avoid any curve-fitting discussion, you can test by your own.

Btw, you can use the trick with the “non-valid Tickers” also for other purposes. For example if you have futures data or ETF/Stock data from other data sources, just create some artificial file names and the needed file structure and you can use external data sources.

[caption id="attachment_39853" align="aligncenter" width="500"] Extended Maximum Yield Rotation Strategy Backtest (Click to enlarge)[/caption]

Attached also the synthetic tickers which you can simply copy into your QT folder to eliminate some of above steps. The Excel file shows the calculations for the TLT manipulation.

_ZIVSyn _TMFsyn _TMFsyn Excel

[Caleb Mock]

Thanks Alexander! It’s not geeky at all.

[Dan Yaklin]

In looking at your spreadsheet, I would suggest using actual data as far back as possible and use synthetic data prior to the earliest actual data.

[Alex @ Logical Invest]

Yes, fully agree. It was just easier as an example to start at the earliest data as base. Actually best in this case would be to use actual TMF as far as possible, and then plug the 3x TLT below, just a bit more complex to implement.

[Tom Gnade]

I just tested these synthetic price files in the latest version of QT and they work fine. The ZIV data is from Alex’s file above, just merged with up-to-date actuals. The TMF data is computed from -2.8x daily TLT returns starting from 4/15/2009.

https://drive.google.com/drive/folders/0B0pvnD9e2gFva1NNQzVrQzFEQ3c?usp=sharing

[Tom Gnade]

I have now also added SPXL and TQQQ synthetic data, generated from SPY and QQQ, respectively. I looked at the relationship of the two over some time and they do seem to track pretty closely to 3x, so I used that value instead of 2.8x as in the TMF calculations. I’ll keep adding more synthetic data as I try to do longer-term analysis of other strategies. For now, I have:

ZIV
TMF
SPXL
TQQQ

and these 4 enable analysis back to 2004 for the main growth driver strategies MYRS, Nasdaq 100, UIS 3x, and perhaps a few others.

[Derrick]

I just downloaded the latest QuantTrader with updated leveraged strategies that now use UGLD. The 3x gold ETN only goes back to 2012 or so. Has anyone found or generated synthetic data to extend the backtests to 2004? Thanks.

[Alex @ Logical Invest]

Hi Derrick,

you can probably use GLD data, multiply daily returns by 2.8 and then calculate prices backwards using the last available UGLD price, see TMF/TLT example I´ve posted in Excel. Have not done this for UGLD yet, so to be sure also calculate forward to see how good it matches. Bit of fiddling around in Excel, but works fine – we´re looking into how to automate this further.

Pls share your findings with the crowd. :-)

[Tom Gnade]

Synthetic Prices

I just added UGLD_SYN to the collection.

Just add recent data to the others to bring them up-to-date. I’m not having the best of luck getting Quant Trader to recognize synthetic prices, it’s very sporadic. It would be nice if there were a built-in feature that would allow synthetic prices for leveraged funds based upon the underlying – i.e. allow a checkbox in symbol manager “use synthetic price history” that would download all available actual data, and then designate a related symbol – say GLD for UGLD, and a leverage multiplier (3x), and QT could create 2 files, one would be _UGLD.csv, the other would be _UGLD_SYN.csv. Both files would be used to construct backtests for any strategy that used those symbols, but only if the portfolio were configured in Portfolio Manager to “use synthetic prices when available”.

Thoughts?

[Tom Gnade]

FYI I’ve tried many times to get synthetic pricing to work and it just doesn’t. It’s one of those things that really could use some attention in the program, just my 2 cents. I gave up again tonight =(

[Author]

Posts