The MetaTrader 4 Backtesting Blues – Track 1

The recent introduction by Alpari UK of "interbank spreads" on their MetaTrader 4 demo accounts has given us the perfect opportunity to demonstrate one of the potential problems you might encounter using the MetaTrader 4 strategy tester to evaluate a forex trading strategy you have developed for yourself, or even bought off a virtual shelf.  Our regular readers will know that we have performed numerous live tests and backtests on the Forex MegaDroid "robot". Today is a Saturday, so I had a few minutes spare to run yet another backtest on the Droid. I started the simulation on March 30th 2009, the date the Forex MegaDroid team launched their creation amid great fanfare, and finished it yesterday evening.  Here's what the Alpari UK MetaTrader 4 demo strategy tester showed me:

Forex MegaDroid weekend backtest on Alpari UK demo account

The bottom line? A net loss of $492.91 following a drawdown of 60.28%. Not very impressive, I'm sure you'll agree!

Here's the equity curve from another backtest that I performed yesterday evening, using exactly the same settings on exactly the same account:

Forex MegaDroid backtest on Alpari UK demo account with Pro spreads

The bottom line this time? A net profit of $36,468.75 following a drawdown of 65.78%. Rather more impressive, although still nowhere near as impressive as the backtested results shown on the Forex MegaDroid sales page. If you want a good laugh pop over there now. After 9 months they have finally removed their ridiculous claim about "No Losing Trade Since 20th March 2009!". It seems that at long last Albert and John do admit that their creation can lose money from time to time. Their own highly tuned version of the Droid logged a loss of $19,866.00 on December 3rd.

Getting back from the realms of fantasy to the realities of the MetaTrader 4 strategy tester.  It suffers from a number of limitations that become particularly obvious when you're testing expert advisors like the Droid that use relatively small profit targets.  The problem we're demonstrating today is that MT4 only keeps the bid price in it's store of historical data.  It doesn't store the ask price, so when it's doing a backtest it has to make a guess at the ask price for every tick. To do that it adds the spread to the bid price it has got safely stored away.  Of course it has no idea what the spread actually was for your particular broker at that particular time in the past, so it guesses that too. It uses the spread the MT4 client terminal is displaying at the time you press the start button on the strategy tester.  This might have made a bit of sense in the good old days when MetaTrader forex brokers offered fixed spreads, but in this modern age of floating spreads and straight through processing it means that MetaTrader 4 strategy tester backtest results for scalping strategies are not far short of useless.

I can tell you what spread the MetaTrader strategy tester used for the backtest I did earlier today because it's still displayed in my Market Watch window. EURUSD: 1.43897 – 1.43833 = 6.4 pips. Unfortunately the strategy tester doesn't tell me what its guess was for the spread when I performed the exact same backtest yesterday.  My best guess is somewhere between 0.5 pips and 1.4 pips, since that was what the Alpari UK demo spread was varying between at the time I ran that backtest.