The Forex MegaDroid Backtesting Blues – Track 3

An essential part of evaluating any trading system, whether you developed it yourself or bought it over the internet, is to discover how robust your chosen system is to changes in the system input parameters. Unfortunately the developers of Forex MegaDroid don't give their users the ability to alter the parameters that control their robot's "artificial intelligence". We assume there must be a few hidden somewhere inside the Droid's electronic brain, but there is no way for us to twiddle them with our virtual screwdriver. We assume that Albert and John spent many months tuning their creation to perfection. They have, however, made it very difficult for us to try and work out if they have over-optimised their robot's mega-mind. Maybe it will suffer a virtual epileptic fit one day soon? We have no way of knowing.

One input setting that is available to us is called GMT Offset. It seems as though the expert advisor is very choosy about exactly when it will trade, and it needs to keep track of time accurately. One of our readers even seems to think we didn't have this number set right in one of our backtests. Since this is the only parameter available to us that might affect entries and exits, rather than money management, we tried a few different values. Maybe if we got that number right our backtest results would suddenly look more like the MegaDroid team's? Their manual even goes so far as to say:

If you find that test results with your broker are poor then try adding or subtracting 1 hour i.e. if the chart GmtOffset shows 3 then try 2 or 4.

We did just that, still using our demo MetaTrader 4 account with FXDD, as recommended in the MegaDroid manual. This time we ran our tests over the second quarter of 2009. It was after the end of March that our results began to look different to the MegaDroid team's. Maybe we or the robot were getting confused by Daylight Saving Time?

This is the equity curve that the FXDD MetaTrader 4.00 build 224 strategy tester produced for us using a GMT Offset setting of +3, as recommended in the manual:

This shows the by now familiar drawdown near the end May.  Then we tried increasing the GMT Offset to +4.  This is what the MetaTrader strategy tester showed us:

Now this doesn't really look like an improvement to me.  There seem to be many more drawdowns, and the equity curve still doesn't look anything like the one proudly displayed by the Forex MegaDroid team. We pressed on and tried reducing the GMT Offset to +2.  This is what the strategy tester showed us:

That is definitely not an improvement! It still looks nothing like the equity curve on the Forex MegaDroid site, and that drawdown at the end of the period looks pretty terrifying.  Here are those trades in more detail:

Three losers in a row for a total loss of $32,299.80!

It does seem as though Forex MegaDroid is overly sensitive to changes in the one input setting we have any control over, doesn't it? In addition we still haven't worked out how Albert and John can possibly justify their headline that their Droid has "No Losing Trade Since 20th March 2009!"