tl;dr In the next weeks, I am planing to releasy my private Walk Forward Analyzer, but before I do this, first I want to make sure that also the beginners in mechanical trading understand why common backtests are quite useless and why walk forward analysis is better in so many ways, so I decided to start this discussion.
I WILL RELEASE THIS TOOL FOR FREE AND OPEN SOURCE!
And as people keep asking why I release free stuff, let me explain my motivations: I am targeting a job in the trading-economy, and this is not easy to get. Beeing known in the community would help me a lot to accomplish this goal, so I decided to release some of my private tools for free and/or open source to make myself a name over the next months. I am not here to sell you stuff, keep that in mind, please!
For the record: I do not claim that WFA is the holy grail in system testing, but I argue that it is way better than normal backtests.
THE GENERAL APPROACH TO SYSTEMATIC TRADING
Take a trading logic => Generate an EA that trades exactly like it => Optimise Parameters => Do a backtest => Trade it live => $$$
BUT: Everybody that has ever had an EA that was a money-printing machine in the backtests (and this is very, very easy to be done), knows that a good backtest does not imply profitable live trading.
And that is the problem. We need a test-method on whichs evaluation we can rely!
STATEMENT 1: Your parameter choices are not good
Well, every EA consists of 2 things:
1. the logic/script itself
2. the parameters (like periods for moving averages, stop loss values etc. Just everything that can be adjusted!)
The first thing is static and "given". And a lot traders only focus on this part.
But, an EA can behave in very different ways, depending on the parameters. And there can be billions of possible parameter-combinations (=parameterset). There are 3 approaches to determine the parameterset for live trading.
1. the intuitive approach (non adapting):
The trader just chooses the parameters based on his expert knowledge.
But, there are just too many possible parametersets. You can't just "guess" them without evaluation and testing.
It does not matter how good you are, you are never good enough to "think through" this huge amount of possibilities in a reliable fashion!
2. the "optimise on all data" approach (non adapting):
The trader chooses the parameters based on all past data.
So, the configuration/parameters that worked best for the last 12 years, for example, are chosen to be traded live.
But, to stay up-to-date it is not a very good method to optimise the parameters on so much and therefore OLD data.
Because the market today is not the same as it was 12 years ago.
3. the "optimise on the last few years" approach:
The trader chooses the parameters based on the last few years of data.
But you can not just take this approach without testing how this "optimisation method" would have worked in the past.
And this is exactly what a walk forward analysis will do: "Optimise on the last few years", but it tests this approach on all data in the past!
STATEMENT 2: A non-adapting EA can never make longterm profits
I mean, there is an infinite amount of possible trading-systems AND an infinite amount of ways the market can change AND an "infinite" amount of possible parameters for your trading-system.
Considering this, would you really want to bet money on the fact that you have a trading system that will always work, in the same way, without adaption, on all future market conditions? I would not.
"But why shouldn't I be able to put a traders knowledge into a script and trade it?"
Its simple: A trader always learns, he takes input from many sources, he has knowledge about the markets and therefore he adapts his trading strategy so he can always be as close to the markets as possible.
A simple EA script can't do this, a Walk Forward Analysis can (to some extent).