Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
Alright after working with the guys at MC, I have gotten some more files to work with. I tried to recreate the example they gave but since they used something that moved instead of a set date it was impossible. So i decided to try my own small test, it seems like it works now. I need to go more indepth but basically here is the run down. You can only do 1 window at a time (this is all i have tested), the trades must be closed at the end of the window, slippage and commision was simply $1 per contract (this was a major issue before), You must have enough lead time in your window to accommodate your max bars back number. If you have these things then your strategy performance report should match any 1 given window of the WFO.
Caveats: I have not tried multiple windows at once, I have not tried complicated multi data stream strategies, I have no tried tiered commissioned systems. Basically i used a preloaded strat and just filled in some blanks. But its a promising start. After I do some more testing I will be able to come to a more concrete answer. But until then at least i am able to replicate to under certain conditions what the WFO is generating.
edit: you still require the hotfix that was sent to me personally from MC. I don't know if they have it publicly available but without it you cant do anything. They should probably post something thats hard coded update released publicly but you can ask.
Alright, honestly I am getting fed up with this work.
So basically they sent me a cherry picked example where I could replicate. How did i determine this, I started testing other strategies on other time frames. This is using the production release 8.5. I was able to get very close but again, if there are trades that are crossing the period switching barrier then you have yourself a problem. Multicharts does not explain how they deal with it. are they closed at the end of the period, are they held over? I have tried both methods and haven't come up with a way to replicate exactly.
I put 8.7 beta on another machine to try it out, figuring they possibly fixed something. Its actually worse performing than my hotfixed 8.5. just adding commision without slippage creates problems with replication.
Alright, so far its been a mixed bag. I have been able to verify individual walk forward windows under some very specific conditions. They include, there are no trades that pass from 1 window to the next, you add maxbarsback (the exact amount) which can be extremely difficult to get down, and using the same start date and end date which i set inside the code itself.
However, if you try to string together your WFOs like Kevin Davey does, you will not be able to sum up your WFO windows to the same answer. Also if any 1 of your windows has a trade that passes through to the next window, you won't be able to validate the entire series (extremely likely). So does it work, kinda. But its highly limited and not reproducible in a window by window approach. I have worked with Multicharts for several weeks and they have basically told me, this is how it is supposed to function. So I dont expect any upgrades or modifications to make it more robust, results reproducible or verifiable in anyway.
If your worried about the results, you can always walk forward the old fashioned way. window by window using the date/max bars back function and be sure you get solid results.
I mean setting your date/time settings and then running the strategy there and saving the performance report. But like you mentioned close any currently open trades during that , then manually set the date/time to new settings in order to see the next window. Run a Normal optimization on each of these windows. Basically you are doing the job of the auto WFO by hand. Its the only way to guarantee a clean wf.
I am reading this thread as I was about to embark on some walk forward analysis, and now I am pretty concerned if its worth the effort on the MC platform.
Is Ninja any better?
I did a google search and found a product by codefortraders and another one called Diamond Backtsesting that is some sort of add in that does walk forward, would this be worth getting?
PS I have no affiliation of w/ CodeForTraders nor Diamond -- I am not even a customer of any of these companies, I surely am interested in a good backtesting solution, and if MC is flawed I would like to know which platform is good at backtesting and Walk Forward Analysis.
I do have a copy of AMIBroker but am having a lot of concern regarding using Tick Data with that platform. So to be frank I am pretty unsure what to do at this point in testing my strategies.