Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
Backtesting & Optimization -- worthless? How to get value from them?
I've seen a lot of replies buried in threads that subtlety suggest that backtesting & optimization do not have as much value as we might be led to believe. People have posted "successful" strategies, and advanced users quickly reply to say the results are unrealistic and can not be replied upon (and often fail, miserably, in live trading).
Simple questions -- Why? And how should we be utilizing these functions to realize the most value from them?
I'm sitting here trying to develop a strategies (as a new NT user) using the Backtest feature of Ninjatrader. I have been giving weight and considerations to the minute differences in results from the Backtesting engine. However, after reading some of the posts here at Futures.io, I wonder if I am throwing out profitable strategies, going in the wrong directions towards improving a strategy due to unreliable(?) data coming from the Backtesting .
I've seen some people suggest to use optimizer on half of your data series, then backtest using the optimized parameters for the other half. If that's something that is sound advice, that is the kind of information & tips that I am seeking with this thread.
Hopefully the experts and advanced users can shed some light here.
It boils down to 'know' what you are doing and what to expect from back-testing and optimization.
It is possible to take a period of data and find 'magic' settings that create the best possible (optimized)
result over that period. Now adding data from another period, or just from tomorrow, will change the
results and they may be horrible.
Is all optimization bad ? no, you need to understand what you are doing
If a series of values for a parameter 16,17,18,19,20 all create positive profitable results and you pick
the best value, the chances are that your system is less vulnerable. If you have 18 being profitable
and all the rest loosers, guaranteed your strategy will fail in forward.
Probably a good read would be the book of Kevin. it walks you step by step through an approach.
Is it the only possible approach ? no.. for sure not
Is it a good approach ? if you apply and the strategy passes the tests, then you are on good track
Can you link to "The book of Kevin"? I tried some searches and I guess that's not the title?
You bring up a good point with regards to general optimizations. For example, many people favor SMAs (or EMAs) of 50/200 or 9/50. I assume these numbers are the favorites because they were tested (long before NT existed) and seemed to be the most consistently proven winners across the board? Or were they just rabbitoutofhat starting points that stuck after a couple people wrote books about them? Even today on many wise TA educational sites, you will find "standard" sets of numbers to use for SMA crossovers (and RSI, and MACD, ADX, etc. etc.). But of course I agree... 50/200 may be hat magic today, but terrible tomorrow.
For optimization specifically though, I do like the idea of using the same specific periods of difficult historical trading periods + a few "regular" trading periods to test against as a sort of benchmark. A backtesting profile? Something like that. Then when you find something that worked through the bulk of those periods within your standard profile of periods to test, retry against a current or more recent period of time and see how it works out.
With regard to back testing I'm interested in larger historical time periods and thus have a need for a good data source. I am specifically interested in YM, ES, NQ, ZN,ZB,E6 DAX etc.... I'm not interested in free stuff etc... Just a quality source where I can purchase the needed data and upload it into NT7. Anyone have any experience with this, hopefully going back at least 3 years.
I made a move from NT8 to MC recently. I just find the Easy Language stuff faster for me to work with. I'm still trading with NT but should be migrated shortly. As a result my data needs changed so I switched providers for historical. I had good luck with these guys for NT and for that matter it migrates to MC if you ever use that. Their not cheap but for me that wasn't my prime deciding factor.
I believe Tckdata is cheaper but DTN has the NXCore product with a wide reach.
I just downloaded the MultiCharts.Net platform for a trial. Thanks for the data links.
I had good luck with these guys for NT and for that matter it migrates to MC if you ever use that.
So, to ensure I follow you, you were able to initially use that data in ninjatrader and eventually port it over to MC ? If correct, that is a nice bonus !
