Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
I am currently backtesting a new model in excel, where I tested 47 trades (I am doing this manually; don't have any backtesting software, thats why I currently only tested 47 trades) here are my statistics:
So I was reading a thread that Mike posted and downloaded this Monte Carlo simulation model (which is attached), I ran numerous simulations and it reported positive profit after 47 trades (ranging from $400 to $2000)
I am not sure if this is too good to be true or not, or how valid/relevant the model Mike posted is.
Here is just a quick rundown of what I saw and how I interpret my own backtests. so take this with a grain of salt. I am not judging manual vs automated testing or any of that at all, just purely at the results given.
So, the first thing i notice is low sample size 47 trades. Yes statisticians say 30+ is "significant" for a sample however more is always better. A quick plug into the old formula you get 14.43% standard error with this sample set alone. So any monte carlo based on that is going to have a built in 14% error either way. Monte carlos are only as good as the data set they can take from.
Looking at the trade stats you NET .5(pt) per trade and win 55% of the time. Is this with or without commissions? If not then they are going to take a sizable portion of your edge and make this system very difficult to trade. If it is with commissions then go on right ahead to incubation mode.
Another thing you did not mention is how many rules you have or filters. When you have a sample size this small it really can effect and skew both results and monte carlos. If you have 500 + trades and have 5 filters your ok, but if you 50 trades and 5 filters thats not ok. Statistically speaking, although i am a huge proponent of less filters, and less rules are more robust.
Also mention the date range that your using to test. Is this over 1 day, 1 month, 1 year? Both insample and out of sample to cover atleast 1 full market cycle. Though optimal is not very realistic, however more time is better. I look to have at least 3-5 years segregated for out of sample.
First, I have to thank you for taking the time to help me out. I really appreciate it . (sitting here at home trying to figure this all out on my laptop).
Please realize that this is my first model ever I am attempting to implement using the @kevinkdog style. I have no experience whatsoever in backtesting (but did read his 'Taking it Live' thread).
My challenge is being able to backtest 200 samples without using any software.
Yes I realize N is small, this is due to the fact I am manually backtesting this (ie looking at the trade and manually recording the entry/exit, profit/loss details into excel.
Currently N= 50 Ave win = 1.38point Ave loss = 0.87
I currently havent figured out how to use backtesting software, (most likely will get Ninja Strategy Analyzer)
Sorry, which formula is this? ( I still have a lot to learn).
No I forgot to mention commissions, they are 4.7, but I did factor them into the monte carlo analysis.
I have 5 rules. So should I eliminate some of them and rebacktest?
The sample ranges from 14 April to 2 September.
How do you determine a market cycle?
Btw how much experience do you have backtesting? and what software you use to do it?
I have CQG and DTN iq feed. You can search their websites to see which service fits your needs. i would say DTN is best bang for the buck. However the best free resource is quandl, you can download a lot of free good data there.
Glad you've built your first model. Just like you, when I started my first model from excel I had tons of questions and wanted to share any some info that might be of use to you.
1. On Backtesting Software - Treydog has given excellent points. Apart from the speed and scale which you can backtest many markets/asset classes, there is also the huge benefit of automated execution. I found my usage of time much more productive in research instead of monitoring trades. One can now trade up to 20 or more positions simulatneously (long term or short term). The caveat of course is that the model developed is accurate and robust and monitoring systems from time to time is still a prerequisite.
2. On Monte Carlo - It is an interesting measure but wouldn't put too much faith in it. Actual drawdowns are a function of a specific market events/environment in relation a series of bad trades produced. Trade scrambling removes the serial correlation which happens during this period and how the drawdown was actually produced. It would be useful to look into the periods which drawdowns occured and understand what caused them. This is crucial to deduce if risk management approaches are needed (stopping the trading model, develop non-correlated models etc.) ... All these decisions have 'costs' to the long term outcome of the eventual pnl. Alternatively, one could accept that this is a function of markets where drawdowns are inevitable from time to time and just continue to stick to the plan.
3. Number of trades - Perhaps instead of measures of statistical significance, one could take a qualitative approach and see what the system was designed to capture. Warren buffett as well as many long term trend followers although might fail statistical tests by their number of trades, but their performance is certainly more than random. One needs to take into account the holding frequency and frequency of conditions present.
4. Clean Data - Good models need to be built on clean and more importantly adjusted data. Even accredited data providers do not provide properly adjusted data for backtests and will lead to incorrect trading conclusions. Thus, the need for backtesting software to adjust data to generate a proper backtest.
Lastly, I personally found the most crucial parts in building good trading models was in thinking and capturing the stationarity of the market. I got this general idea from visiting the local library and reading books by clifford j sherry. The 2nd part was asking myself how long this 'stationarity' would tend to persist. The better which these 2 questions could be answered, the better the models have performed going forward. I hope some of my personal experiences would be of use to you going forward.