Consistently profitable mechanical traders are versed in programming, math, and statistics. First and foremost, they know how to back-test correctly, and how to avoid the multitude of statistical biases and errors that can arise. Even traders who aren’t looking for a magic system often make the mistake of thinking that their back-testing has any merit. More often than not, it’s mired in numerous statistical errors that leave it not only incorrect, but worse yet dangerous to their trading account. the math and statistics of correct back-testing and system building are extremely complicated. Unless you’re a math and statistics whiz, you don’t stand a chance.

System traders that are successful, understands the markets in a deep way. This is of vast importance because market conditions (like volatility and directional conviction) are cyclical and they go through different kinds of patterns over time. So even if you were given a solid algorithmic system by a great mechanical trader, you still wouldn’t be able to make money from it over long periods of time because it would need to be fine-tuned to adapt to changing conditions. Moreover, the great mechanical traders will often have several systems and they’ll choose which ones to employ at what times based on their deep understanding of market behavior.

I would ask myself if there is any fundemental reason for this strategy to perform like this. Are you taking on risks others are not comfortable with, or exploiting other real market mechanisms. And to be honest, a backtest like that is unrealistic as you know everybody in the world is looking for that curve, institutions with lot more resources than us retail traders.

Sent using the NexusFi mobile app

First of all I wanto to thank you all for your replies ans suggestion. Any critics will help me to improve my validation process.

amoeba 

Hi David, These comments lead me to think it is possible you have accidentally fallen into curve fitting your strategy; For system research and testing, I know some traders who will develop on a small historic sample, then test on a out of sample walkforward. If that out of sample walkforward test not produce good results they abandon the original concept and move on to a new idea. But if you go back to the development data sample and make changes to test again on the forward data, you no longer have out of sample data, and have fitted your results to the whole data set. There are some really good webinars on this site by fellow autotraders that explore many of these issues, do a search for Kevin Davey, he outlines a very good methodology for developing and testing trading systems, you can also find him on this site @kevinkdog Don't be discouraged though, I believe Kevin also say's he might get a couple profitable strategies out of 100 ideas. So it is a case of having a solid methodology and continuing the efforts.

Hi Amoeba,

Yes I completely understand this point of view and I also thought that I should be carefull to do not overfit with the system but I also don't want to be scared to do some kind of optimization because sometimes it can lead you to new ideas and values that otherwise you woulnd't ever think about.
Anyway your point about the Out of the Sample data is true because I started to develope it using the last year. BUT when I started to use data from 2013 the backtest shown me cases and patterns ----> entry/exit cases that I wasn't able to see in a market movement during the last year.
My purpose is to develope a system at least using 3/5 years... doing it on only on 1 year can lead you to too optimistic and maybe unrealistic results and the only reason I started with just one year was to set up a raw system of rules and just see if it can have a chance and then I moved to 5 years development.

Talking abount out of the sample data:
please note that when I completed the 2013-2018 system (with a slight optimization just over the STOP LOSS VALUE) then I just switched the start date WITHOUT TOUCH ANYTHING to 2008 and as you can see in the backtest the 5 years 2008/2013 are performing good without any optimizazion over that !
I know it's not a classic Walk Forward Optimization that uses the last 6 month as OTS data but 5 year of past out of the sample should be taken in consideration as a good UNTOUCHED DATA isn't it ?

martinhunting 

If you think it is profitable test with real money!

Very helpful thanks

@jackbravo and @matthew28

Please read my answer above to @amoeba where I'm saying that the system has been tested on untouched data from 2008 to 2013.

Of course I'll plan to do an incubation period by leaving the strategy on multichart and Iqfeed to see how it works on the charts in real time and without do real operations. That's always useful , I'm also doing it for other Trading systems and by that way you can also understand if your backtests are trustable or not (it really depends on your system's logic).

hughesfleming 

Thanks for posting your back test Dvdkite, Earlier in the year, you mentioned that you were testing Heiken Ashi bars with your strategies. Does this use normal OHLC bars or HA bars? If they are normal bars then well done. That would be a very impressive result and I would consider moving forward with the strategy. If it does not happen to use a standard bar type, then I would be very careful risking real money. A good test is to use IB's paper trader for forward testing. If your paper trading statements match what you see in terms of win rate in the MC back tester then go for it. Best of luck, Alex

Hello Alex,

in the past ( at the beginning ) I used to create strategy directly on HA chart just to discover that 99% of entries are made at unreal prices with totally unrealistic and simply FALSE backtesting but it's not this case.

This system is running over a 5 minutes REGULAR BAR chart. I've coded manually the HA bars logic so and I just use them somewhere to see how the dax is trending (example if a previous HA bar is RED or GREEN I do have a confirmation ... otherwise not).
Anyway ALL the logic is coded over REGULAR bars and it does entries based on VAH levels. So I'm using the Yesterday Value Area High and Low and Point of Control for my entry and exit setup.

As I can read all you guys always test it on SIM for 3 /6 months?
In my case I'll leave the strategy live on a MC chart with DATAFEED every day with no-broker configured and I'll save all entries at the end of every day into an excel to compare them later with the backtesting. As I've understood this seems the only way to discover if the system is performing the same in backtesting and it real time trading.

PK 1 

What I always do is thoroughly checking entry and exit-points on the chart on one hand and on the other the expectations from my algo. Developing the system you have to have a good understanding regarding the context a trade is getting triggered. So comparing the expectations compared to the actual entry/exit-points is worth a lot. Regarding some common problems even without knowing your strategy one could see on the chart whether you are trapped into a programming problem which causes the great result. Maybe you could post some screenshots with trades. As you are using Multicharts it could be easy to check whether the strategy is satisfying in live mode or not. Just run the strategy with your broker-feed after applying the correct configurations. Correct configurations and it's no problem to use the real-account, then the trades will occur in your chart and performance report and no trades are forwarded to the broker. Just configure it first, this way I do it for myself and it's a great way to test it before going live with transmitted orders. At the end of the session I always were evaluating the daily outcome of the strategy and comparing it with backtests made after the session for that day. This way you can achieve the test with out of sample data, which seems necessary.

Bosch777 

So perhaps you would like to tell us some more about this strategy such as the set-up, entry triggers etc.. This way you might let us run a similar strategy to help with the validation process...

Hello @PK 1 and @Bosch777,
this is exacly what I'm planning to do (and I'm already doing for other systems) to check if entries are triggered where I really want. And as I said above I think that the only way is by having the strategy live on MC just with the datafeed and see if it makes entries in real time as expected.

You talked about great result eheh I think the strategy is doing good ok, but I don't think it is soo incredible .. that curve is over 10 years... and it seems that is pointing straight to the top but it also have its own Drawdowns.... please view the LONG entries equity .. can you see that it is not so perfect? Also if you look at the monthly earning list you can see that it is LOSING EVERY MONTH... of course it EARN MORE than it lose but I mean It's not so unrealistic... the system has negative trades too...
I simply made a rule with NO MORE than 1 STOP loss a day to limit my daily loss personal tollerance eh eh

Also by checking entries and exits carefully in the charts I don't see anything that could point me to a programming trap expecially because this strategy is not doing some magical stuff. I'll not reveal all the logico but as I said it is simply woking on TPO concept. It define a BIAS (that can change) and then it make entries using the VAH levels of the previous day wich are precise defined prices as Yesterday Value Area High and Low[/I] and Point of Control and then I search some patter over the past X numbers of bars... that's it.

You know ...using TPO has a lot theory behind it, I just made my own rules to use those levels...

rlstreet 

I would ask myself if there is any fundemental reason for this strategy to perform like this. Are you taking on risks others are not comfortable with, or exploiting other real market mechanisms. And to be honest, a backtest like that is unrealistic as you know everybody in the world is looking for that curve, institutions with lot more resources than us retail traders. Sent using the NexusFi mobile app

Hello rlstreet,
as I said above in my last reply over this one, I don't think this is a super performer system. It has a decent curve but it's a ten year curve...you should check for example the LONG equity and see that it is NOT perfect and it still have a lot of DrawDown periods ...the largest DD is around 5K .... and i you look at montecarlo analisys it could be twice as that.

I'm honestly not taking any special risk. I'm tryng to take risks as low as possible expecially because this year I'll go live for the first time ever with a strategy and I simply would like to do all tests and theory at the best I can just to say to my self that even if it will lost money I've done the best that I could.
However this system really doesn't do anything special than entry/exit over the previous day VALUE AREA LEVELS....the difference between this and other TPO systems is that I have my own pattern that trigger the entry over the level... and as I said it LOSES many time too... so I don't think it is so unrealistic.

In the past I designed a system that has a PROFIT FACTOR of 88 .. it was the beginning and I was thrilled about that.. but I quickly discovered that despite is was doing like 9k a day over the dax then in the real time simulation it was losing 3k eh eh ... I quickly managed to recognize UNREALISTIC backtests and it helped me a lot...

Now I have a PF of 1,92 and it is not so incredible or unrealistic....don't you think?

martinhunting 

Consistently profitable mechanical traders are versed in programming, math, and statistics. First and foremost, they know how to back-test correctly, and how to avoid the multitude of statistical biases and errors that can arise. Even traders who aren’t looking for a magic system often make the mistake of thinking that their back-testing has any merit. More often than not, it’s mired in numerous statistical errors that leave it not only incorrect, but worse yet dangerous to their trading account. the math and statistics of correct back-testing and system building are extremely complicated. Unless you’re a math and statistics whiz, you don’t stand a chance. System traders that are successful, understands the markets in a deep way. This is of vast importance because market conditions (like volatility and directional conviction) are cyclical and they go through different kinds of patterns over time. So even if you were given a solid algorithmic system by a great mechanical trader, you still wouldn’t be able to make money from it over long periods of time because it would need to be fine-tuned to adapt to changing conditions. Moreover, the great mechanical traders will often have several systems and they’ll choose which ones to employ at what times based on their deep understanding of market behavior.

Hello Martin,

I appreciate your point of view and maybe many of your concerns are has been already replied in someone else's replies. Anyway this sounds more like a teaching theory speech that points to discourage me just because the trading world is completely in the hands of mathematichans and statistics. But honestly I didn't say nothing about my background ( I have experience with prgramming and even with some mathematica and statistics from my University Degree in Electronic Engeneer )but that's not the point here.
All your statements are CORRECT and you are right , but this answer doesn't help me to IMPROVE my system validation process at this time. I made this post just to discuss constructive feedback and to see what other automated trades do to validate their own systems.

Regarrds,

David

Hi Dvdkite,

I would highly recommend that you do use the IB paper trader if you can. For example, I have to run my strategy at second intervals for the Multicharts back tester to resemble anything close to what I get at IB. It is a second opinion in a way to make sure that your fills are not overly optimistic. If your broker fills match what you see in back testing then that is one less thing to worry about. The next step is to see how sensitive your strategy is to the feed itself. You may have trades that execute or not execute in real time that never show up when you reload your historical data. You need to track how often that happens. If it happens a lot then you can't trust your back test. Think about your latency. You may want to have a data feed that keeps everything in Europe if you are in Italy and trading the Dax.

Allow several months of forward testing. The middle of summer is not an ideal time to start collecting data. Your important months will be September to January.

Best of luck,

Alex

What is the highest performing algo you have running right now? If you compare these results with this algo, don't you think this is the best performer of all?

Sent using the NexusFi mobile app