So I've been testing a strategy on market replay. I downloaded 20 days from the ninjatrader server, and as well downloaded the historical data for those same 20 days, (same instrument, same contract, same date/times). Then when I backtetsted that same strategy on the historical version, I got DIFFERENT results!
I checked the graphs, to make sure they lined up, and they did.
Then I went to look at the trades they took, and noticed that on the historical data some of the trades weren't being taken.
Mind you, this is the same strategy, with the same time-frame, same expiration, same dates, same chart.
What is going on? Can anyone enlighten me?
Has anyone come across something similar?

It's impossible to answer without the strategy logic.
You may add some debugging instructions in your strategy to get some clues. Simple Print() should be enough (high was X, indicator Z at bar Y say this, etc).

sam028 

It's impossible to answer without the strategy logic. You may add some debugging instructions in your strategy to get some clues. Simple Print() should be enough (high was X, indicator Z at bar Y say this, etc).

It's the identical strategy. Why would it be different? Does it get interpreted differently?

kidvic 

So I've been testing a strategy on market replay. I downloaded 20 days from the ninjatrader server, and as well downloaded the historical data for those same 20 days, (same instrument, same contract, same date/times). Then when I backtetsted that same strategy on the historical version, I got DIFFERENT results! I checked the graphs, to make sure they lined up, and they did. Then I went to look at the trades they took, and noticed that on the historical data some of the trades weren't being taken. Mind you, this is the same strategy, with the same time-frame, same expiration, same dates, same chart. What is going on? Can anyone enlighten me? Has anyone come across something similar?

If you haven't read the following you should:
https://ninjatrader.com/support/helpGuides/nt7/?discrepancies_real_time_vs_bac.htm

When running a backtest you only use the end of the bar for OnBarUpdate calculations. In replay/real time if you use "CalculateOnBarClose = false" then each of the OnBarUpdate calculations takes place in each tick.

The fills are different as well as stated in the link above.

Other reasons are all given in the link.

Hope this answers your question...