Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
Trying to summarize some of the essentials of coding multi-timeframe indicators...
Options
There are three options
(1) loading secondary bar series and calculating indicator values from secondary bars
(2) calculating composite bars from primary bars and calculating indicator values from composite bars
(3) tweaking the indicator
Let us have a look at the three options in detail:
(1) Calculating from secondary bar series
-> it is possible to reproduce the genuine values which would be obtained from the original bar series
-> for real-time bars, NinjaTrader processes the primary bars first and the secondary bars afterwards, which introduces a lag of 1 tick (1-tick problem of NinjaTrader)
-> for historical bars the value of the secondary bars can be accessed prior to these bars being processed by OnBarUpdate(), if timestamps of primary and secondary bar series mathches
-> the transition from historical to real-time bars is difficult to code
-> a special effort has to be made for session breaks, as time-built bars and tick-built bars with the time stamp of the session break belong to different sessions
-> the drunken sailor path of snapshots can only be avoided by smoothing or interpolating the distance covering the unstable period of secondary bars
-> when the secondary bars are range or Renko bars, the MTF indicator cannot be called from a strategy
(2) Calculating from composite bars
-> it is possible to reproduce the genuine values which would be obtained from the original bar series
-> composite bars are calculated and stored in arrays, which are similar to DataSeries objects
-> the array are accessed in a similar way as the DataSeries in standard indicators
-> when building composite bars, session breaks have to be taken into account
-> to avoid the drunken sailor path produced by snapshot data during the unstable period of the composite bars, either interpolation (repainting) or smoothing is required
-> the composite bars do not suffer from the 1-tick lag as above and can easily be called by a strategy
-> composite bars are limited to the same bar type as the primary bars
-> the period can only be an integer multiple of the bar period of the primary bars
-> genuine composite bars can only be built for time-based (day, minute, second), tick or volume charts, but not for range or Renko charts
-> for range or Renko charts an approxmination is possible via the square root relation ship between range and time
-> the composite bars approach is relatively CPU intensive
(3) Tweaked indicators
-> by definition the tweaked indicator is slightly different from the genuine higher timeframe indicator, but can be used as a substitute, or might even be superior
-> the tweaked indicator will not cut off higher timeframe bars at the session break or day break
-> tweaked indicators are limited to the same bar type as the primary bars
-> the period can only be an integer multiple of the bar period of the primary bars
-> the indicators do not suffer from lag and can be easily called from a strategy
-> the CPU load is lower compared to the other solutions
Conclusion
(1) When to use a genuine MTF indicator ....
Avoid MTF indicators, if you can. Don't use range or Renko bars as secondary bars, because you will run into trouble sooner or later.
The only reason to use a genuine MTF indicator, is when your indicator values need to be calculated from a different bar type than the primary bars. In fact there are quite a few applications, where this is necessary. Here are some examples.
(a) calculating an exact opening range on a tick, volume or range chart requires minute data
(b) calculating the genuine market profile on a tick, volume or range chart requires minute data
(c) efficient calculation of VWAP can be done by loading minute data
(2) When to use an indicator that uses composite bars ...
Easy answer: When ever primary and secondary bars are identical and you do not find a formula to tweak it.
(3) When to use a tweaked indicator...
Easy answer: When ever primary and secondary bars are identical and you find a formula to tweak it.
I am still relying on NinjaTrader to code my indicators and have recently played around with adding secondary bar series to indicators, so I just want to share my experience and show what currently can be done and what can't be done.
Of course …
However, this is an opportunity to discuss some of the key points of NinjaTrader MTF coding again.
Genuine MTF Indicators
A genuine MTF indicator is created by adding a secondary bar series to the primary bars shown on the chart, such that the indicator processes both sets of bars. In this case OnBarUpdate() will consecutively process both bar series. It is important to understand the details of how those bars are being processed.
Historical and real-time data
When a chart is backfilled, or when you refresh your chart via F5, all data on your chart becomes historical data. Historical data only contains the open, high, low, close and volume of a bar. A historical bar is therefore built from only 5 data points.
However, when you connect to a real-time data feed or market replay NinjaTrader processes and stores every incoming tick as it arrives. A real-time bar therefore contains much more information than a historical bar, which just contains the summarized information but no single ticks.
There is always one transition or hybrid bar, which is generated at the moment that you backfill or refresh your chart. Let us assume that you have a 5-minute bar, which is refreshed at 9:02 AM. Then the period from 9:00 AM to 9:02 AM is covered by a historical "tick" containing open, high, low and close of that period. Afterwards newly incoming ticks are added to that fractional bar. The hybrid bar shows a different behaviour than the historical and the real-time bars. When coding multi-time frame indicators this has to be taken into account.
The 3 key problems you need to deal with
(P 1) Order of bar processing: NinjaTrader always processes the primary bars first. The secondary bars are processed afterwards. This is a really annoying feature, because it means that the values calculated from the secondary bars are not available at the time the primary bars are being processed.
Let us assume that there you wish to display a 15-min EMA on a historical 5-min chart. The order in which NinjaTrader processes the bars is
9:05 AM bar of primary bar series -> 9:00 AM EMA can be displayed
9:10 AM bar of primary bar series -> 9:00 AM EMA can be displayed
9:15 AM bar of primary bar series -> 9:00 AM EMA can be displayed
9:15 AM bar of secondary bar series -> 9:15 AM is calculated
9:20 AM bar of primary bar series -> 9:15 AM EMA can be displayed
This means that the bar processing introduces a lag of 5 min, as the result of the calculation obtained from the secondary bars is only available after the primary bar with the same time stamp has been processed. It is obvious that you do not encounter this problem, when you build false multi-timeframe indicators via composite bars, as MetaTrader does and as can be done with NinjaTrader.
Now let us have a look at the same problem on real-time data, when every incoming tick is processed by the indicator (CalculateOnBarClose = false). For that purpose I assume that there are 3 incoming ticks, one at 8:59:57 AM, the next one at 8:59:59 AM and the last one at 9:00:00 AM. This is how NinjaTrader processes the bars
8:59:57 tick processed for primary bars -> indicator can display the EMA calculated from the prior tick
8:59:57 tick processed for secondary bars -> new value is calculated for EMA
8:59:59 tick processed for primary bars -> indicator now displays EMA value calculated at 8:59:57
8:59:59 tick processed for secondary bars -> new value is calculated for EMA
9:00:00 tick processed for primary bars -> prior primary bar is closed as it is, with the EMA calculated from the tick at 8:59:57, new bar shows the EMA value calculated from the 8:59:59 tick
9:00:00 tick processed for secondary bars -> prior secondary bar is closed, new value is calculated for EMA
This shows that the way NinjaTrader processes the secondary bars - that is after the primary bars - introduces a lag of 1 tick.
If you run a multi-timeframe indicator on real-time data with setting COBC = true, it will behave in a similar fashion as it behaves on historical data.
Workarounds: There is a workaround for the 1-bar lag on historical data, as the indicator values calculated from the secondary bar series can be accessed, before the secondary bars are accessed. For the 1-bar lag on real-time data (COBC = true) or the 1-tick lag on real-time data (COBC = false) the only option to avoid the lag, is to repaint the prior bar after it has been completed.
From this analysis, we come to an important conclusion.
Tip
Whenever a NinjaTrader MTF indicator is coded, there are 4 different cases to consider
- processing on historical data
- processing of the transition bar
- processing on real-time data with COBC = true
- processing on real-time data with COBC = false
(P2) Time stamp granularity: NinjaTrader time stamp granularity is 1 second. As far as real-time data is concerned, this is not a problem, as every incoming tick is processed for both sets of bars. However, if you refresh your chart via F5, all real-time bars are turned into historical bars. This may change the order of bars being processed.
Again let us put up a small example. This time I want to display a 150-tick EMA on a 50-tick chart. During a news release at 9:00:00 AM there are assumed 235 incoming ticks which have the identical time-stamp of 9:00:00 AM. At 9:00:01 there are another 120 ticks. For simplicity reasons let us also assume a session break at 9:00 AM, which has closed the prior (typically incomplete) bars. Therefore both the 50-tick bar series and the 150-tick bar series start with the first tick after 9:00:00 AM (time stamp of that tick is 9:00:00 AM). This is what happens
bar A1 of primary bars is closed after 50 ticks
bar A2 of primary bars is closed after 100 ticks
bar A3 of primary bars is closed after 150 ticks
bar B1 of secondary bars is closed after 150 ticks
bar A4 of primary bars is closed after 200 ticks
bar A5 of primary bars is closed after 15 ticks with timestamp 9:00:01
bar A6 of prmary bars is closed after 65 ticks with timestamp 9:00:01
bar B2 of primary bars is closed after 65 ticks with timestamp 9:00:01
The order of the bar processing is therefore A1 - A2 - A3 - B1 - A4 - A5 - A6 - B2.
Now let us have a look at what NinjaTrader does after a refresh with F5. The refresh transforms the real-time bars into historical bars, and they are now processed by using their time stamps. If bars have identical timestamps, primary bars will be processed first, secondary bars will be processed afterwards.
This leads us to the following sequence A1 - A2 - A3 - A4 - B1 - A5 - A6 - B2
In this case the order of processing the bars B1 and A4 has been inverted. The reason is that historical bars cannot be correctly synchronized, if they have identical time stamps.
Tip
When you refresh a chart via F5
- real-time bars (multi-tick) are converted into historical bars (OHLC)
- synchronization of bars with the same time stamp can be false
- MTF indicator values displayed maybe be inaccurate on historical data
- all of which can be explained by NinjaTrader's timestamp granularity of one second
(P3) Different time stamps for time-based and tick-built bars: The logic used for minute bars and for bars built from ticks is different when it comes to time stamps. This is the third major nuisance to be taken into account, when coding MTF indicators.
Again this can be shown with an example. Let us assume that I have a minute bar and a tick bar (periods do not matter) which both have a time stamp of 9:00:00 AM. In that case
- the closing price of the minute bar shows the last tick that printed prior to 9:00:00 AM
- while the closing price of the tick bar refers to a tick that printed after 9:00:00 AM
The two timestamps, although identical refer to different periods. This is particulary unlucky, if you have introduced a session break at 9:00:00 AM. In that case the minute bar with the 9:00:00 AM time stamp belongs to the old session, while the tick bar wiht the 9:00:00 AM bar belongs to the new session.
Having understood the 3 basic limitations of NinjaTrader, we are now ready to code a simple MTF indicator.
This post has the objective to show how to code a multi-timeframe SMA that
-> calculates SMA values from a secondary bar series (BarsInProgress == 1 in Ninja Speak)
-> and displays the result for the primary bars of the chart (BarsInProgress == 0 in Ninja Speak))
It is further understood that the values shall be only calculated from the secondary bars, when a secondary bar has been completed. I do not want to have all the intermediate snapshots of the MTF SMA plotted, but the chart shall for each bar of the primary bar series display the value calculate for the last completed bar of the secondary bar series.
MTF indicators have a small in-built lag
The bars of the primary and secondary bar series are not usually calculated at the same time. Imagine that the primary bars are 233-tick bars, while the secondary bars are 3-min bars. Then there will be a small lag, as
-> the indicator value will be calculated when a 3-minute bar has been completed
-> that indicator value will be shown on the next 233-tick bar, the difference between the timestamp of the 3-minute bar and the timestamp of the 233-tick bar is the lag
-> in case that the next 233-tick bar only plots 4 minutes later, it will not show the indicator value from the first 3-minute bar, but from the next one, the value of the first one will never be shown
The main reason to use MTF indicators with NinjaTrader is to avoid the non-equidistant bar spacing, which comes as a side effect of a linear time scale which is used by NinjaTrader for charts with multiple bar series. Please be aware that you get rid of the non-equidistant bar spacing, but that you pay with a small lag. This lag is part of any concept of MTF indicators, it has nothing to do with NinjaTrader.
Special case: Unnecessary lag introduced by NinjaTrader
I would like to come back to the problem (P1). In case that your secondary bar series uses the same bar type as the primary bars and a multiple of the primary bar period - this is the simplest case of MTF indicators - NinjaTrader produces that delay of 1 bar (historical bars or COBC = true) or 1 tick (real-time bars with COBC = false). This lag can be neutralized on the chart by painting back a single bar (historical bars or COBC = true) or paint back a single tick (real-time bars with COBC = false). Typical cases where this occurs would be
- a 60 min SMA plotted on a 5 min bar
- a 899 SMA plotted on a 233 tick chart
You will have the problem when both bar series are built from minute bars, or when both bar series are built from tick bars. In those cases I assume that there may be frequent bars with identical timestamps, which is exactly the case where I need to paint back one tick. If the secondary bar supplying the indicator value and the primary bar displaying the indicator value have different time stamps, the dispaly suffers from the unavoidable lag. But if they have identical time stamps, the chart suffers from the lag caused by NinjaTrader processing the secondary bars after the primary bars for the same incoming tick.
Special Case: Avoiding the lag on historical bars
For historical bars there is a workaround for the problem. If you study the NinjaTrader user manual, you will learn that OnBarUpdate() processes primary bars and secondary bars. If you wish to know which of the bars are currently being processed, you can check for the value of "BarsInProgress". The natural architecture of a multi-timeframe indicator would be
I have made the observation that for historical bars the value of CurrentBars[1] already changes when the primary bars are being processed. This allows me to access the indicator value calculated from CurrentBars[1] prior to the processing of the secondary bars, and I can avoid the lag. The changed indicator architecture is now
Calculating the indicator value
The first section of the code has the objective to retrieve the value. There are two cases to be considered
Case 1:
The indicator value is calculated from a historical bar or in COBC = true. In this case there is only one "tick" - this refers to a single value for OHLC - and the only calculation produced is the final indicator value for that secondary bar.
Case 2:
The indicator value is calculated from a real-time bar in COBC= false. In this case we only wish to perform a calculation for the prior secondary bar, which is now completed, as indicated by the first tick of the current bar. No calculation will be performed from the first or any intermittent ticks, I am only interested the final OHLC values of each bar. Otherwise my indicator would plot something like a drunken sailor's path.
Case 3:
This is a bit tricky. In most cases NinjaTrader produces a hybrid bar at the transition between historical and real-time data on your chart. In this case the hybrid bar is built from a single historical "tick", which comes as OHLC and the following real-time ticks. However, in some rare cases, if the user refreshes the chart exactly at the point when the prior bar has been completed, there is no need of a hybrid bar, but the last historical bar is followed by a bar only built from real-time ticks. This case also needs a different treatment.
I think that I have now explained some of the considerations, and will proceed to posting the indicator code with detailed explanations.
Good thread. As mentioned by FT, MTF indies are a pain in the arse
So, I'm wanting to do something pretty simple. Which is essentially just to take any TF desired, and plot the opening price of that TF onto a 1 min chart. In the end, I will simply be exporting the data of the 1min with other metrics, into a tabular format for later analysis.
It seems it would be a simple affair. I have tried the MTF approach several ways. I am getting the bar lag mentioned by FT. I have tried to use the primary 1 min series to set the open, as well as using the Open from the secondary series as well. But I am not getting the exact print out on the chart that I want. It's either a bar off to the right, or a bar to the left.
I am wanting to use the MTF approach as I like the idea of simply being able to export various TFs to test many different things.
Since I will be exporting data, and only am concerned about doing this on historical bars, I'm not concerned with needing to get this to work with real-time or hybrid bars.
I think my current issue is just the order in which the bars are being processed. Just for an example, I have a 1min primary series, with a 15 min secondary. The operation is basically:
Initialize a dataSeries @ the open to the Open price of the day.
If BarsInProgress==1, set the secondary Open to a variable
After above step, set the dataseries that plots on the chart to the above initialized variable if BarsInProgress==0
Here is a stripped down version with only 1TF, but in practice would have many more for export.
-----EDIT-----
As the secondary bar closes, I managed to find a crappy work around by indexing into the next secondary bar via a negative index. Bad practice but this might be ok since I'm only looking to use this for historical data export.
For the case that you mention - 1 min bars versus 15 min opening price - you dou not need to load a secondary bar series at all. Just code a little indicator that does what you wish.