HOW MUCH RISK IS BEING TAKEN BY MARKET MAKER

Attached is a screenshot which pushes things a bit further along.

It's interesting to know whether Market Maker's relative Inventory
is "long" or "short" over the retention interval... but

It's more interesting to know whether there is "Risk"... but I'm
sure you'll agree that...

It's even more interesting to know HOW MUCH RISK in money value
is being taken?

Now, this isn't a "full inventory analysis" but you'll find that the results
are proportional, or a good "estimate" of the situation which should
further help you to evaluate the situation.

I WILL POST THE CODE A BIT LATER ON TODAY MAYBE.

Screenshot attached. There are TWO series simultaneously running,
which is why you see 2 numbers. These can be interpreted as
something proportional to Total Risk in Dollars; and you'll gain
experience with what values are important and large; and what values
are just trivial...

hyperscalper

Hello.

In first place, thanks for your work.

I'm trying to make market replay with this configuration but only get a two lines with same values.

Does your indicator have a problem with the market replay?

Thanks in advance and sorry for the inconveniences.

hyperscalper 

CLASSIC NASDAQ DIVE TOWARD END OF SESSION The TradeFlowRisk indicator (2 instances) is shown here; and note the situation before the "classic dive" which NQ/MNQ performs toward the close of Tuesday's session (today). Note the Short Risk at the top; and significant Short Inventory prior to the dive. I've said before that this is a "situational analysis" but it is warning us that a dive is entirely possible... Params are: comment line RISK_THRESHOLD=10 RETENTION_SECONDS=120 MULTIPLIER=2.5 BIGLOT_MINIMUM=2 TAPER_SIZE=True SUPER_SPIKE_THRESHOLD_RATIO=4 and... # comment line RISK_THRESHOLD=10 RETENTION_SECONDS=900 MULTIPLIER=1 BIGLOT_MINIMUM=2 TAPER_SIZE=True SUPER_SPIKE_THRESHOLD_RATIO=6 Note the 2 Indicator instances are set to Left margin; dragged over Price, and also Zero is Cntl-Dragged to middle, and vertical scale is dragged to suit; plus one of them is using a MULTIPLIER to make it relatively bigger in swings up/down... etc. lol hyperscalper

alazif 

Hello. In first place, thanks for your work. I'm trying to make market replay with this configuration but only get a two lines with same values. Does your indicator have a problem with the market replay? Thanks in advance and sorry for the inconveniences.

Thanks for the good werdz

Well, turns out that to replay something like that, and get
any accuracy, you need special processing... I'm still mulling
over whether it's worth it at all... I doubt it.

I've looked at it, but it's really just too much trouble,
most likely... purely from a technical perspective, and
it's by no means "natural". Even handling backfill Historical
phase is a pain in the arse.

Nevertheless, if I think it's worth the trouble, I may do it
but I don't really do things the "NinjaScript Way"; so it's
straight to Damnation for me; no doubt LOL

I've had requests for backtesting/replay, but from people quite naive;
who have not done such a thing themselves !!

I'm not saying you're naive, at all. But there's a bunch of
extra coding to convert it from RealTime; hardly seems
worth the trouble...

So I suggest if you use it; that you just do Real Time
processing, and fiddle with it for a few days to see whether
it has value. It was designed for the Nasdaq Contracts, but
should work with anything vaguely similar.

hyperscalper

UPDATED CODE SHOWING A TOTAL RISK VALUE

Call this TradeFlowRisk 1.0

It makes an attempt to write a value which can be taken as
the "How much Risk?" estimate; so, the bigger it is, the more
significant is the signal which tends to turn the market price
trend.

 

Code

#region Using declarations using System; using System.IO; using System.Collections.Generic; using System.ComponentModel; using System.ComponentModel.DataAnnotations; using System.Linq; using System.Text; using System.Threading; using System.Threading.Tasks; using System.Windows; using System.Windows.Input; using System.Windows.Media; using System.Xml.Serialization; using NinjaTrader.Cbi; using NinjaTrader.Gui; using NinjaTrader.Gui.Chart; using NinjaTrader.Gui.SuperDom; using NinjaTrader.Gui.Tools; using NinjaTrader.Data; using NinjaTrader.NinjaScript; using NinjaTrader.Core.FloatingPoint; using NinjaTrader.NinjaScript.DrawingTools; #endregion //This namespace holds Indicators in this folder and is required. Do not change it. namespace NinjaTrader.NinjaScript.Indicators { /* Code is freely offered as tutorial examples; but is "As Is" and the user is fully responsible for the consequences of any changes. The purpose is to encourage modifications; but be sure you have saved stable versions, perhaps on a daily basis, in case you break it. */ public class TradeFlowRisk : Indicator { protected override void OnStateChange() { if (State == State.SetDefaults) { Description = @"in Futures.IO an exercise in addding Risk to a simple Trade Flow analyzer."; Name = "TradeFlowRisk 1.0"; Calculate = Calculate.OnBarClose; IsOverlay = false; DisplayInDataBox = false; DrawOnPricePanel = true; DrawHorizontalGridLines = false; DrawVerticalGridLines = false; PaintPriceMarkers = true; ScaleJustification = NinjaTrader.Gui.Chart.ScaleJustification.Left; //Disable this property if your indicator requires custom values that cumulate with each new market data event. //See Help Guide for additional information. IsSuspendedWhileInactive = false; // always run FileName = "replaceThisFilename.txt"; AltColor = false; //VirtualTimebaseProperty = false; // default is Real Time only return; // good practice } else if (State == State.Configure) { ninjaPrint2("Configure state..."+getObjectID(this)); indexCounter = 0; // start at zero // index named variables automatically assign in order AddPlot(new Stroke(Brushes.Black), PlotStyle.Hash, "zeroIndex"); // index zeroIndex = indexCounter++; // starts with zero; plot idx Plots[zeroIndex].Width = 2; AddPlot(new Stroke(AltColor?Brushes.Blue:Brushes.Purple), PlotStyle.Line, "inventory"); // index inventoryIndex = indexCounter++; // starts with zero; plot idx Plots[inventoryIndex].Width = 4; ninjaPrint2("End Configure state..."+getObjectID(this)); return; // good practice } else if (State == State.Historical) { ninjaPrint2("Historical state..."+getObjectID(this)); try { if (ENABLE_VIRTUAL_TIMEBASE && VirtualTimebaseProperty) { initOnce(); isReady = true; } } catch(Exception ex) { ninjaPrint2("Exception "+ex.Message+" in State.Historical"); } ninjaPrint2("End Historical state..."+getObjectID(this)); return; // good practice } else if (State == State.Transition) { ninjaPrint2("Transition state after Historical, before RealTime..."+getObjectID(this)); return; // good practice } else if (State == State.DataLoaded) { ninjaPrint2("DataLoaded state..."+getObjectID(this)); return; // good practice } else if (State == State.Realtime) { ninjaPrint2("Realtime state..."+getObjectID(this)); try { initOnce(); isReady = true; } catch(Exception ex) { ninjaPrint2("Exception "+ex.Message+" in State.Realtime"); } ninjaPrint2("End Realtime state..."+getObjectID(this)); return; // good practice } else if (State == State.Terminated) { ninjaPrint2("Terminated state..."+getObjectID(this)); isReady = false; deInit(); ninjaPrint2("End Terminated state..."+getObjectID(this)); return; // good practice } } private string getObjectID(Object o) { return ""+o.GetHashCode(); } #region Properties [NinjaScriptProperty] [Display(Name="FileName.txt", Description="Properties file in default folder", Order=1, GroupName="Parameters")] public string FileName { get; set; } [NinjaScriptProperty] [Display(Name="Alternate colors", Description="blue/orange vs red/green", Order=2, GroupName="Parameters")] public bool AltColor { get; set; } private bool VirtualTimebaseProperty = false; /* this may come later [NinjaScriptProperty] [Display(Name="Process Historical?", Description="Use Virtual Timebase from data", Order=3, GroupName="Parameters")] public bool VirtualTimebaseProperty { get; set; } */ #endregion private int indexCounter = 0; private int zeroIndex = 0; // assigned in config private int inventoryIndex = 0; private bool isReady = false; private bool isInitialized = false; #region Startup and Initialization private void initOnce() { if ( isInitialized ) return; // once only isInitialized = true; customRootPath = Environment.ExpandEnvironmentVariables( "%USERPROFILE%\\Documents\\NinjaTrader 8\\bin\\Custom\\"); myIntegrator = new VolumePriceExpiryIntegrator(this, 1000, retentionSecs); // allocate objects if (myPropertyReader==null) { myPropertyReader = new SimplePropertyReader(this); myPropertyReader.setPropertiesFileNameDotTxt(FileName); myPropertyReader.setPropertiesSamplingSecs(30); // file sampling frequency // now assign default property values myPropertyReader.setKeyValue(riskThresholdTicksProp, ""+riskThresholdTicks); // stringify it myPropertyReader.setKeyValue(retentionSecsProp, ""+retentionSecs); // stringify it myPropertyReader.setKeyValue(multiplierProp, ""+multiplier); myPropertyReader.setKeyValue(bigLotMinimumProp, ""+bigLotMinimum); myPropertyReader.setKeyValue(taperSizeProp, ""+taperSize); myPropertyReader.setKeyValue(superSpikeThresholdRatioProp, ""+superSpikeThresholdRatio); myPropertyReader.start(); } onBarUpdateLimiter = new SimpleRateLimiter(this, 20); // milliseconds superSpikeReferenceLimiter = new SimpleRateLimiter(this, 10000); } #endregion #region Synchronization and PROPERTY Management // begin of synchronization support // use getUpdateMutex() private object updateMutex = new object(); private SimplePropertyReader myPropertyReader = null; // asynchronous property updates use this // synchronization mechanism // in Visual Studio, of course, we'd use // interface definitions, and multiple // files, but we're staying in a single file // for usage within the NinjaScript environment protected object getUpdateMutex() { return updateMutex; } // CONTROL VARIABLES MODIFIABLE BY PROPERTY MANAGER private static string retentionSecsProp = "RETENTION_SECONDS"; private int retentionSecs = 60; // default 60 seconds private static string multiplierProp = "MULTIPLIER"; private double multiplier = 1.0; // default unity multiplier private static string riskThresholdTicksProp = "RISK_THRESHOLD"; private double riskThresholdTicks = 3.0; // private static string bigLotMinimumProp = "BIGLOT_MINIMUM"; private int bigLotMinimum = 2; // default unity multiplier private static string taperSizeProp = "TAPER_SIZE"; private bool taperSize = true; // default useTaper private static string superSpikeThresholdRatioProp = "SUPER_SPIKE_THRESHOLD_RATIO"; private static double superSpikeThresholdRatio = 4.0; protected void updatePropertiesCallback() { if (myPropertyReader==null) return; lock(getUpdateMutex()) { // a thread which already holds the lock // will always get the same lock again.... // the file-based property manager will // firstly lock(getUpdateMutex()) // and then it will call this routine // where your property values can // be updated safely // myPropertyReader.setKeyValue(retentionSecsProp, retentionSecs); // myPropertyReader.setKeyValue(multiplierProp, multiplier); // myPropertyReader.setKeyValue(bigLotMinimumProp, bigLotMinimum); //if ( !myPropertyReader.hasChanged()) return; try { int retentionSecsTmp = myPropertyReader.getIntegerProperty(retentionSecsProp); if (retentionSecsTmp<10) retentionSecsTmp = 10; // ten sec min retentionSecs = retentionSecsTmp; myIntegrator.setRetentionSecs(retentionSecs); // change the Integrator } catch (Exception ex) { ninjaPrint2(retentionSecsProp+" error!", true); } // do nothing try { double multiplierTmp = myPropertyReader.getDoubleProperty(multiplierProp); if (multiplierTmp<0.1) multiplierTmp = 0.1; multiplier = multiplierTmp; // used in OnBarUpdate } catch (Exception ex) {ninjaPrint2( multiplierProp+" error!", true);} // do nothing try { int bigLotMinimumTmp = myPropertyReader.getIntegerProperty(bigLotMinimumProp); if (bigLotMinimumTmp<1) bigLotMinimumTmp = 1; bigLotMinimum = bigLotMinimumTmp; // used when reporting Signed Trade Volumes } catch (Exception ex) {ninjaPrint2( bigLotMinimumProp+" error!", true);} // do nothing try { taperSize = myPropertyReader.getBooleanProperty(taperSizeProp); myIntegrator.setUseTaper(taperSize); } catch (Exception ex) {ninjaPrint2( taperSizeProp+" error!", true);} // do nothing try { riskThresholdTicks = myPropertyReader.getDoubleProperty(riskThresholdTicksProp); if (riskThresholdTicks<0) riskThresholdTicks = 0; // no negatives } catch (Exception ex) {ninjaPrint2( riskThresholdTicksProp+" error!", true);} // do nothing try { double superSpikeThresholdRatioTmp = myPropertyReader.getDoubleProperty(superSpikeThresholdRatioProp); if (superSpikeThresholdRatioTmp<1) superSpikeThresholdRatioTmp = 1; superSpikeThresholdRatio = superSpikeThresholdRatioTmp; // used when reporting Super Spike Trade Volumes } catch (Exception ex) {ninjaPrint2( superSpikeThresholdRatioProp+" error!", true);} // do nothing } // end of any property value updates } // end of synchronization support #endregion #region NinjaScript Output Window printing private static bool USE_NINJA_PRINT = true; // suppress ALL printing private void ninjaPrint(string msg) { // primary tab if (!USE_NINJA_PRINT) return; PrintTo = PrintTo.OutputTab1; Print(msg); } private void ninjaPrint2(string msg) { ninjaPrint2(msg, false); // do not force } private void ninjaPrint2(string msg, bool force) { // secondary tab if ( !force && !USE_NINJA_PRINT) return; PrintTo = PrintTo.OutputTab2; Print(msg); } #endregion #region The Trade Price Volume (tapering) Integrator private VolumePriceExpiryIntegrator myIntegrator = null; /* This version will enforce a strict moving time window, and optionally taper the trailing portion of that window gradually to zero to avoid artifacts associated with size leaving the time window*/ protected /*inner*/ class VolumePriceExpiryIntegrator { private double[] volumeArray = null; private double[] priceArray = null; private DateTime[] timestampArray = null; // these are treated as "circular buffers" bool useTaperFlag = false; // TODO int size = 0; int ptr = 0; int _expiryMsecs = 60000; TradeFlowRisk parent = null; public VolumePriceExpiryIntegrator(TradeFlowRisk parentRef, int sizeArg, int expirySecsArg) { // constructor parent = parentRef; size = sizeArg; _expiryMsecs = 1000 * expirySecsArg; volumeArray = new double[size]; priceArray = new double[size]; timestampArray = new DateTime[size]; DateTime now = parent.DateTimeUtcNow(); for(int i=0; i<size; i++) { volumeArray[i] = 0; priceArray[i] = 0; timestampArray[i] = now; } } public void addVolumePriceData(double volumeArg, double priceArg) { // RULE: ptr should always index the // next valid array index, not the // most recent one. ptr always // increments. if (ptr>=size) { ptr = 0; } timestampArray[ptr] = parent.DateTimeUtcNow(); priceArray[ptr] = priceArg; // including price volumeArray[ptr++] = volumeArg; // increment after usage if (ptr>=size) { ptr = 0; } } public void setRetentionSecs(int seconds) { _expiryMsecs = seconds * 1000; } public void setUseTaper(bool flag) { useTaperFlag = flag; } public double getAverageVolumeUnsigned() { return getSumVolumeImpl(0, true); // return AVERAGE UNSIGNED VOLUME } public double getAverageVolumeUnsigned(int msecsOverride) { return getSumVolumeImpl(msecsOverride, true); // return AVERAGE UNSIGNED VOLUME } public double getAverageVolumeInterval(int ageMsecsOverride) { return getSumVolumeImpl(ageMsecsOverride, true); // return AVERAGE UNSIGNED VOLUME } public double getSumVolume() { return getSumVolumeImpl(0, false); // not average, SIGNED Integration } private int lastSumVolume = 1; public int getLastSumVolume() { return lastSumVolume; } // this has a dual-purpose, as well as optional retention interval override private double getSumVolumeImpl(int ageMsecsOverride, bool returnAvgUnsignedFlag) { // must work backwards in time from most // recent until items are expired, or we // traverse the entire circular ring buffer DateTime now = parent.DateTimeUtcNow(); // TODO, use Virtual Timebase; this is REAL TIME ONLY double sum = 0; int n = 0; int tmpPtr = ptr - 1; // most recent if (tmpPtr<0) tmpPtr = size - 1; else if (tmpPtr>=size) tmpPtr = 0; // array bounds int _expiryMsecsLocal = _expiryMsecs; if ( ageMsecsOverride > 0) { // substitute the given window time _expiryMsecsLocal = ageMsecsOverride; } int taperBeginAge = (int)(_expiryMsecsLocal * 0.75); // if tapering... bool expired = false; // always using tmpPtr here ! while( !expired && tmpPtr!=ptr) { DateTime ts = timestampArray[tmpPtr]; double item = volumeArray[tmpPtr--]; // REMINDER: we are moving from most recent // backward in time here !! if (tmpPtr<0) tmpPtr = size - 1; // array bounds TimeSpan elapsed = ( now - ts ); int ageMsecs = (int)elapsed.TotalMilliseconds; if ( ageMsecs > _expiryMsecsLocal ) { expired = true; continue; // and exit loop } if ( tmpPtr == ptr ) { // we exhaused all of our circular buffer (unlikely) expired = true; continue; // and exit loop } double finalItem = item; if (useTaperFlag) { // taper last 25% of time window // OK, so we'll taper the oldest 25% of the // time window trade volume linearly down to zero // e.g. Expiry might be 10000, so begin tapering beyond age 7500 if (ageMsecs > taperBeginAge) { // e.g. from 7500 msecs into a 10000 max window int numerator = _expiryMsecsLocal - ageMsecs; // progressing 2500 toward zero if (numerator<0) numerator = 0; int denominator = _expiryMsecsLocal - taperBeginAge; // e.g. 10000 - 7500 = 2500 double ratio = (double)numerator / (double)denominator; finalItem *= ratio; // LINEAR TAPER } } sum += (returnAvgUnsignedFlag? Math.Abs(finalItem) : finalItem); // summation of signed (tapered) trade volume OR unsigned Average volume n++; } double returnValue = 0; if (returnAvgUnsignedFlag) { lastSumVolume = (int)sum; // we won't use this returnValue = ( n>0? sum/n : 0 ); } else { // signed summation returnValue = (int)sum; lastSumVolume = (int)Math.Abs(sum); } return returnValue; } public static int VWAP_POSITIVE_VOLUMES = 3; // arbitrary constants public static int VWAP_NEGATIVE_VOLUMES = 4; public static int VWAP_ALL_VOLUMES = 5; public double getVwapPrice(int calcMethod) { // throws Exception for ZERO valued prices // must work backwards in time from most // recent until items are expired, or we // traverse the entire circular ring buffer DateTime now = parent.DateTimeUtcNow(); // this is REAL TIME ONLY double sumV = 0; // SUM( abs(volume) ) double sumPV = 0; // SUM (price times abs(volume)) int n = 0; int tmpPtr = ptr - 1; // most recent if (tmpPtr<0) tmpPtr = size - 1; //else if (tmpPtr>=size) tmpPtr = 0; // array bounds int taperBeginAge = (int)(_expiryMsecs * 0.75); // if tapering... bool expired = false; // always using tmpPtr here ! while( !expired && tmpPtr!=ptr) { DateTime ts = timestampArray[tmpPtr]; double price = priceArray[tmpPtr]; double volume = volumeArray[tmpPtr--]; if (tmpPtr<0) tmpPtr = size - 1; // array bounds if (price==0) { //parent.ninjaPrint2("Price is zero"); continue; } if (calcMethod!=VWAP_ALL_VOLUMES) { if (calcMethod==VWAP_POSITIVE_VOLUMES) { if (volume<0) continue; // ignore NEGATIVE } else if (calcMethod==VWAP_NEGATIVE_VOLUMES) { if (volume>0) continue; // ignore POSITIVE } } volume = Math.Abs(volume); // always Positive weightings // REMINDER: we are moving from most recent // backward in time here !! TimeSpan elapsed = ( now - ts ); int ageMsecs = (int)elapsed.TotalMilliseconds; if ( ageMsecs > _expiryMsecs ) { expired = true; continue; // and exit loop } if ( tmpPtr == ptr ) { // we exhaused all of our circular buffer (unlikely) expired = true; continue; // and exit loop } double taperedVolume = volume; if (false) { // do not taper weights (useTaperFlag) { // taper last 25% of time window // OK, so we'll taper the oldest 25% of the // time window trade volume linearly down to zero // e.g. Expiry might be 10000, so begin tapering beyond age 7500 if (ageMsecs > taperBeginAge) { // e.g. from 7500 msecs into a 10000 max window int numerator = _expiryMsecs - ageMsecs; // progressing 2500 toward zero if (numerator<0) numerator = 0; int denominator = _expiryMsecs - taperBeginAge; // e.g. 10000 - 7500 = 2500 double ratio = (double)numerator / (double)denominator; taperedVolume *= ratio; // LINEAR TAPER } } ++n; sumV += taperedVolume; // summation of signed (tapered) trade volume sumPV += ( taperedVolume * price ); } // a zero return PRICE means no data double vwap = sumV==0 ? 0 : sumPV / sumV; //parent.ninjaPrint2("vwap data items: "+n+" price: "+vwap); return vwap; } } #endregion #region Calculations and Utilities private static double EXP = 0.75; // exponent less than 1 and >0.5 // non-linear signed size compression private static double compressSizeMinOneSigned(double valueArg) { if ( EXP > 1 ) EXP = 1.0; // no compression at all if ( EXP < 0.5 ) EXP = 0.5; // square root bool isNegative = ( valueArg < 0 ); double absValue = Math.Abs(valueArg); // calculate on positive if (absValue < 1) absValue = 1; // enforce minimum of size 1 (contract) double x = Math.Pow(absValue + 0.0001, EXP); // just adding a smidgeon // 1^0 = 1 so 1 should be the minimum possible if (x < 1) x = 1; return ( isNegative? -x : x); // restore the sign } #endregion #region Shutdown de-Initialization private void deInit() { // must properly De-Allocate ALL user allocated objects // at least ones explicitly allocated at the Outer Scope // not counting things like Plot objects; and anything // which is running a Thread needs to have that stopped myIntegrator = null; if (myPropertyReader!= null) { myPropertyReader.stop(); // kill the thread myPropertyReader = null; } if (onBarUpdateLimiter!=null) { onBarUpdateLimiter = null; } if (superSpikeReferenceLimiter!=null) { superSpikeReferenceLimiter = null; } if (myIntegrator !=null) { myIntegrator = null; } // stop and deallocate objects } #endregion #region Time Based Functions // add the ability to use a Virtual Time Base, rather // than "now" or Real Time, so DateTime.UtcNow is // not used for a Virtual Timebase; rather the function // call DateTimeUtcNow() returning the last Virtual // time; which should help the Indicator to respond // to Historical Data as well as Real Time private DateTime lastVirtualNowPrivate = DateTime.MinValue; // not initialized private Object timeMutex = new object(); private static bool ENABLE_VIRTUAL_TIMEBASE = false; private bool isTimeBaseInitialized() { lock(timeMutex) { if ( !ENABLE_VIRTUAL_TIMEBASE ) return true; else return ( lastVirtualNowPrivate != DateTime.MinValue ); } } private void setVirtualTime(DateTime newTime) { lock(timeMutex) { lastVirtualNowPrivate = newTime; } } private DateTime DateTimeUtcNow() { lock(timeMutex) { if (ENABLE_VIRTUAL_TIMEBASE && VirtualTimebaseProperty) { //throw new Exception("Virtual Time Base NOT Implemented yet..."); if ( !isTimeBaseInitialized() ){ //throw new Exception("Virtual Time Base No Timestamp yet..."); return lastVirtualNowPrivate; } else { return lastVirtualNowPrivate; } } else { return DateTime.UtcNow; } } } #endregion private static bool LOG_RAW_VOLUME = true; private int lastRawTradeVol = 0; #region Processing Trade Flow private void reportTrade(int tradeRawVol, double price) { if (Math.Abs(tradeRawVol) < bigLotMinimum) return; // reject volume if ( LOG_RAW_VOLUME ) { if (tradeRawVol>0) { ninjaPrint("Retail Buy "+tradeRawVol+" @"+price); } else { ninjaPrint(" Retail Sell "+tradeRawVol+" @"+price); } } lastRawTradeVol = tradeRawVol; // do not compress volume here, due to Risk Value calculations double rawVol = tradeRawVol; //compressSizeMinOneSigned( tradeRawVol ); // Super Spike Volumes test here //if (Math.Abs(compressedVol)>superSpikeThreshold*getLastReferenceVolume()) { if (Math.Abs(rawVol)> (superSpikeThresholdRatio*getLastReferenceVolume())) { if (rawVol>0) wasSuperSpikeBuy = true; else wasSuperSpikeSell = true; } reportToIntegrator( rawVol, price ); } private void reportToIntegrator(double volArg, double priceArg) { myIntegrator.addVolumePriceData(volArg, priceArg); } #endregion private double lastBidPrice = 0; private double lastAskPrice = 0; private string customRootPath = null; private static int REFERENCE_MSECS = 10 * 60 * 1000; // 5 mins in Msecs private double lastSuperSpikeReferenceVolume = 50; // start with this pending warm-up private SimpleRateLimiter superSpikeReferenceLimiter = null; public double getSuperSpikeReferenceVolume() { if ( !isPrimed() || !superSpikeReferenceLimiter.allowUpdate() ) { return lastSuperSpikeReferenceVolume; } // when primed, the Super Spike reference volume updates lastSuperSpikeReferenceVolume = myIntegrator.getAverageVolumeInterval(REFERENCE_MSECS) ; // remember the last Volume summation, to use for weighting for Risk Value lastIntegratorAvgVolume = myIntegrator.getLastSumVolume(); return lastSuperSpikeReferenceVolume; } private int lastIntegratorAvgVolume = 1; private double superSpikeThreshold = 8; private double lastReferenceVolume = 5; // updated private int lastReferenceVolumeCounter = 0; private double getLastReferenceVolume() { ++lastReferenceVolumeCounter; if (lastReferenceVolumeCounter>10000) lastReferenceVolumeCounter=0; if (lastReferenceVolumeCounter%50==0) { lastReferenceVolume = myIntegrator.getAverageVolumeUnsigned(REFERENCE_MSECS); //ninjaPrint2("lastRefVol: "+lastReferenceVolume, true); } return lastReferenceVolume; } #region Market Depth Updates (used for timestamp only) protected sealed override void OnMarketDepth(MarketDepthEventArgs marketDepthUpdate) { if (DBG_MARKET_DATA) ninjaPrint2("DBG OnMarketDepth..."); this.setVirtualTime(marketDepthUpdate.Time); // using timestamps virtual time only } #endregion #region OnMarketUpdate callback /* NinjaScript/Language Reference/Common/OnMarketData() If used with TickReplay, please keep in mind Tick Replay ONLY replays the Last market data event, and only stores the best inside bid/ask price at the time of the last trade event. You can think of this as the equivalent of the bid/ask price at the time a trade was reported. As such, historical bid/ask market data events (i..e, bid/ask volume) DO NOT work with Tick Replay. To obtain those values, you need to use a historical bid/ask series separately from TickReplay through OnBarUpdate(). More information can be found under Developing for Tick Replay. */ private static bool DBG_MARKET_DATA = false; protected sealed override void OnMarketData(MarketDataEventArgs marketDataUpdate) { if (DBG_MARKET_DATA) ninjaPrint2("DBG OnMarketData..."); if ( !isReady ) { if (DBG_MARKET_DATA) ninjaPrint2("DBG OnMarketData is not ready..."); return; } try { lock(getUpdateMutex()) { // wait to allow property updates now if (DBG_MARKET_DATA) ninjaPrint2("DBG OnMarketData inside lock..."); MarketDataType type = marketDataUpdate.MarketDataType; this.setVirtualTime(marketDataUpdate.Time); if (type==MarketDataType.Last) { // Trade if (DBG_MARKET_DATA) ninjaPrint2("DBG OnMarketData Last..."); if (lastBidPrice==0) { if (DBG_MARKET_DATA) ninjaPrint2("DBG OnMarketData No Bid yet..."); return; } if (lastAskPrice==0) { if (DBG_MARKET_DATA) ninjaPrint2("DBG OnMarketData No Ask yet..."); return; } if (lastAskPrice <= lastBidPrice) return; double tradePrice = marketDataUpdate.Price; // price of the Trade int tradeVolume = (int)marketDataUpdate.Volume; lastBidPrice = marketDataUpdate.Bid; // carried with the Last event ! lastAskPrice = marketDataUpdate.Ask; // carried with the Last event ! double mid = 0.5 * ( lastAskPrice + lastBidPrice ); // mid price if (tradePrice < mid ) { // trade toward BID negative Retail Sell reportTrade( -tradeVolume, tradePrice ); } else { // trade toward ASK/OFFER positive Retail Buy reportTrade( tradeVolume, tradePrice ); } } else if (type==MarketDataType.Bid) { if (DBG_MARKET_DATA) ninjaPrint2("DBG OnMarketData Bid..."+marketDataUpdate.Price); lastBidPrice = marketDataUpdate.Price; // BID price } else if (type==MarketDataType.Ask) { if (DBG_MARKET_DATA) ninjaPrint2("DBG OnMarketData Ask..."+marketDataUpdate.Price); lastAskPrice = marketDataUpdate.Price; // ASK/OFFER price } } // end lock } catch(Exception ex) { ninjaPrint2("EXCEPTION OnMarketData..."+ex.Message); } if (DBG_MARKET_DATA) ninjaPrint2("DBG OnMarketData end..."); return; } #endregion private int primedCounter = 0; private bool isPrimedFlag = false; private bool isPrimed() { return isPrimedFlag; } // wait some Bars before some functions active // in Real Time, often you want some data before // doing some stuff private void incrementPrimedCount() { if (isPrimed()) return; ++primedCounter; if (primedCounter>20) isPrimedFlag = true; } private double lastInventory = 0; int tagCounter = 0; private string getShortTag() { return "SRisk"+tagCounter++; } private string getLongTag() { return "LRisk"+tagCounter++; } private Brush shortRiskBrushAlt = Brushes.MediumBlue; private Brush longRiskBrushAlt = Brushes.DarkOrange; private Brush shortRiskBrush = Brushes.Lime; private Brush longRiskBrush = Brushes.Red; private Brush superSpikeSellBrush = Brushes.Fuchsia; private Brush superSpikeBuyBrush = Brushes.Green; private bool wasSuperSpikeBuy = false; private bool wasSuperSpikeSell = false; private SimpleRateLimiter onBarUpdateLimiter = null; // init and deInit private static bool DBG = false; protected sealed override void OnBarUpdate() { if ( !isReady ) return; // must have if ( !isTimeBaseInitialized() ) return; // Real Time versus Virtual Time bool doUpdate = onBarUpdateLimiter.allowUpdate(); if (IsFirstTickOfBar) doUpdate = true; // allow on first tick of bar if ( !doUpdate ) return; // rate limited incrementPrimedCount(); if ( !isPrimed() ) return; // isFirstTickOfBar can be used to clear // any inter-bar cumulative variables lock(getUpdateMutex()) { // no property updates now Values[zeroIndex][0] = 0; double inventory = myIntegrator.getSumVolume(); lastInventory = inventory; double multipliedInventory = inventory * multiplier; Values[inventoryIndex][0] = multipliedInventory; if (Math.Abs(multipliedInventory) < 30) return; // minimum inventory required ! // place DOTs against Price indicating possible RISK { double tickSize = this.TickSize; // price increment of 1 price tick double midPrice = ( 0.5 * ( lastBidPrice+lastAskPrice )); if (wasSuperSpikeBuy) { wasSuperSpikeBuy = false; Draw.Dot( this, getShortTag(), true, 0, (midPrice+2*tickSize), superSpikeBuyBrush); } if (wasSuperSpikeSell) { wasSuperSpikeSell = false; Draw.Dot( this, getLongTag(), true, 0, (midPrice-2*tickSize), superSpikeSellBrush); } //double riskTicks = 0; if (multipliedInventory>0) { // positive inventory "Short Risk" double vwapPositive = myIntegrator.getVwapPrice(VolumePriceExpiryIntegrator.VWAP_POSITIVE_VOLUMES); if ( vwapPositive != 0 ) { // valid; a zero price is INVALID double deltaTicksActual = (midPrice - vwapPositive) / tickSize; if (DBG) { ninjaPrint2("vwapPositive price: "+vwapPositive); } if (deltaTicksActual>=riskThresholdTicks) { double deltaTicks = deltaTicksActual; if (deltaTicks>30) deltaTicks=30; // for display Draw.Dot( this, getShortTag(), true, 0, (midPrice+4+(0.15*deltaTicks)), AltColor?shortRiskBrushAlt:shortRiskBrush); double absDeltaTicks = Math.Abs(deltaTicksActual); double pointValue = Instrument.MasterInstrument.PointValue; int ticksPerPoint = (int)( 1 / tickSize ); double deltaPoints = absDeltaTicks * ( tickSize / 1 ); int riskValue = (int)(deltaPoints * pointValue * lastIntegratorAvgVolume); Draw.Text( this, "RiskTag","___"+riskValue, 0, (midPrice+6+(0.15*deltaTicks))); } } } else { // negative inventory "Long Risk" double vwapNegative = myIntegrator.getVwapPrice(VolumePriceExpiryIntegrator.VWAP_NEGATIVE_VOLUMES); if (vwapNegative != 0) { // valid; a zero price is INVALID double deltaTicksActual = ( vwapNegative - midPrice ) / tickSize; if (DBG) { ninjaPrint2(" vwapNegative price: "+vwapNegative); } if (deltaTicksActual>=riskThresholdTicks) { double deltaTicks = deltaTicksActual; if (deltaTicks>30) deltaTicks=30; // for display Draw.Dot( this, getLongTag(), true, 0, (midPrice-4 - (0.15*deltaTicks)), AltColor?longRiskBrushAlt:longRiskBrush); double absDeltaTicks = Math.Abs(deltaTicksActual); double pointValue = Instrument.MasterInstrument.PointValue; int ticksPerPoint = (int)( 1 / tickSize ); double deltaPoints = absDeltaTicks * ( tickSize / 1 ); int riskValue = (int)(deltaPoints * pointValue * lastIntegratorAvgVolume); Draw.Text( this, "RiskTag", "___"+riskValue, 0, (midPrice-6+(0.15*deltaTicks))); } } } } } } #region periodic SimplePropertyReader private static bool DBG_KV = false; // debug only protected class SimplePropertyReader { // USAGE: Create object // setPropertiesFileNameDotTxt // use name "." (default directory) \\ double backslash // desiredName.txt // if file exists, then // the given properties will be ignored, and the // properties dictionary contents taken from the file // otherwise, the given properties are written to // a newly created properties file // setPropertiesSamplingSecs // call start() once only // on termination, call stop() TradeFlowRisk parent = null; public SimplePropertyReader(TradeFlowRisk parentArg) { parent = parentArg; parent.ninjaPrint2("SimplePropertyReader constructor"); // client should set all properties, prior to start() } // please call start only once public void start() { parent.ninjaPrint2("SimplePropertyReader start()"); ThreadStart ts = new ThreadStart(propertiesReaderLoop); Thread runner = new Thread(ts); isRunning = true; runner.Start(); } private string fileName = "myProperties.txt"; // if used, must do this BEFORE start() public void setPropertiesFileNameDotTxt(string fileNameArg) { fileName = fileNameArg; } // if used, do this BEFORE calling start(); public void setPropertiesSamplingSecs(int secondsArg) { propertiesSampleDelayMsecs = secondsArg * 1000; } private bool isRunning = false; private int propertiesSampleDelayMsecs = 30000; public void stop() { isRunning = false; notifyUpdate(); // break thread wait; and exit } private object threadWaitObject = new object(); private bool isNotifiedLocked = false; public void notifyUpdate() { lock(threadWaitObject) { isNotifiedLocked = true; Monitor.Pulse(threadWaitObject); } } private void propertiesReaderLoop() { try { parent.ninjaPrint("propertiesReaderLoop started"); while (isRunning) { try { lock (threadWaitObject) { if (!isNotifiedLocked) { Monitor.Wait(threadWaitObject, propertiesSampleDelayMsecs, true); // every XX secs or sooner isNotifiedLocked = false; } if (!isRunning) continue; // thread will stop // read property file if ( !updateFromPropertiesFile()) continue; // error in file or first time lock(parent.getUpdateMutex()) { parent.updatePropertiesCallback(); } } } catch(Exception ex) { } } } catch(Exception ex) { // nothing } } // client must set ALL properties and values BEFORE start() public void setKeyValue(string keyArg, string value) { lock (propertiesDictionary) { string key = keyArg.ToUpper().Trim(); string strValue = "" + value; if (propertiesDictionary.ContainsKey(key)) { propertiesDictionary[key] = strValue; } else { propertiesDictionary.Add(key, strValue); } } } private Dictionary<string, string> propertiesDictionary = new Dictionary<string, string>(); public int getIntegerProperty( string key) { return (int)getDoubleProperty(key); } public double getDoubleProperty(string keyArg) { string key = keyArg.ToUpper().Trim(); double retDbl = 0; lock (propertiesDictionary) { if (propertiesDictionary.ContainsKey(key)) { try { retDbl = Double.Parse(propertiesDictionary[key]); } catch(Exception ex) { throw; } // propogate exception to caller } } return retDbl; } public bool getBooleanProperty(string keyArg) { string key = keyArg.ToUpper().Trim(); bool retBool = false; lock (propertiesDictionary) { if (propertiesDictionary.ContainsKey(key)) { try { retBool = Boolean.Parse(propertiesDictionary[key].Trim().ToLower()); } catch (Exception ex) { throw; } // propogate exception to caller } } return retBool; } private bool updateFromPropertiesFile() { parent.ninjaPrint("updateFromPropertiesFile called... "); // if the properties file doesn't exist, then we create one // and populate with the provided properties in form xxx=yyy bool successful = true; string fullFilePath = parent.customRootPath + fileName; if ( !File.Exists(fullFilePath)) { // default dir; backslash is special escape char try { // write a new one //parent.ninjaPrint("updateFromPropertiesFile new file "+fullFilePath); using (StreamWriter outputFile = new StreamWriter(fullFilePath)) { outputFile.WriteLine("# comment line"); lock (propertiesDictionary) { foreach (var item in propertiesDictionary) { string key = item.Key; string value = item.Value; outputFile.WriteLine(key + "=" + value); } outputFile.Close(); } } } catch(Exception ex) { successful = false; parent.ninjaPrint("Exception creating file: "+ex.Message);} return successful; } // existing file, need to re-read and update our properties dictionary if (File.Exists(fullFilePath)) { // should definitely exist !!! // write a new one try { using (StreamReader inputFile = new StreamReader(fullFilePath)) { lock (propertiesDictionary) { //propertiesDictionary.Clear(); string lineRaw = " "; while ((lineRaw = inputFile.ReadLine()) != null) { if (lineRaw==null) continue; string trimmedLine = lineRaw.Trim(); if (trimmedLine.Length == 0) continue; if (trimmedLine.StartsWith("#")) continue; // skip comment if (!trimmedLine.Contains("=")) continue; string[] splitString = trimmedLine.Split('='); // no equals character if ( splitString.Length != 2) continue; // must be key=value string key = splitString[0].ToUpper().Trim(); string value = splitString[1].ToLower().Trim(); // e.g. FALSE to false if (DBG_KV) { parent.ninjaPrint("Key: "+key+" value: "+value); } if ( !propertiesDictionary.ContainsKey(key) ) { propertiesDictionary.Add(key, value); } else { propertiesDictionary[key] = value; } } // handled by the "using" logic... inputFile.Close(); } } // end of the "using" block } catch(Exception ex2) { successful = false; parent.ninjaPrint("Exception: "+ex2.Message);} } return successful; } } // end class #endregion #region Simple Rate Limiter class SimpleRateLimiter { protected static Random randomizer = new Random(); int msecs = 10; // default bool useRandom = false; // slightly randomize interval which // would untentionally de-synchronize any large collection of RateLimiters public void setUseRandom(bool flag) { useRandom = flag; if (!useRandom) deltaMsecsRandom = 0; } private DateTime lastUpdate = DateTime.MinValue; TradeFlowRisk parent = null; public SimpleRateLimiter(TradeFlowRisk parentRef, int msecsArg) { parent = parentRef; msecs = msecsArg; } // randomly return -1..+1 protected static int nextDelta() { lock (randomizer) { double ran = randomizer.NextDouble(); if (ran < 0.3333) { return -1; } else if (ran < 0.6666) { return 0; } return 1; } } public void setMsecsDelay(int msecsArg) { int _msecs = msecsArg; if (_msecs < 1) _msecs = 1; msecs = _msecs; } private int deltaMsecsRandom = 0; public bool allowUpdate() { if ( !parent.isTimeBaseInitialized()) return false; DateTime now = parent.DateTimeUtcNow(); TimeSpan elapsed = (now - lastUpdate); if (elapsed.TotalMilliseconds < msecs + deltaMsecsRandom) return false; lastUpdate = now; if (useRandom) deltaMsecsRandom = nextDelta(); return true; } } #endregion } } #region NinjaScript generated code. Neither change nor remove. namespace NinjaTrader.NinjaScript.Indicators { public partial class Indicator : NinjaTrader.Gui.NinjaScript.IndicatorRenderBase { private TradeFlowRisk[] cacheTradeFlowRisk; public TradeFlowRisk TradeFlowRisk(string fileName, bool altColor) { return TradeFlowRisk(Input, fileName, altColor); } public TradeFlowRisk TradeFlowRisk(ISeries<double> input, string fileName, bool altColor) { if (cacheTradeFlowRisk != null) for (int idx = 0; idx < cacheTradeFlowRisk.Length; idx++) if (cacheTradeFlowRisk[idx] != null && cacheTradeFlowRisk[idx].FileName == fileName && cacheTradeFlowRisk[idx].AltColor == altColor && cacheTradeFlowRisk[idx].EqualsInput(input)) return cacheTradeFlowRisk[idx]; return CacheIndicator<TradeFlowRisk>(new TradeFlowRisk(){ FileName = fileName, AltColor = altColor }, input, ref cacheTradeFlowRisk); } } } namespace NinjaTrader.NinjaScript.MarketAnalyzerColumns { public partial class MarketAnalyzerColumn : MarketAnalyzerColumnBase { public Indicators.TradeFlowRisk TradeFlowRisk(string fileName, bool altColor) { return indicator.TradeFlowRisk(Input, fileName, altColor); } public Indicators.TradeFlowRisk TradeFlowRisk(ISeries<double> input , string fileName, bool altColor) { return indicator.TradeFlowRisk(input, fileName, altColor); } } } namespace NinjaTrader.NinjaScript.Strategies { public partial class Strategy : NinjaTrader.Gui.NinjaScript.StrategyRenderBase { public Indicators.TradeFlowRisk TradeFlowRisk(string fileName, bool altColor) { return indicator.TradeFlowRisk(Input, fileName, altColor); } public Indicators.TradeFlowRisk TradeFlowRisk(ISeries<double> input , string fileName, bool altColor) { return indicator.TradeFlowRisk(input, fileName, altColor); } } } #endregion

So, how to use this? Create a NinjaScript Indicator "TradeFlowRisk"
generate; and then we'll Select All and delete everything.
Paste the Entirety of the code above into the editor; and compile.
It should compile...

The Name of the Indicator will show as "TradeFlowRisk 1.0"

Obviously, as I'm encouraging all of you to immerse yourself
in some coding; many changes can be made. If you don't know
what you're doing, I'd recommend creating an Indicator of a
different class name, and display name, and fiddling with it.

There is code in there which is "on the way toward" providing a
Virtual Timebase, which would potentially catch Historical backfill,
and moves things more in the direction of Backtesting; but it's
not activated.

My view, as I've said before, is that Backtesting is not that valuable,
given that you can observe and use the Indicator a few days and
get a good idea how it works to help with "situational awareness"
in supporting turning points of the market.

True Believers in Backtesting will Disagree. Let the "beotching"
begin... LOL

Let me know if there are bugs; I'll update if you find anything
important. You have the weekend to study the code; and make
your list of complaints about it... As I've said before, I use a
"Java" formatting style; which is not the C# standard. That's
a debate where I'm on one side; and you are probably on the
other !

hyperscalper

I had a little bit of time to convert the last code HyperScalper posted to using NinjaScript properties. It should work historically with TickReplay as well. I avoided any changes to the logic and data structures. Some of this code can be simplified by just relying on tick replay when you want to reconstruct things or change settings. Doing so may also yield performance gains. For now though it's not worth worrying about unless users are running into difficulty adding features or backtesting.

TWDsje 

I had a little bit of time to convert the last code HyperScalper posted to using NinjaScript properties. It should work historically with TickReplay as well. I avoided any changes to the logic and data structures. Some of this code can be simplified by just relying on tick replay when you want to reconstruct things or change settings. Doing so may also yield performance gains. For now though it's not worth worrying about unless users are running into difficulty adding features or backtesting.

Wow, Thanks Seth. I'm looking forward...

As I was looking into getting accurate Inventory during
either simple Historical backfill, never mind Backtesting...
I seemed to me that special code actions had to be taken to
allow for the Bid/Ask/Trade categorization to be as accurate
as the RealTime case.
Perhaps not; so I'm looking forward to your code.

THANKS AGAIN for taking the time !

hyperscalper

hyperscalper 

Wow, Thanks Seth. I'm looking forward... As I was looking into getting accurate Inventory during either simple Historical backfill, never mind Backtesting... I seemed to me that special code actions had to be taken to allow for the Bid/Ask/Trade categorization to be as accurate as the RealTime case. Perhaps not; so I'm looking forward to your code. THANKS AGAIN for taking the time ! hyperscalper

Well you are correct that it will not be quite as accurate as the real time case. The differences are pretty insignificant though. You can test for yourself by running it on two charts, and then refreshing one chart to cause it to reload data using tick replay to compare. In my testing it's something that you're only going to be able to see over the course of a thousand trades or something like that. You gain far more in return in my opinion. Plus when you actually run it you're only going to be using live data anyways. So you're not going to lose any accuracy when you actually use it to trade live.

MEA CULPA; I REMOVED VOLUME COMPRESSION

In my latest codebase, I removed the Volume compression;
to Raw volumes are used into the Integrator.

So, to match up your revision against mine, I just made
the following change, eliminating the compression:

 

Code

// removal of trade volume compression double compressedVol = tradeRawVol; //compressSizeMinOneSigned( tradeRawVol );

Thanks again; will be interesting to see whether my
concerns about accuracy were just overblown...

hyperscalper

VIRTUAL TIME VERSUS REAL TIME

In my thinking about some of these issues, my last codebase
goes "some way" toward establishing a Virtual Time Base.

So, instead of RealTime-only usage of:

DateTime now = DateTime.UtcNow;

One would want to "virtualize" this, using a function like
this pseudo-code:

 

Code

private DateTime realOrVirtualDateTimeUtcNow() { if (usingRealTime) { return DateTime.UtcNow(); } else { // return a DateTime from the Historical playback } }

Everywhere that DateTime.UtcNow() happens; it would have to
call the "virtualized function" etc...

Seems to me that would be necessary; or am I wrong ??

hyperscalper

hyperscalper 

VIRTUAL TIME VERSUS REAL TIME .... Everywhere that DateTime.UtcNow() happens; it would have to call the "virtualized function" etc... Seems to me that would be necessary; or am I wrong ??

Your conclusion looks correct to me.

HedgePlay