NexusFi: Find Your Edge


Home Menu

 





Speeding up NinjaTrader with a RAM drive (ramdisk)


Discussion in NinjaTrader

Updated
      Top Posters
    1. looks_one Big Mike with 20 posts (13 thanks)
    2. looks_two Zondor with 18 posts (24 thanks)
    3. looks_3 kronie with 14 posts (2 thanks)
    4. looks_4 bob7123 with 12 posts (15 thanks)
      Best Posters
    1. looks_one pawnbroker with 3.4 thanks per post
    2. looks_two Zondor with 1.3 thanks per post
    3. looks_3 bob7123 with 1.3 thanks per post
    4. looks_4 Big Mike with 0.7 thanks per post
    1. trending_up 60,977 views
    2. thumb_up 94 thanks given
    3. group 41 followers
    1. forum 111 posts
    2. attach_file 6 attachments




 
Search this Thread

Speeding up NinjaTrader with a RAM drive (ramdisk)

  #91 (permalink)
 
kronie's Avatar
 kronie 
NYC + NY / USA
 
Experience: Advanced
Platform: "I trade, therefore, I AM!"; Theme Song: "Atomic Dog!"
Trading: EMD, 6J, ZB
Posts: 796 since Oct 2009


Big Mike View Post
Yes. BTW, after more uptime my cache hit ratio is 75%

Sent from my LG Optimus G Pro

@bigmike


I left a few Q's as replies, above, please see if there's an answer.

I got instructions from one of the vendors, that said, the first step with a ramdisk was to create an image file, then create or allocate the available or desired additional ram as a ramdisk and associate that ram to that image, then proceed.

this detail is important, as it has something more to do with linkage and save/restore features. did you encounter this issue?

(btw - on a very serious note! congrats on achieving your goals early (under 40's) in life and in moving to where you have already found your peace! We all wish to achieve similar results, but you actually did, so we live vicariously through your experience!)

Reply With Quote

Can you help answer these questions
from other members on NexusFi?
Quantum physics & Trading dynamics
The Elite Circle
NT7 Indicator Script Troubleshooting - Camarilla Pivots
NinjaTrader
ZombieSqueeze
Platforms and Indicators
REcommedations for programming help
Sierra Chart
Trade idea based off three indicators.
Traders Hideout
 
  #92 (permalink)
 Koepisch 
@ Germany
 
Experience: Beginner
Platform: NinjaTrader
Broker: Mirus Futures/Zen-Fire
Trading: FDAX
Posts: 569 since Nov 2011
Thanks Given: 440
Thanks Received: 518

I also use an ramdisk but i don't know in detail if the windows file cache system exclude the files at the ramdisk. I know that there is a flag (Win32_Volume|DriveType|6) in the internals that mark a volume as ramdisk, but i'm not sure if the cache subsystem take care of that (don't cache anything from that volume!!!!). Has anyone more knowledge about that topic?

Releated to BigMike's solution i've looked at the product and read that this work's as a sector level cache. Due to the lower level interface i think that the file cache is still active.

Reply With Quote
  #93 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,469 since Jun 2009
Thanks Given: 33,247
Thanks Received: 101,669


My recommendation is not to use the ramdisk, but instead to use PrimoCache. There is a free trial.

Mike



Join the free Markets Chat beta: one platform, all the trade rooms!

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
  #94 (permalink)
 
kronie's Avatar
 kronie 
NYC + NY / USA
 
Experience: Advanced
Platform: "I trade, therefore, I AM!"; Theme Song: "Atomic Dog!"
Trading: EMD, 6J, ZB
Posts: 796 since Oct 2009


Big Mike View Post
My recommendation is not to use the ramdisk, but instead to use PrimoCache. There is a free trial.

Mike

thanks


that's a very good recommendation, after more research, the level-2 cache settings are recommended as being alternate storage, such as an SSD or USB drive (although one has to decide whether that's useful)

ASRock includes a completely unlocked version of a ramdisk implementation that provides auto load, auto save, auto shutdown upon using its implementation. Fortunately they're the mfgr of my mobo(s)

one tech support email response made the instructions mentioned previously in this thread seem childishly simplistic. He stated first one establishes an image file; then a ramdisk and associates that image to it; then begins to profile it.

However, in the use of Ninja with its static location files (essential for operation), these solutions require more steps to implement. One would need to junction selected files and folders within the sub-folder "\db" of the ninja files in order to achieve the performance gains that one trader claimed.

Clearly ninja needs serious re-work to achieve Sierra Charts, AT Traders and other package(s) performance standards and reliability. So many work around(s) and maintenance steps, and the gap having been closed on indicators and complex indicator setups; Ninja is in serious challenge from other packages, and hence, threads like these only serve to detract instead of attract.

I can share more details on this journey, but they only serve to retrace the faults and flaws of using the speed up steps. I am sure there is a working methodology, but it has not been as advertised earlier.

@bigmike; using the UIM (unused memory) just below the 4gb threshhold is useful, however, be prepared for the occasional system BSOD or terminal hang/hung screen. I will say that it was noteworthy that you brought that additional speed up approach to our attention

Reply With Quote
  #95 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,469 since Jun 2009
Thanks Given: 33,247
Thanks Received: 101,669

I am making the assumption everyone already has an SSD, no hard drives. And that everyone is using x64 Windows with at minimum 8GB of RAM.

If not, you should make those a priority over anything else.

Mike



Join the free Markets Chat beta: one platform, all the trade rooms!

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
  #96 (permalink)
 Koepisch 
@ Germany
 
Experience: Beginner
Platform: NinjaTrader
Broker: Mirus Futures/Zen-Fire
Trading: FDAX
Posts: 569 since Nov 2011
Thanks Given: 440
Thanks Received: 518

Due to the loyalty to my current RamDisk software producer SuperSpeed i will check the SuperCache product first, if it can improve my overall performance (for my daily working stuff). It has an interesting mirror mode, which creates a 1:1 copy of a volume into the RAM.

Reply With Quote
  #97 (permalink)
 
kronie's Avatar
 kronie 
NYC + NY / USA
 
Experience: Advanced
Platform: "I trade, therefore, I AM!"; Theme Song: "Atomic Dog!"
Trading: EMD, 6J, ZB
Posts: 796 since Oct 2009

If you've made it this far, you've gone too far!

Here's help

Attached Thumbnails
Speeding up NinjaTrader with a RAM drive (ramdisk)-link-shell-extentions-creating-juntions.pdf  
Reply With Quote
  #98 (permalink)
 
kronie's Avatar
 kronie 
NYC + NY / USA
 
Experience: Advanced
Platform: "I trade, therefore, I AM!"; Theme Song: "Atomic Dog!"
Trading: EMD, 6J, ZB
Posts: 796 since Oct 2009


Big Mike View Post
My recommendation is not to use the ramdisk, but instead to use PrimoCache. There is a free trial.

Mike

Primo Cache didn't work and crashed my system more than once; operating system. I wasn't able to discern any appreciable improvement in caching or speed of access to the internals of Ninja, hence faster loading and handling times. @bigmike - didn't you move off of Ninja quite sometime ago, and adopted another (superior) or comparable trading / charting system? Perhaps those improvements from that trading platform were able to take advantage of the operating implementation of a super cache.

After some serious work effort and findings, and sloshing through the technical internals and contradictions of what seemingly was a simple endeavour, I will condense my findings here:

A) in the main NinjaTrader directory, I had no success (ninja aborted upon start up) with moving to Ramdisk the sub-directories of:

\\cache
\\templates
\\workspaces
\\tmp

b) in the sub-directory of "\db" the data files, I had success in moving these components to a persistent RamDisk, and verify that they were being preserved during shutdown and restored after start-up (of the operating system)

\NinjaTrader.sdf
\db\cache
\db\data

** note: just having the ".sdf" in ram really sped things up substantially
** note: \workspace -- would be excellent to junction
** note: \templates -- would be double excellent since this is where the hanging occurred

If you have the additional ram to dedicate to a ramdisk, then assigning the entire "\db" file would be easier. This occurs when you have above 8GB overall system ram, because some "\db" sub-directories easily acquire over 12gb themselves, not counting what the GomFolder would be, if you use Gomi stuff. So, a system memory of 16GB would be appropriate, especially when your "\db" is in the 12gb range.


more results to be posted,

one thing so far, is the conclusion that it was all worth the efforts!

Reply With Quote
  #99 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,469 since Jun 2009
Thanks Given: 33,247
Thanks Received: 101,669


kronie View Post
Primo Cache didn't work and crashed my system more than once; operating system. I wasn't able to discern any appreciable improvement in caching or speed of access to the internals of Ninja, hence faster loading and handling times. @bigmike - didn't you move off of Ninja quite sometime ago, and adopted another (superior) or comparable trading / charting system? Perhaps those improvements from that trading platform were able to take advantage of the operating implementation of a super cache.

Yikes. I'm just guessing it has something to do with your 32-bit OS and <4GB of RAM (if I remember right from your prior posts) and you trying to use the extended area of RAM. None of these apply to me luckily. I have never tried anything like that (using RAM outside the boundary)

Did you look up the blue screen codes and contact the author to see what he said?

I still use Ninja for backtesting, and the point of the cache is really just for Ninja as I don't notice any slow down with any other activities on my system. I would say it has made a considerable improvement, but don't have time right now to do any type of actual benchmarking with it on and off.

Mike



Join the free Markets Chat beta: one platform, all the trade rooms!

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote
  #100 (permalink)
 
kronie's Avatar
 kronie 
NYC + NY / USA
 
Experience: Advanced
Platform: "I trade, therefore, I AM!"; Theme Song: "Atomic Dog!"
Trading: EMD, 6J, ZB
Posts: 796 since Oct 2009



Big Mike View Post
Yikes. I'm just guessing it has something to do with your 32-bit OS and <4GB of RAM (if I remember right from your prior posts) and you trying to use the extended area of RAM. None of these apply to me luckily. I have never tried anything like that (using RAM outside the boundary)

Did you look up the blue screen codes and contact the author to see what he said?

I still use Ninja for backtesting, and the point of the cache is really just for Ninja as I don't notice any slow down with any other activities on my system. I would say it has made a considerable improvement, but don't have time right now to do any type of actual benchmarking with it on and off.

Mike

win 7 x64 ultimate

actually the ramdisk lumps in the 4GB memory stick of the 8GB system memory, with all the other un-accessed memory of the 1st 4GB memory stick, and manages it as one block, surprisingly

fortunately I didn't go as far a BSOD, but the resulting system lock up caused the power button to be the only way of recovery and subsequent invokation of Ninja to be avoided, effectively locking up the system

either way to Sunday, these are in the hind-sight mirror a few miles back.

worthy of note, because Ninja is not consistent and has not been in their reference to files and locations within their directories and sub-directories, simply moving enmasse entire sub-components of these directories does not work. Junctions of a higher level directory "in theory" makes access to any lower level directory redirected. Because of the direct references in code, xml and other means within their code, and other templates, processes and schemas, its next to impossible to move the "bin" master directory to ram. By extention, its impossible to move the entire NT directory structure to ram. However, because of indirect access to the "\db" directories, moving that entire master directory works almost without problems.


more findings to be released, I'm 85% there.



one thing I noticed is when launching or changing or initalizing / serializing a workspace, Ninja gets stuck in a repetitive loop without handlers and as such will remain there adnauseum or for hours unless forced shutdown. watching the handlers and processes shows that pattern

Reply With Quote




Last Updated on August 6, 2020


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts