NexusFi: Find Your Edge


Home Menu

 





How can I convert a .csv file to text for import into NT?


Discussion in NinjaTrader

Updated
      Top Posters
    1. looks_one spinnybobo with 5 posts (3 thanks)
    2. looks_two sburtt with 4 posts (0 thanks)
    3. looks_3 Big Mike with 3 posts (1 thanks)
    4. looks_4 gunsnmoney with 2 posts (0 thanks)
      Best Posters
    1. looks_one NinjaTrader with 4 thanks per post
    2. looks_two sam028 with 1 thanks per post
    3. looks_3 spinnybobo with 0.6 thanks per post
    4. looks_4 Big Mike with 0.3 thanks per post
    1. trending_up 18,827 views
    2. thumb_up 9 thanks given
    3. group 8 followers
    1. forum 18 posts
    2. attach_file 6 attachments




 
Search this Thread

How can I convert a .csv file to text for import into NT?

  #11 (permalink)
 
Big Mike's Avatar
 Big Mike 
Manta, Ecuador
Site Administrator
Developer
Swing Trader
 
Experience: Advanced
Platform: Custom solution
Broker: IBKR
Trading: Stocks & Futures
Frequency: Every few days
Duration: Weeks
Posts: 50,470 since Jun 2009
Thanks Given: 33,249
Thanks Received: 101,673

You can also look here:


And ultimately, eventually, here:


Mike



Join the free Markets Chat beta: one platform, all the trade rooms!

We're here to help: just ask the community or contact our Help Desk

Quick Links: Change your Username or Register as a Vendor
Searching for trading reviews? Review this list
Lifetime Elite Membership: Sign-up for only $149 USD
Exclusive money saving offers from our Site Sponsors: Browse Offers
Report problems with the site: Using the NexusFi changelog thread
Follow me on Twitter Visit my NexusFi Trade Journal Reply With Quote

Can you help answer these questions
from other members on NexusFi?
MC PL editor upgrade
MultiCharts
Strategy stop orders partially filled
EasyLanguage Programming
Pivot Indicator like the old SwingTemp by Big Mike
NinjaTrader
Better Renko Gaps
The Elite Circle
Cheap historycal L1 data for stocks
Stocks and ETFs
 
  #12 (permalink)
 
spinnybobo's Avatar
 spinnybobo 
Crete, IL/USA
 
Experience: Intermediate
Platform: NinjaTrader, Mt4
Broker: Tradestation/Tradestation, NinjaTrader, FXCM and Tallinex
Trading: ES, CL, EUR/USD, TF
Posts: 173 since Aug 2009
Thanks Given: 105
Thanks Received: 61


sburtt View Post
Hi spinnybobo,

thanks for taking the time to post this useful material. When trying to replicate what you are doing I have an issue. When I run the code in basicNinjaParse.jave , I get the following message error:

Error: Main method not found in class basicNinjaParse, please define the main method as:
public static void main(String[] args)

Are you able to help me to understand what I'm doing wrong

Thanks for your time, John

Hi John.
It cannot find the main method for some reason. Did you name it .java or .jave ?
Just remember that there is only 1 public class per .java file and the name of the .java file has to be exactly the same name of the public class. So, I have an example below of what everything would be if the 1st class is called public class Data. With this, the fail name is: Data.java

The 2nd file is public class getData. With this, the main method is inside of it and is the starting point of the program. The file name is getData.java

Here is the 1st class

 
Code
public class Data {
	
	private String Date;
	private String Open;
	private String High;
	private String Low;
	private String Close;
	private String Volume;
	
	//default constructor  having the same name of the class
	public Data(){
		this.Date = null;
		this.Open = null;
		this.High = null;
		this.Low = null;
		this.Close = null;
		this.Volume = null;
		
	}//regular constructor
	public Data(String Date, String Open, String High, String Low, String Close, String Volume)
	{
		this.Date = Date;
		this.Open = Open;
		this.High = High;
		this.Low = Low;
		this.Close = Close;
		this.Volume = Volume;
	}
	public String getDate(){
		return this.Date;
	}
	public String getOpen(){
		return this.Open;
	}
	public String getHigh(){
		return this.High;
	}
	public String getLow(){
		return this.Low;
	}
	public String getClose(){
		return this.Close;
	}
	public String getVolume(){
		return this.Volume;
	}
}
2nd class

 
Code
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.*;
import java.util.Vector;
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.File;

//these will work once you import the jar file from www.csvreader.com 
import com.csvreader.CsvReader;
import com.csvreader.CsvWriter;


public class getData {

	private static Vector<Data> dailyData;
	
	public void inputData(){
		//the vector to hold the data
		dailyData = new Vector<Data>();
		try
		{
		CsvReader products = new CsvReader("C:\\AAPL.txt");
		products.readHeaders();
			while(products.readRecord())
			{
				dailyData.add(new Data(
					products.get("Date"),
					products.get("Open"),
					products.get("High"),
					products.get("Low"),
					products.get("Close"),
					products.get("Volume")));
				
				
			}
		products.close();
		}
		catch(FileNotFoundException e){
			e.printStackTrace();
		}
		catch(IOException e){
			e.printStackTrace();
		}
		
	}
	public void outputData(){
		//nothing yet
		for(int i = 0; i<dailyData.size();i++){
			System.out.println(dailyData.get(i).getDate()+";"+dailyData.get(i).getOpen()+";"+dailyData.get(i).getHigh()+";"+dailyData.get(i).getLow()+";"+dailyData.get(i).getClose()+";"+dailyData.get(i).getVolume());
		}
		String outputFile = "AAPL_ninjaTrader.txt";
		//check if file exists
		boolean alreadyExists = new File(outputFile).exists();
		try
		{
			CsvWriter csvOutput = new CsvWriter(new FileWriter(outputFile, true), ';');
			
			//if the file didn't already exist, then we need to write out the header line
			if(!alreadyExists)
			{
				csvOutput.write("Date");
				csvOutput.write("Open");
				csvOutput.write("High");
				csvOutput.write("Low");
				csvOutput.write("Close");
				csvOutput.write("Volume");
				csvOutput.endRecord();
			}
			//else assume that the file already has the correct header line
			//write out a few records
			for(int i = 0; i<dailyData.size();i++){
				csvOutput.write(dailyData.get(i).getDate());
				csvOutput.write(dailyData.get(i).getOpen());
				csvOutput.write(dailyData.get(i).getHigh());
				csvOutput.write(dailyData.get(i).getLow());
				csvOutput.write(dailyData.get(i).getClose());
				csvOutput.write(dailyData.get(i).getVolume());
				
				csvOutput.endRecord();
			}
			csvOutput.close();
		}
	
		catch(IOException e)
		{
			e.printStackTrace();
		}
	}
	public static void main(String args []){
		getData load = new getData();
		load.inputData();
		load.outputData();
		
	}
}
I uploaded a file called AAPL.TXT
Put this in the C://AAPL.TXT directory. Create a project in Eclipse called whatever you want. Add a class to the project called Data. Add another class called getData

Copy and paste both into Eclipse.

There are errors because the .jar file is not added. Go to the zip file attached and unzip. Go to the subdirectory of Eclipse project you created and create a folder called "lib".

Copy and paste the javacsv.jar to this folder. Now go back to Eclipse and expand the project from the Package Explorer view. Right click on project and hit "refresh" so you see the lib folder. Right click on top of JRE Package Explorer and go to Build Path, and then Configure Build Path. Add JARs... and find the lib folder in the project and then click on the javacsv.jar and press OK.

The errors should disappear. Left click on the Project folder in Eclipse and then find the Green button called Run. Hit it and it should run. If you go back to the subdirectory for Eclipse and go into your project, you should see a file called AAPL_ninjaTrader.txt

This has the formated data ready to be imported into Ninja Trader. Just erase the first line of the header and erase the space as well. It should work fine.

Let me know if you have a problem

Spencer

Attached Files
Elite Membership required to download: lib.zip
Elite Membership required to download: AAPL.TXT
Follow me on Twitter Reply With Quote
  #13 (permalink)
 sburtt 
London, UK
 
Experience: Advanced
Platform: NinjaTrader
Trading: EUR/USD
Posts: 58 since Jul 2012
Thanks Given: 3
Thanks Received: 4



Big Mike View Post
You can also look here:


And ultimately, eventually, here:


Mike

Thanks Mike very useful, but I see there has been little progress up to now. However, it is a great idea, and I would really appreciate if somebody could develop a generic converter.

Reply With Quote
  #14 (permalink)
 sburtt 
London, UK
 
Experience: Advanced
Platform: NinjaTrader
Trading: EUR/USD
Posts: 58 since Jul 2012
Thanks Given: 3
Thanks Received: 4


spinnybobo View Post
Hi John.
It cannot find the main method for some reason. Did you name it .java or .jave ?
Just remember that there is only 1 public class per .java file and the name of the .java file has to be exactly the same name of the public class. So, I have an example below of what everything would be if the 1st class is called public class Data. With this, the fail name is: Data.java

The 2nd file is public class getData. With this, the main method is inside of it and is the starting point of the program. The file name is getData.java

Here is the 1st class

 
Code
public class Data {
	
	private String Date;
	private String Open;
	private String High;
	private String Low;
	private String Close;
	private String Volume;
	
	//default constructor  having the same name of the class
	public Data(){
		this.Date = null;
		this.Open = null;
		this.High = null;
		this.Low = null;
		this.Close = null;
		this.Volume = null;
		
	}//regular constructor
	public Data(String Date, String Open, String High, String Low, String Close, String Volume)
	{
		this.Date = Date;
		this.Open = Open;
		this.High = High;
		this.Low = Low;
		this.Close = Close;
		this.Volume = Volume;
	}
	public String getDate(){
		return this.Date;
	}
	public String getOpen(){
		return this.Open;
	}
	public String getHigh(){
		return this.High;
	}
	public String getLow(){
		return this.Low;
	}
	public String getClose(){
		return this.Close;
	}
	public String getVolume(){
		return this.Volume;
	}
}
2nd class

 
Code
import java.io.BufferedReader;
import java.io.FileInputStream;
import java.io.FileNotFoundException;
import java.io.IOException;
import java.io.InputStream;
import java.io.InputStreamReader;
import java.net.*;
import java.util.Vector;
import java.io.BufferedWriter;
import java.io.FileWriter;
import java.io.File;

//these will work once you import the jar file from www.csvreader.com 
import com.csvreader.CsvReader;
import com.csvreader.CsvWriter;


public class getData {

	private static Vector<Data> dailyData;
	
	public void inputData(){
		//the vector to hold the data
		dailyData = new Vector<Data>();
		try
		{
		CsvReader products = new CsvReader("C:\\AAPL.txt");
		products.readHeaders();
			while(products.readRecord())
			{
				dailyData.add(new Data(
					products.get("Date"),
					products.get("Open"),
					products.get("High"),
					products.get("Low"),
					products.get("Close"),
					products.get("Volume")));
				
				
			}
		products.close();
		}
		catch(FileNotFoundException e){
			e.printStackTrace();
		}
		catch(IOException e){
			e.printStackTrace();
		}
		
	}
	public void outputData(){
		//nothing yet
		for(int i = 0; i<dailyData.size();i++){
			System.out.println(dailyData.get(i).getDate()+";"+dailyData.get(i).getOpen()+";"+dailyData.get(i).getHigh()+";"+dailyData.get(i).getLow()+";"+dailyData.get(i).getClose()+";"+dailyData.get(i).getVolume());
		}
		String outputFile = "AAPL_ninjaTrader.txt";
		//check if file exists
		boolean alreadyExists = new File(outputFile).exists();
		try
		{
			CsvWriter csvOutput = new CsvWriter(new FileWriter(outputFile, true), ';');
			
			//if the file didn't already exist, then we need to write out the header line
			if(!alreadyExists)
			{
				csvOutput.write("Date");
				csvOutput.write("Open");
				csvOutput.write("High");
				csvOutput.write("Low");
				csvOutput.write("Close");
				csvOutput.write("Volume");
				csvOutput.endRecord();
			}
			//else assume that the file already has the correct header line
			//write out a few records
			for(int i = 0; i<dailyData.size();i++){
				csvOutput.write(dailyData.get(i).getDate());
				csvOutput.write(dailyData.get(i).getOpen());
				csvOutput.write(dailyData.get(i).getHigh());
				csvOutput.write(dailyData.get(i).getLow());
				csvOutput.write(dailyData.get(i).getClose());
				csvOutput.write(dailyData.get(i).getVolume());
				
				csvOutput.endRecord();
			}
			csvOutput.close();
		}
	
		catch(IOException e)
		{
			e.printStackTrace();
		}
	}
	public static void main(String args []){
		getData load = new getData();
		load.inputData();
		load.outputData();
		
	}
}
I uploaded a file called AAPL.TXT
Put this in the C://AAPL.TXT directory. Create a project in Eclipse called whatever you want. Add a class to the project called Data. Add another class called getData

Copy and paste both into Eclipse.

There are errors because the .jar file is not added. Go to the zip file attached and unzip. Go to the subdirectory of Eclipse project you created and create a folder called "lib".

Copy and paste the javacsv.jar to this folder. Now go back to Eclipse and expand the project from the Package Explorer view. Right click on project and hit "refresh" so you see the lib folder. Right click on top of JRE Package Explorer and go to Build Path, and then Configure Build Path. Add JARs... and find the lib folder in the project and then click on the javacsv.jar and press OK.

The errors should disappear. Left click on the Project folder in Eclipse and then find the Green button called Run. Hit it and it should run. If you go back to the subdirectory for Eclipse and go into your project, you should see a file called AAPL_ninjaTrader.txt

This has the formated data ready to be imported into Ninja Trader. Just erase the first line of the header and erase the space as well. It should work fine.

Let me know if you have a problem

Spencer

spinnybobo,
thank you very much! Sorry it took me time to revert, I've become a dad and the last 3 weeks have been very intense for me.. I've managed to run the program following your instruction.


If it doesn't bother you too much, I would really appreciate your help with an issue I have
.

I have a lot of 1minute data for different securities in the following format:

"Date","Time","O","H","L","C","U","D"
04/20/1998,0919,1.0982,1.0982,1.0982,1.0982,1,0
04/20/1998,0920,1.0982,1.0982,1.0982,1.0982,1,0
04/20/1998,0922,1.0980,1.0981,1.0980,1.0981,1,1
04/20/1998,0924,1.0983,1.0983,1.0983,1.0983,2,0

Would you be able to quickly change a few variables in the code you've sent me in order to recognize input data such as the one above (that for your guide is EUR/USD) and generate an output file able to run in ninjatrader?

I would really be thankful and happy to share the data I have if you would like.

Again thanks, John
input.txt

Reply With Quote
  #15 (permalink)
 
spinnybobo's Avatar
 spinnybobo 
Crete, IL/USA
 
Experience: Intermediate
Platform: NinjaTrader, Mt4
Broker: Tradestation/Tradestation, NinjaTrader, FXCM and Tallinex
Trading: ES, CL, EUR/USD, TF
Posts: 173 since Aug 2009
Thanks Given: 105
Thanks Received: 61


sburtt View Post
spinnybobo,
thank you very much! Sorry it took me time to revert, I've become a dad and the last 3 weeks have been very intense for me.. I've managed to run the program following your instruction.


If it doesn't bother you too much, I would really appreciate your help with an issue I have
.

I have a lot of 1minute data for different securities in the following format:

"Date","Time","O","H","L","C","U","D"
04/20/1998,0919,1.0982,1.0982,1.0982,1.0982,1,0
04/20/1998,0920,1.0982,1.0982,1.0982,1.0982,1,0
04/20/1998,0922,1.0980,1.0981,1.0980,1.0981,1,1
04/20/1998,0924,1.0983,1.0983,1.0983,1.0983,2,0

Would you be able to quickly change a few variables in the code you've sent me in order to recognize input data such as the one above (that for your guide is EUR/USD) and generate an output file able to run in ninjatrader?

I would really be thankful and happy to share the data I have if you would like.

Again thanks, John
Attachment 86838

Hey John,

Hey, congrats on becoming a Dad! that is awesome. wow, you must be a busy guy right now.
So, did you get your data from PiTrading.com ? I got a bunch of data from them for my Masters Thesis and wrote code to parse the data, but later learned they had a special Ninja Trader parser they built for their clients.

I ask because your data seems to be in the same format as their data you can just use their parser. You download the attached file, unzip it, then put the ascii2nt.exe on your desktop. Then the the input.txt file and drag it over the ascii2nt.exe file. Then take your finger off the mouse. Then you should see a file show up on your desktop called inputNT.exe

This is now in the NinjaTrader format. The other thing this parser does is it checks for logical errors like if the low is greater then the high. It basically changes it so it is logical, but it might not be the exact price that happened in the past. This is needed because ninja will not let you import a dataset if it has logical errors.

Also, you don't double click on the asii2nt.exe file. It is not a GUI (graphical user interface) and does not install. It just runs when you drop a text file over it. So it is portable.

Last, is the last line U and D important? So, I attached the output inputNT.txt and the last line should be volume and as you can see it took the 2nd to the last column and made that volume, and cut off the very last line.

If this is ok then great. If it is not and you wanted something different, then a program would have to be written for this. I did write something for 1 min parsing that I initially used, but later started using their parser.

thanks
Spencer

Attached Files
Elite Membership required to download: ascii2nt.zip
Elite Membership required to download: inputNT.txt
Follow me on Twitter Reply With Quote
Thanked by:
  #16 (permalink)
 sburtt 
London, UK
 
Experience: Advanced
Platform: NinjaTrader
Trading: EUR/USD
Posts: 58 since Jul 2012
Thanks Given: 3
Thanks Received: 4


spinnybobo View Post
Hey John,

Hey, congrats on becoming a Dad! that is awesome. wow, you must be a busy guy right now.
So, did you get your data from PiTrading.com ? I got a bunch of data from them for my Masters Thesis and wrote code to parse the data, but later learned they had a special Ninja Trader parser they built for their clients.

I ask because your data seems to be in the same format as their data you can just use their parser. You download the attached file, unzip it, then put the ascii2nt.exe on your desktop. Then the the input.txt file and drag it over the ascii2nt.exe file. Then take your finger off the mouse. Then you should see a file show up on your desktop called inputNT.exe

This is now in the NinjaTrader format. The other thing this parser does is it checks for logical errors like if the low is greater then the high. It basically changes it so it is logical, but it might not be the exact price that happened in the past. This is needed because ninja will not let you import a dataset if it has logical errors.

Also, you don't double click on the asii2nt.exe file. It is not a GUI (graphical user interface) and does not install. It just runs when you drop a text file over it. So it is portable.

Last, is the last line U and D important? So, I attached the output inputNT.txt and the last line should be volume and as you can see it took the 2nd to the last column and made that volume, and cut off the very last line.

If this is ok then great. If it is not and you wanted something different, then a program would have to be written for this. I did write something for 1 min parsing that I initially used, but later started using their parser.

thanks
Spencer


Spencer it works! Thanks! This really saved me loads of time.. I see you're US based, I live in London, If you ever come to London please send me a message I will invite you out for dinner, it's the least I can do.

I'm not sure about the source of data, I has been given to me by a friend that works in a Bank (btw I work in a bank to), however I will ask him and let you know. Thanks

Reply With Quote
  #17 (permalink)
 
spinnybobo's Avatar
 spinnybobo 
Crete, IL/USA
 
Experience: Intermediate
Platform: NinjaTrader, Mt4
Broker: Tradestation/Tradestation, NinjaTrader, FXCM and Tallinex
Trading: ES, CL, EUR/USD, TF
Posts: 173 since Aug 2009
Thanks Given: 105
Thanks Received: 61


sburtt View Post
Spencer it works! Thanks! This really saved me loads of time.. I see you're US based, I live in London, If you ever come to London please send me a message I will invite you out for dinner, it's the least I can do.

I'm not sure about the source of data, I has been given to me by a friend that works in a Bank (btw I work in a bank to), however I will ask him and let you know. Thanks

Hey John

hey, glad it worked out for ya.
thanks, I will keep it in mind when it comes time for me to visit London :-)
until then, take care
Spencer

Follow me on Twitter Reply With Quote
Thanked by:
  #18 (permalink)
 
ElChacal's Avatar
 ElChacal 
Houston, TX
 
Experience: Intermediate
Platform: NinjaTrader TWS Barchart
Broker: IB / IB
Trading: SPY, QQQ
Posts: 418 since Nov 2014
Thanks Given: 473
Thanks Received: 278

@NinjaTrader could it be this easy instead?

https://www.multicharts.com/trading-software/index.php/OpenInt

Reply With Quote
  #19 (permalink)
MDNATOUROK
dublin irelanf
 
Posts: 1 since Feb 2013
Thanks Given: 2
Thanks Received: 0

Would you also kindly be able to do this for daily data also?

But from the following formats

"Date","O","H","L","C","V"

3/19/2020,5080.6,5181,4942.4,5151.6,1963412200

and

"Date","Time","O","H","L","C","V"

2014.07.01,00:00,2.1898,2.1853,2.1868,2.1858,81

both to convert to Ninja trader format

Thank You.





spinnybobo View Post
Hey John,

Hey, congrats on becoming a Dad! that is awesome. wow, you must be a busy guy right now.
So, did you get your data from PiTrading.com ? I got a bunch of data from them for my Masters Thesis and wrote code to parse the data, but later learned they had a special Ninja Trader parser they built for their clients.

I ask because your data seems to be in the same format as their data you can just use their parser. You download the attached file, unzip it, then put the ascii2nt.exe on your desktop. Then the the input.txt file and drag it over the ascii2nt.exe file. Then take your finger off the mouse. Then you should see a file show up on your desktop called inputNT.exe

This is now in the NinjaTrader format. The other thing this parser does is it checks for logical errors like if the low is greater then the high. It basically changes it so it is logical, but it might not be the exact price that happened in the past. This is needed because ninja will not let you import a dataset if it has logical errors.

Also, you don't double click on the asii2nt.exe file. It is not a GUI (graphical user interface) and does not install. It just runs when you drop a text file over it. So it is portable.

Last, is the last line U and D important? So, I attached the output inputNT.txt and the last line should be volume and as you can see it took the 2nd to the last column and made that volume, and cut off the very last line.

If this is ok then great. If it is not and you wanted something different, then a program would have to be written for this. I did write something for 1 min parsing that I initially used, but later started using their parser.

thanks
Spencer


Reply With Quote




Last Updated on April 19, 2020


© 2024 NexusFi™, s.a., All Rights Reserved.
Av Ricardo J. Alfaro, Century Tower, Panama City, Panama, Ph: +507 833-9432 (Panama and Intl), +1 888-312-3001 (USA and Canada)
All information is for educational use only and is not investment advice. There is a substantial risk of loss in trading commodity futures, stocks, options and foreign exchange products. Past performance is not indicative of future results.
About Us - Contact Us - Site Rules, Acceptable Use, and Terms and Conditions - Privacy Policy - Downloads - Top
no new posts