Welcome to NexusFi: the best trading community on the planet, with over 150,000 members Sign Up Now for Free
Genuine reviews from real traders, not fake reviews from stealth vendors
Quality education from leading professional traders
We are a friendly, helpful, and positive community
We do not tolerate rude behavior, trolling, or vendors advertising in posts
We are here to help, just let us know what you need
You'll need to register in order to view the content of the threads and start contributing to our community. It's free for basic access, or support us by becoming an Elite Member -- see if you qualify for a discount below.
-- Big Mike, Site Administrator
(If you already have an account, login at the top of the page)
Summary: this post gives my subjective speed measurements on portfolio-level backtests performed on Tradestation's (TS) Portfolio Maestro and Multicharts 64-bit (MC).
If you don't want details, just skip to the end of the post for these results.
Hi all,
I'm a new member to the forums, so I thought it could be useful to give my 2 cents, and maybe it can help some beginner like I am.
I am trying to create:
a. a few properly backtested strategies
b. for the US equity markets
c. to trade systematically, ie no discretionary trading
Towards this goal, I think that what is important for me, software-wise is:
1. portfolio-level backtesting capability
2. that is as fast as possible, in order to spend time testing ideas and not waiting for tests to finish
3. accurate backtesting calculations
4. good client service
I started using TS a few months ago, when I saw that they released the Portfolio Maestro platform, which is basically the add-on to TS that performs portfolio-level backtesting.
I spent these months learning easylanguage (it's easy) and creating and testing strategies in Portfolio Maestro, and being a generally happy customer.
2 weeks ago, and after many positive comments from these forums, I thought I should give MC a test as well.
I compare the highlights of my experiences with each platform:
General
TS offers a 1-stop shop (charting, testing, data feed, broker), whereas MC is a 'shell' that needs to be filled with a data feed from another company (they have many partners), and also to be connected to a broker - MC is not a broker, either.
At first I thought 'why bother with MC if TS offers everything in a single, simple package? I have enough problems trying to find strategies, I don't need to fiddle with 10 pieces of software for the next 2 years to make them work together'.
After these last 2 weeks, I reviewed my position: except for the differences that I note below, I found that making MC work is very easy - yes, it took me a few days to sort some things out, but it's a negligible investment in time compared to everything else that needs to be done to call yourself a trader.
Charting
I am not big on charting since I don't focus on discretionary trading, but as far as I use it, both platforms are very similar - even the menu choices etc. My feeling is that both are quite comprehensive.
The general look and feel of both platforms is also very similar.
Data feed
TS has its own data, so there's nothing to do. Plug and play.
MC has its QuoteManager (QM) app, which is the database holding your data. You need to connect QM to your preferred data feed, and then QM interfaces automatically with the other MC programs (charting, portfolio testing etc).
This is the part that frustrated me a little bit for a few days. Once you understand how everything works it's very simple and very handy to have that database to admin and store your data locally, but it did take me 2-3 days of wondering if I am walking on the right path.
Support
TS has manuals, webinars, and a huge forums community. It's very likely that your question has been addressed by someone else previously. This, however, does not apply to Portfolio Maestro - being a new product, there is practically no documentation, no forums history, so you are on your own - it's not nuclear physics, but little support nonetheless.
MC lacks in this area - written documentation is almost nowhere to be found (in my experience). I found some info on its Wiki page, some info in various forums left and right.. I understand that MC is now making an effort to provide guidance and education, but it's a shame that this excellent product doesn't have adequate material behind it yet.
To counter my negative comments on MC support, were the live text chat and email support guys (and gals?) - I was amazed! Live chat was friendly, super supportive, they would remotely log in to your computer and explain solutions. Email support was given in 24 hours maximum, compared to TS's many days, or even never.
Overall, I would say this is a tie - these platforms are not difficult to learn to use, to begin with.
easylanguage
As you all know, both platforms use the same language, which was initially built by TS.
Ironically, for me MC worked better! There is a simple reason for this: TS's Portfolio Maestro (its portfolio-level backtesting program) was purchased and assimilated into TS this year. Unfortunately the current results are less Borg and more like ..an ugly stupid species. Portfolio Maestro does not support all easylanguage functions! I don't envy TS engineers who are working on this project..I had to find some weird ways to make Portfolio Maestro work like the TS chart backtester. With MC, things work just properly! I am not doing anything fancy, but what I do worked without a hitch.
By the way, both the code and the results I got, from both TS and MC backtesters, are exactly (to the cent) the same (16 years of daily data tested, about 2600 trades, using the TS data feed in MC).
Portfolio backtesting
My comments below refer to:
1. a strategy of about 100 lines of code
2. run on 1800 equity symbols
3. from Jan 1997 - Nov 2012, daily data
4. on an i7 laptop with 6gb of RAM (the HP Envy 17)
The strategy that I tested and report here, was tested in Portfolio Maestro many many painful times, and there are two comments to make:
a. Portfolio Maestro's local database cannot keep all the downloaded data - so you have to reload a lot (or all of it if your computer was just turned on) every time! For me, downloading 1800 symbols of daily data (not intraday) takes about half an hour.
b. Including the aforementioned half hour, the total backtest takes about 2 and a half hours.
c. OK, one more point: I tried an optimisation on Portfolio Maestro, with 125 combinations, I had to reboot the computer after 2 days because there was no end in sight (and I tried this twice).
Now, MC: once I got things set in MC as I wanted them, I loaded my symbols and strategy in the portfolio backtester, pressed the backtest button, and waited in anticipation for the local database to reward me for the past 3 days of setting it up. The symbols loaded from the local database in a few seconds, and the backtest started! No more half-hour waits? I like!
So I sat there with my cup of tea and my new trading book, to wait for the backtest to finish. Unfortunately, I didn't have time to open the book - as I looked at the screen, the indicator was showing 2% completed, 3% completed, and I wondered what was going on. In 30 seconds, the backtest results came up on screen, and it took me another 30 seconds to come out of the shock, and check whether the MC did something stupid and blurted out 2600 wrong trades in a few seconds.
The results were exactly the same as TS's!
MC is advertising that its new 64-bit software and multithreading capabilities made testing faster, but when do ads tell the truth??
Even with my disbelief freezes taken into account, reducing test times from 150 minutes to 2 minutes is, well.. insert your own adjective here.
I suspect the 64-bit thing and all that makes a positive difference, but I also feel that Portfolio Maestro is inefficiently built and that makes a negative difference.
As far as I am concerned, it's goodbye TS and hello MC.
I hope TS picks up the pace on its product upgrades, I realise now what others have been saying in forums - TS is getting left behind.
Any comments, suggestions, questions, are very welcome!
Take care
Andreas
Can you help answer these questions from other members on NexusFi?
Andreas, I want to thank you for that exceptional review. For someone who trades mostly equities, has MC, and recently signed-up with TS (great offer) - this was most relevant. I might add that one of the main reasons I got MC was for its back-testing capabilities, so your experiences there were of much interest.
If you haven't already done so, you might want to share the relevant parts of your above post (like symbols) with MC directly. Or, do so on their forum as you might hear some valuable feedback from them and other users.
Again, thanks for making the time to help the Community here.
Tony, if you would like to compare anything else between your setup (TS) and mine (now MC), feel free to ask, and we could help each other with info, and maybe someone else scanning these forums.
I just wired funds today into TS (yesterday was a US holiday), so I should finally have access to the TS Platform and data. So, no TS set-up just yet... That said, from the feedback received in the MC forum, I might be best served by delving into MC and just focusing there; as opposed to spending much time building-out and tailoring the TS platform. I linked to that discussion, but I think it was in the other futures.io (formerly BMT) thread. Here is that link regarding MC / TS data issues:
I'd ideally like to use TS as a data feed for MC, and if I can do so without some of the noted data issues, I'll be a happy camper. I'd then consider getting that software (RT Soft) to allow for trading TS data in MC...
Andreas, let's keep-in-touch. I'll most likely have some growing-pains, as you noted too, so I might need some time to get MC set-up properly with TS data...
If you plan on doing any backtesting (especially portfolio backtesting and optimisation), I'd very much agree with your idea to concentrate on MC. In my opinion, it would save you a lot of valuable time.
Having said that, since both platforms are so similar and use the same language, once you get into one of the two, you can spend a few days testing the other one also, just to confirm to yourself which is the best for you - and then come here and tell us.
I'm around in case the plentiful mistakes i've achieved the last two years can help!
Back-testing validation would be great, as you performed, so I understand. Good idea. That said, I don't have Portfolio Maestro, or whatever it is called (it's a monthly add-on). I guess it would be free based upon activity though... Given what you shared, I'm not so sure I want to spend much time using the TS back-tester. But as you said, there might be instances where going into both can serve a purpose.
My funds were received and I can finally download the TS platform, which I'll do in a bit. This is an equity account. I opened futures and forex ones too, but have not yet sent funds...
Thanks for availing yourself Andreas! Mighty kind of you.