I’ve noticed another disturbing trend. A lot of businesses use Java for their high-frequency trading systems.
Knight Trading had a high-profile software flop last week. I don’t know if their trading systems were written in C/C++ or Java or some combination of the two.
I interviewed at some places that write FIX engines. FIX is a common trading communication protocol. Their FIX program was in Java! If it’s being used for high-frequency trading, then their customers are a step behind everyone else.
Some people say “With modern Just-In-Time (JIT) compilers, Java is as fast as compiled C.” That hasn’t been my experience actually using Java. Both Java and .NET are much slower than compiled C or C++.
For something less performance critical, like the GUI or a website, it’s OK to use a higher-level language. For something performance-critical, you have to use C or C++. For high-frequency trading, every millisecond counts.
Also, as a non-insider, don’t bother trying high-frequency trading. Unless you co-locate a server at an exchange, forget it. The big brokers get to peek at incoming customer orders. They have the option to fill the orders from their own book, or send the order to an exchange. You can’t compete with them. Don’t waste time and money on high-frequency trading.
I’m surprised that so many high-frequency trading systems are written in Java. With Java, you aren’t going to get the same performance as with C or C++. At lot of people say that Java and .NET are as fast as C/C++. They are wrong.
I mostly agree with you. I started programming in C, did a vast amount of programming in C and still have fond memories of it.
But… But…
For some types of software, doing one type of work in an inefficient matter actually doesn’t effect the speed of the overall software. I seen this first hand myself.
When I started out programming, I wrote a program in C.
A few years later, I rewrote the same software from scratch in Java.
Because over a long period of time I had seriously thought of efficient way to do certain things, my Java software was faster than my C software. The nature of the software was a complex algorithm.
Then I waited a few more years and rewrote the Java program in compiled C++ as a modern Windows piece of software. It was much faster still. But really as far as the user was concerned the speed of the Java was as fast as necessary.
I am not saying you are wrong. In fact I mostly agree with you.
But it is not the whole story. The same software can be faster in Java than C, even if written by the same person as I have proven. The reason is the design and algorithms got much smarter as the person matured.
For example Microsoft say Early Binding is faster than Late Binding.
However in practice, it doesn’t matter for a big piece of software.
The speed sinks will be elsewhere.
I have seen this first-hand myself.
When I go on an interview, I also evaluate the intelligence of the interviewer. For the C/C++ jobs, the interviewers seem more intelligent than for the Java jobs.
A bad implementation in C is worse than a good implementation in Java. Assuming a skilled programmer, C/C++ will always be faster than Java.
Someone might say “Why not assembler?” For a high-frequency trading program, assembler might be worth it. If you’re trying to get a fraction of a millisecond faster, C/C++ with assembly may be best.
LOL! That’s just awesome!
I can just see it now – some MegaBank is running it’s Java based HFT software to trade the market – and the JRE decides – it’s time to do some massive garbage collection!
… 2 seconds goes by …
… various people’s limit orders get activated …
finally, the garbage collection completes, the HFT tries to resume it’s trades – meanwhile due to the limit orders, the stock price is now much higher than expected – and the MegaBank is out MegaBucks.
Seriously, if you interviewed at some place that was using Java for their HFT algos, DON’T SAY ANYTHING, in fact – just agree with them – maybe even give them tips on how to improve their Java (like how to add a nice GUI to their Java so that it will be easier to configure and use – lol).
If they don’t know any better – just let those bastards rot. (Any job you could get with them just isn’t worth it, if they’re that stupid. It’s way to difficult to try to educate them, so don’t even try – just let their own incompetence destroy them and move on).
LOL – your blog post today just brought a huge smile to my face – there is Justice in the world.
Wow! Im trying to imagine a Java guy taking that much glee in flaming a C++ dude.. Im guessing your hard to collaborate with!
Early optimisation is the biggest problem Ive seen in HFT.. Using a managed language with no surprises makes sense for team work. Use the highest-level language you got for everything to start with – work out the hotspots then drop the abstractions layers (java->jni->C++->C->Assembly->Hardware) as necessary
You don’t have much knowledge about Java, do you? You shouldn’t comment on things you have no clue then.
I guess you can only imagine a single JVM running everyting and when GC starts everything else is on hold?
Really, there are different options to configure GC, and its behavior may differ very much on the particular JVM used. It doesn’t do the STW (Stop-the-World) process anymore. Also, there can be something called immortal and scoped memory additionally to normal Java heap. These areas are not GC’ed.
If we assume that HFT is a Real Time computing case – then you may get familiar with RTJS spec…
Summing up – saying that $LANGUAGE (this case – Java) is wrong for some big project is simply stupid. In case of real performance-oriented systems you usually find them heterogeneous – different parts written in different languages, running on different hardware. BUT, you never do this at the beginning, so even writing the whole thing in Java may be a good choice. Then, you may start to optimize stuff, maybe rewriting critical bits in Assembler or C. You may spread it onto some processing farm or whatever will get you to the required performance. But only if you really need that. There is this saying:
1. rule of optimisation:
Don’t do it
2. rule of optimisation:
Don’t do it yet
You may of course start with Assembler at the first place… but this is ALWAYS about money. Will a big app written in Assembler be cheaper than the hybrid?
The very fact some programmers need a garbage collector shows they are not aware of the memory they are allocating…….. not good for speed!
You probably have no idea you can write a piece of java where the GC never kicks in, but what do I know?…
One of the reasons why my software had good sales for a number of years and eclipsed all of the competition for almost all time up until a year ago, was that in my area there is a certain type of operation that is very, very slow.
Almost by accident one evening I picked up on the right thing to do to make something that is slow, very fast. Microsoft actually don’t provide any documentation on this area. I had to experiment.
Anyway the slow type of work is slow in Java, C, C# and C++. All languages equally suck in this area. If you know the trick, the operation becomes magically fast in whatever language.
Your compiled language and technology isn’t the be all and end all of it.
What you’re referring to is algorithmic efficiency, and this requires an intelligent programer to come up with something efficient and quick – works well in all languages – can result in 100x or more speedups of slow problems.
That’s worth a LOT. Companies don’t understand it, and won’t pay for it. It requires an intelligent programer to create it, and other intelligent programmers to NOT SCREW IT UP.
Companies like to hire interchangeable workers (cogs) – that way, they can budget how long something should take to do. Intelligent workers are not interchangeable – it’s not what you’re paid for.
Worst part of intelligent workers – they usually command premium salaries and bonuses, and are not easily replaced. They also have trouble with being tasked to do stupid things (ie: hard to control by their less intelligent and higher paid bosses, who have trouble justifying their own existance – it’s their job to ensure that their workers are easily interchangeable).
That’s how the “machine” works
I actually visited the offices of a hedge fund in London on four different occasions for interviews!
They use Java.
Despite actually doing well technically in the interviews, I didn’t get offered a job for the silliest of reasons.
During my last interview, the interviewer casually dropped into conversation they work 60 hour weeks. I have in the past worked many 60+ hour weeks. I don’t remember what my reply was at the time, but I either said something non-committal or made no comment.
After the interview, the smarmy recruitment consultant telephoned me on my cell phone as I was walking down the street. I really wasn’t in the mood for a conversation with a smarmy, pushy git at that time as I wanted some time to wind down after a barrage of technical questions.
Anyway the recruitment consultant asked me what I thought of the position. I replied that I wasn’t quite sure as they said they work 60 hour weeks.
The smarmy guy hung up and then rang me back. He told me they don’t work 60 hour weeks but the comment about 60 hour weeks was a trick question and I had failed it. Hence I wasn’t being offered a job!
Oh yes, during the interview, the main technical interviewer spoke for about 30 minutes solid on garbage collection. It was getting tedious. I much prefer to read a book that listen to someone else. Anyway I think I was looking a bit bored after his 30 minute monologue. He then attacked me for not being interested in efficiency.
I then replied I had written the fastest piece of software in my field and am currently selling it on the Internet. I told him anyone could download a trial version of my software and see for himself/herself that my software is fastest than all the competition.
Anyway this seemed to stop the guy attacking me.
There is another side. I once knew a clown who believed that software developers were interchangeable idiots. He believed if you only found the right combination of language, technology and library, then that would do all the work.
He wasted all the time looking for this magical combination of languages and libraries.
I knew all long that a programmer trusting his instincts and himself and making the right decisions would trump.
He was wrong.
I was right.
Let me give some point why High Frequency trading in JAVA.
I and nearly everybody knows C++ is way faster than JAVA and perl & CGI is even more faster than every other language.
But it’s a high frequency domain, just being fast enough is not the solution. Let me tell why Java is used mostly in this domain.
Firstly, there are parameters to configure your JVM according to your application need, so you can get best out of it, you may even surpass C++ speed. For example, setting the memory segments for your JVM. Young generation, Tenured or Old Generation and Perm Area of heap are different segments of memory given to your application. Distribution of the percentage of available space among these segments, based on the nature of your application, will boost the speed a lot.
Read more:
Secondly, Off course Garbage Collection (GC). In a high frequency domain lots of object is created and destroyed in next instant. So allocation and freeing space are the key and most important part of it. GC provides different set of algorithms to handle this and you can choose one of them based on your application. So biggest issue in C/C++ application is memory leaks and dangling pointers. Coder have to take care of it. But in java, you are free from that effort and so you can put more effort in optimizing your algorithms and approaches.
Memory leaks are only a problem if you’re incompetent.
Bytecode is never going to be as fast as natively compiled C/C++. You can argue that development time is faster in Java (and I’m not sure about that), but the finished product will always be faster in C or C++. Even better, put some inline assembly in your C and C++ (which you can’t do in Java).
I know what are you saying is right. All these are really make c/c++ faster in processing than java.
But still for high frequency trading system that is not enough, and nowhere close to the requirement.
There are hell lot of complex algorithm works in this domain, even a fast processing system developed in c/c++ will fail over the period of time. Managing the memory and developing these algorithm by a team and a developer will never be possible.
You might have to allot two teams one to develop and other to check memory management issues. Without hacking c++, you will never get the processing environment your application desire (which is configurable in java’s jvm).
I am not arguing that java is faster than c++ (I know c++ is faster).
Don’t take it personally, but you reply shows you never worked in High frequency domain.
C++ provide high end if the task is small and manageable. But if it is highly complex system, java is always better for problem solving, until someone comes and gives similar power to C++ as GC has in JAVA and flexibility to configure you running environment.
I’ve worked a decent amount in Java, C++, and .NET. In my experience, the performance of Java and .NET is nowhere close to C++.
Also, I have written some high performance code in C++ with complex calculations. There’s no way it would have worked in Java with decent performance.
It certainly is possible that most people who write high frequency trading software are idiots. In that case, an inefficient program written in Java will perform decently. I’ve seen a lot of incompetence in the financial industry. I’m not eager to enter that area, because trading is an activity that produces no value to the rest of the economy.
Java might be faster for writing software (and I question that). It certainly won’t be faster for the final product.
It’s too naive to say that high-frequency trading has to be done with a specific language… it depends on what your needs are:
- pure arbitrage strategies: these are often simple and well understood strategies and only the fastest player will win a given opportunity. These solutions are increasingly implemented directly in hardware (http://www.hftreview.com/pg/blog/accelize/read/56164/successfully-deploying-fpgabased-network-adapter-cards-to-deliver-ultralow-latency-trading). C++ may be lower latency than Java but will stand little chance against pure hardware solutions. Pure hardware strategies can be as fast as single-digit microseconds between receiving a tick and placing an order against it. Downside: high cost, low turnaround time
- complex trading strategies that still require very low latency: these are typically done in c++. Just search the job boards. It may only be 1/2 or 1/3rd as productive as java but the improvement in latency makes it a requirement. And C++ is still way more productive than a hardware solution. I am not sure if it’s possible to get C++ into single-digit microseconds but you can probably get pretty close for a very simple trading algo
- and finally you have a whole slew of other types of “higher-frequency” models that are simply less latency sensitive. Perhaps it’s mostly using passive orders. Perhaps it’s trading with banks instead of open exchanges. Perhaps holding times are measured in minutes instead of seconds or milliseconds. For apps like these, java is a very good choice. But you can’t run arbitrage strategies on open exchanges with apps like these, you just can’t compete. For for most firms, that’s ok. I think a carefully written java app could probably go as low as 50 microseconds for a full round-trip between receiving a tick and placing an order against that tick. Of course, that will be an average of 50 microseconds, but the latency-variance for java tends to be much higher than for compiled languages. Benefits: lower cost, quick turnaround time, easier to develop stable software, etc
So I say: it depends on what your needs are. But most high-frequency firms use C++ as it tends to strike a happy medium between latency and development time
I write HFT/ultra low latency trading software with extremely complex business rules and algos for a living and it’s all done in C++ simply because Java isn’t nearly fast enough. At times we even write code for bare metal to squeeze out that extra microsecond.
> Managing the memory and developing these algorithm by a team and a developer will never be possible.
Spoken like a true idiot. Yes, if you’re incompetent and work with a bunch of incompetent fools, then it’s indeed not possible.
Why do you think it takes longer writing an algorithm in C++ than Java? It doesn’t for the type of C++ guys working in HFT.
Dear Sir or Madam,
The reason why I am writing to you is that I am a last year Master’s degree student at the Louvain School of Management at the university of Louvain-la-Neuve (Belgium).
As to achieve this final year of academic study I decided to write my final thesis on the topic of “High Frequency Trading”. What has been done so far was to emphasize benefits and controversial misdeeds of this recent trading method by going through the academic literature. Nevertheless, I would now like to support or, to the contrary, contradict my conclusions by collecting the opinion of professionals of the sector.
Therefore I hope that you will agree to answer and/or to convey to your relevant employees my online survey which is located to the following internet link : https://docs.google.com/a/student.uclouvain.be/spreadsheet/viewform?fromEmail=true&formkey=dFRwdHREUXpxRUk1VTlZaDNoSmw1bVE6MQ
If a French version of the survey should be more adequate please find the internet link hereafter : https://docs.google.com/a/student.uclouvain.be/spreadsheet/viewform?fromEmail=true&formkey=dF9xTVU4bmpVN19xelVLUFJCSmRob0E6MQ
If you should come across any question please feel free to contact me.
Lastly if it should occur to you that one of your acquaintances may meet the different criteria requested to answer this survey please feel free to forward this email to that person.
With the hope that you will answer positively to my request I thank you very much dear Sir or Madam for your help.
Yours faithfully,
Olivier BAUCHAU.
Student in 2nd year of Master’s degree at the Louvain School of Management (Belgium)
Tel. : +32-478/68.89.45
Email : olivier.bauchau@student.uclouvain.be
LinkedIn : http://www.linkedin.com/pub/olivier-bauchau/48/492/628
Add Visual Basic to the list,too. – the latest version- Since it magically breaks down to the same MSIL as C and C++.
Nobody in their right mind would stake their business with VB, though. Its syntax looks like the Ruby of .NET world- an omen. And the community is languishing.
Oh, and in Node news, it’s almost ready for high frequency trading. Hey, it’s faster than Java these days, and is only getting faster.
I’d give Node.js a few more years, though. Maybe by then I’d have a C++ version in beta.
Again, somebody that thinks his things are faster because are done in C++.
My processor does not process bytecode, and yours? LOL
The only idiots I know are those who say things that doesn’t know about. I have a Java implementation algo engine that has the same latency than a C++ implementation. It’s much easier to built. It’s using Java Real Time, there is no GC pauses for your information. People who says things like “C++ guys are smarter…” are the real idiots. Let them be idiots…
“a C++ implementation”
Yes and that could simply be a bad C++ implementation. C++ developers tend to be better than Java developers because they don’t need to rely on a magical garbage collector to manage memory. You honestly think somebody who needs a background process to manage their memory is going to be better than somebody who manages their own?
Neither Java nor C++. The big players are using FPGA for developing HFT; other parts of system could be done with Java or C++, it depends. A lot of people doesn’t know about Java Real Time. It could be used to build very low latency systems. The latency could be as low as with C++.
I’d like to see someone take the algorithm from this post and do a Java version. I expect it to have performance close to the Javascript version.
Someone has never heard of the Zing VM… Seriously, these companies use Java (actually, a restricted subset of Java, but since you’re clueless, I won’t bother with details) for a reason, and they keep using it because it delivers the results they need. In HFT you make money or fold: Java would have been abandoned if there were better alternatives. Despite all the challenges of making it work, Java is the best combination of agility and performance available for the job, and that’s why HFT shops keep using it. (I’m not saying that HFT shops using Java use it exclusively, because they don’t, but I don’t feel the need to get overly nuanced in a simple comment.)
I agree with this… I am starting to develop my HFT program. Do you recommend win8 & C++ as a HFT platform. Have you used quickfix library?
“Write your own HFT program?” Unless you already work at a big bank (in which case you have to use whatever they order you to use), you’re wasting your time and money.
The big banks have too many advantages, which makes it a waste for anyone else to write an HFT program.
Big banks aren’t the main players in HFT….
Obviously you haven’t heard about Grid computing, caching and other enterprise solutions that come along with Java. Its not only about how fast the code runs, its about the eco-system that is in use. When we are talking about trading systems, its not the speed of the individual process that matters. You might right your code in C/C++ but if you do some IO operation in your business logic, what use is there of the speed of c/c++?
Java provides a tried and tested ecosystem which has demonstrated for years its performance and scalability.
Search LMAX architecture written in Java which process 1.6 million transactions per second.
Nice to hear from you. Do you have any idea about real time systems used in Java ? Also do you think we can have success rate for predicting the movement of stocks ? I am just lost after slogging for hours…
Unless you work at a big bank or have a lot of money to waste, don’t try writing a high-frequency trading system.
The big banks get “payment for order flow” and privileged access to exchange data feeds, getting information ahead of everyone else (including colocated servers). That’s too huge an advantage to overcome, no matter how clever you are.
If you’re an individual small/medium investor, stick with index funds (novice level investing), gold/silver PM etfs (intermediate), or physical gold/silver taking delivery and hiding it in your home (advanced).
Not as an individual. Some HFT companies wants me to work for them (Java). They are planning to collaborate with hedge fund traders. I am entirely new to this field. I have some experience in machine learning. What kind of career advice will you give me ?
It could be good experience. Did you get a formal job offer yet?
I think that writing an HFT program in Java is ridiculous, but a lot of people are doing it, and there are a lot of job ads for HFT+Java.
Personally, I’d rather be working at a job producing something of real value. However, my current employer is an MLM (multi-level-marketing), i.e. a pyramid scheme.
On the other hand, taking career advice from the guy with 15 years of experience who’s in a job doing 100% maintenance of other people’s code might not be such a good idea.
It is possible to write bad code in Java – a lot of people do.
It is also possible to write really good code – a few people do.
If you are coding a HFT system in Java, a good programmer would take care of a few things -
1. Bytecode – when the JVM is interpreting the byte code – performance is bad, it will never match the performance of compiled c or c++ code.
However after a piece of code is run 10,000 times (user configurable, but better leave as is) – the JVM compiles the code into native instructions.
And the produced native code will be significantly faster than code generated by gcc with -O3 with arch=native, because the instructions will be optimized for that specific machine.
So – typically for any service – when you start it – introduce a warmup phase that runs critical code paths at least 10,000 times.
You can then verify that critical code has been compiled and you can even examine the generated assembly.
2. GC – GC can be a killer. While the Zing JVM offers pauseless GC, the best strategy is to generate 0 garbage.
Always re-use objects. Watch for whenever you allocate a new object and ask yourself if this is really necessary.
Make use of off-heap memory – access to this is a little slower, but the benefit is the GC never examines this memory.
With these patterns in place – your HFT system can usually be run with a max managed heap size of <1GB.
You can then use as much off-heap memory as your system can provide, without worrying about long GC pauses.
Finally – dont trust everything on the internet. Always measure. Use a profiler like Intel vTune to analyze what is happening.
Identify the bottlenecks. If you have done 1 & 2 above, you will usually find other sources of latencies like the network stack.
Just an opinion, that speed matters the most probably only if you’re strictly just a programmer looking for nothing except performance and performance only. However if you’re a seasoned trader then I believe you may opine that speed is NOT the most important issue at all. Why?
First, do you write a HFT program because you’re a market maker? Or because you’re a trader? If you’re a market maker, then yes, you strictly need the fastest language ever, no compromise here. But if you’re just a trader, then what is your trading time frame? And what is your trading system’s performance (after back-testing excluding the speed factor)? If your trading time frame needs to be as close to the time frame of market maker as possible in order to be successful, after net commission, spread, and slippage, then yes, you strictly need the fastest language ever. But if your trading time frame is in hourly or even daily, then your concern is just partially about speed and even more to do with your trading performance.
If your trading system’s performance (excluding the speed factor) is very good, then speed would be more of a concern. But if your trading system’s performance is just average, then having the fastest speed will actually add not much value to your profitability. You can look at speed as an amplifier to performance. If your trading system is loss-making, then having the fastest speed means you’re being the fastest in making lousy (or wrong) decisions. In that case, then maybe a much slower speed is what you need. Now, what about if your trading system is high performing? Then speed remain a very important factor, but still not the most important. Now you need to ask yourself what additional benefits you will gain from fastest speed. What are your needs of having a fastest speed? Is it to reduce the slippage? Then how significant is the slippage to your total trading cost? If it’s not significant compare to how your trading system fairs in a market condition that it is not optimized to excel in, then fastest speed would not be the main issue. If your trading system is top performing, in an hourly or daily time frame, then having the fastest speed may only serve to enhance the performance by some margin only, with the major contributor of the bulk of the performance in how your trading system really perform.
As I’ve said, speed is like an amplifier. If your trading system is lousy, then the fastest means you’re the fastest in making lousy decisions. But if your trading system is top performing, then fastest speed may only serve to enhance your performance by some margin only and does not really decide whether your trading would make or break. Unless your role in the market is strictly being a market maker, speed is not necessarily your most important criteria. Many more OTHER things need to be considered to come to the conclusion whether you will make it big or go broke trading.
Strictly in terms of software performance alone, only idiots use Java.
Strictly in terms of trading performance alone, only idiots use C/C++.
Java is slow? Buy another server or PC!
Java software development cost is smaller than C/C++ so with the saved money you can buy a faster CPU.
Today at work, the other programmer implemented auto-scaling. (That’s when you automatically add/remove AWS servers based on load.)
However, he didn’t do it right and cost us half a day in sales when he broke the payment system! Huge savings!
I had set up some Crons to automatically warn us when the payment system failed, but he broke those also.
(He is an idiot. Done right, auto-scaling is a good idea. However, there also are lots of huge inefficiencies in the code. They really don’t need more than one server, except for all the things they do poorly. This is in PHP, which has slower runtime performance than Java.)
I find this article full of arguments.