Is Groovy 60 Times Slower than Java?
(In a word, no!)
In a pair of very intriguing postings (here and here), Tiago details what he has been observing about the comparative performance of Groovy and Java. He is thorough and probing in his attempts to speed up the Groovy performance, and he lays out a detailed line-by-line analysis of where the code is actually spending time in his tests. Tiago makes some excellent points.
1. Language performance is highly overrated
Much is often made of the theoretical "performance" of a language based on benchmarks and arcane tests. There have even been cases where vendors have built cheats into their products specifically so they would score well on benchmarks. In the end, runtime execution speed is not as important a factor as a lot of people would think it is if they only read about performance comparisons. Other factors such as maintainability, interoperability, developer productivity and tool and library support are all very significant, too.
2. In Groovy, the sensitive parts can always be written in Java
Not much to explain here. When you're working in Groovy you always have the full power of the underlying Java platform and language at your fingertips. If you find a bit of your Groovy code is going slower than you'd like, then you can easily drop to Java and see whether that affords the speedup you want.
3. By successive refinement, performance can be optimized
After his original blog posting Tiago received some good insights about how to use idiomatic expressions to get better performance results from his Groovy test code. He shows quite clearly how one change shaved off a full minute, and another shaved 1m10s. He's actually gotten it down to where Groovy now takes only 90s compared to Java's 4s. It's still a good deal slower, but a lot faster than when he started.
Anyway, I really admire this type of explicit analysis and documented experimentation to improve results. IMO, Tiago has done a really good job trying to zero in on where performance differences exist and how they can potentially be narrowed. One of the things I find most interesting here is that the shoe is really on the other foot now! How many years were we reading people's subjective (and usually unscientific) criticism of Java performance in comparison to their language of preference? Now we see Java as the speed reference. You've come a long way, baby!
What are your own observations about the comparative performance of Groovy and Java? Does performance (in this way) matter to you? What tools and techniques have you found useful to improve performance?
- Login or register to post comments
- 4959 reads
- Flag as offensive
- Printer-friendly version







Comments
Osvaldo Doederlein replied on Wed, 2008/03/26 - 11:44am
Tiago's analysis is additional evidence that Groovy is still heavily broken performance-wise. This is not defensible with productivity vs. performance arguments, simply because there are other languages with similar features - MOP, dynamic typing etc. - that perform much better. Including JRuby, which allows an apples-to-apples comparison as it shares the same runtime. And JRuby faces some additional challenges, because the Ruby language is full of bad design choices that impact performance heavily without any good excuse (Charles Nutter blogged extensively over these issues, here for example).
Yes I can put up with a language that is slow because it has some features that I want, and consider more important than some performance disadvantage that is an intrinsice cost of those features uner current compiler/runtime technology. But a very different thing is accepting massive performance disadvantage that's mostly related to a weak implementation. Would anybody, these days, use Java if it had mediocre GC performance, now that advanced GC implementations are available (and even a commodity)? Not me, that's for sure.
Fact is, Groovy is pig slow because it goes through ten layers of dynamic and meta stuff every time you attempt to add two integers. This is absurd. Any competitive dynamic language needs a compiler/runtime that's smart enough to remove most of this overhead (e.g. with type inference).
Francis Perreault replied on Wed, 2008/03/26 - 12:12pm
Rick Ross replied on Wed, 2008/03/26 - 3:32pm
in response to: chiss
I really have to agree with you on this point, Francis. There are lots of scenarios where one product/language/platform is faster than another, but both are more than adequate to get the job done well. It is way too easy to get sucked into comparing big tables of numbers and forget that the human differences may still be negligible. Fast enough is, well...fast enough!
Artur Biesiadowski replied on Wed, 2008/03/26 - 6:03pm
Is groovy 60 times slower than java?
"Performance is overrated"
"It is not 60 times slower, it is 300 times slower, but if you cheat and include jvm startup time, it looks better"
Are we serious here? If somebody wants to hire you as a truck driver and ask for driving license, do you answer "Cars are overrated, I prefer mass transport"? If you want to play this game, title you post "Is performance important?" or something like this.
The blog entries you have linked to are plain stupid sometimes. To quote few cases
"If performance was the fundamental variable, we would all be using assembler."
So, if increasing expressiveness 10 times (assembly to C) is worth 5% slowdown, then obviously increasing expressiveness 2 times (java to groovy?) is worth 30000% slowdown. Actually, why limit ourselves to assembler? If performance would matter, we would be all designing cpus running our programs directly in metal.
"I am currently doing a bit of code to go over tens of millions of lines of text"
If we are comparing speed of two languages, let's try something IO bound. Have I already mentioned including jvm startup time?
"This is not serious as it is possible to write all Groovy intensive parts in Java."
I suppose he meant "performance intensive". Obviously, this is true - it proves the point that groovy is too slow to be anything other than glue language.
Just accept this - groovy is slow (we are talking about 2 orders of magnitude here) and will not be fast. It was not designed to be fast. You cannot add performance to so baroque language afterwards - I'm hearing about how groovy will get faster for last 3 years, "try newest version", "we have improved it so much" and still I see it 100-300 times slower for language speed tests.
You can find usages for it. You can find benchmarks where you don't stress groovy, but network, database, filesystem and it will be almost as fast as java (or even faster if you cheat by using groovy libraries but implementing it by hand in wrong way in java). But language itself is designed to be slow and it's implementation is even slower.
At the time I'll be able to put groovy as a scripting language inside our investment banking system observing 200000 options, I'll officially say "Sorry, I was wrong". And don't tell me it is very special case - it is the case where you may want to speak about the performance. Comparing performance in non-performance critical uses is just wrong.
Rick Ross replied on Wed, 2008/03/26 - 8:46pm
in response to: abies
The blog entries you have linked to are plain stupid sometimes.
Artur! You're coming at me so strong!? Easy there! I'm not playing a game, and Tiago isn't stupid. Be kind to the guy, for goodness sake.
This is a friendly dialogue about one developer's attempt to zero in on performance issues to see if he could make significant improvements. He documents some ways in which he has achieved material gains, acknowledges that he can alway just use Java in the critical areas, and is fundamentally friendly (if not plainly self-effacing) and informative about his process and the details of his attempts. Why attack this? I don't get it.
(Maybe it's just late, and I am in an easygoing, friendly mood. I don't wanna get in some silly argument, but I'd be delighted to hear how further improvements could be made to speed up Tiago's tests.)
Rick
PS - I'd be willing to bet that we could find someone not so many years ago who posted similarly strong statements about how Java wouldn't and couldn't ever reach acceptable performance levels. Never is a long time, so I wouldn't count Groovy out.
Hohonuuli replied on Wed, 2008/03/26 - 10:39pm
Hmmm....groovy is no speed demon. But I've used it for a few Grails apps, a couple of buisness logic layers and currently for doing scientific image analysis. Used judiciously, Groovy's performance is certainly acceptable.
Mike P replied on Wed, 2008/03/26 - 10:59pm
If I wanted to create a database driven, template processing, content management system, and deploy it on a Linux virtual dedicated hosting solution (gives you a linux root online for about $20, run your own name server, apache, mail server, firewall, and whatever you want), then I wouldn't consider using Groovy. I've created twice, a content management system in Java, and the performance was not bad. But if the general flow of the system would run 60 times slower, I really don't think I could put any load on this thing whatsoever.
To get the performance up, isn't it possible that Groovy tries to compile code without all the method calls and introspection, if it knows what you're doing? Perhaps (re)compile things at runtime. For instance, when a method is called and the parameters passed in are all Integers and all the elements inside the methods are known, it could compile an alternative version of that method that it can quickly jump to the next time that method is called with those same parameter types. That code could be cached of course.
Anyway, my standpoint is that unless Groovy makes a serious commitment towards performance, other than just tweaks, I can't use it.
Artur Biesiadowski replied on Thu, 2008/03/27 - 5:09am
in response to: rick
Artur! You're coming at me so strong!? Easy there! I'm not playing a game, and Tiago isn't stupid. Be kind to the guy, for goodness sake.
Sorry about the strong words - I was a bit tired yesterday when I was writing answer yesterday and probably I should wait till next day.
Still, my points stand. We may argue if language performance is important. In same cases it is, in other it isn't. But if we ARE comparing relative language performance, let's not mix unrelated arguments inside.
Plus, please do agree that Tiago's comment about assembly is not very good argument ;)
Rick Ross replied on Thu, 2008/03/27 - 6:50am
in response to: abies
Plus, please do agree that Tiago's comment about assembly is not very good argument ;)
You're a gentleman and a scholar, Artur! I will gladly admit that there's some ambiguity and a bit of confusion present. In truth, I hadn't keyed in on the assembly language statement at first, and I agree with you that better arguments could be found :)
IMO, Tiago was trying to just share notes about his own efforts to find an answer to this question of Groovy's relative performance. In a way, I felt it was more a "thinking our load" and letting the public peek into his notes than an attempt to assert anything conclusively. Tiago seems to be looking for insights that help him better understand the issue and construct reasonable, thoughtful comparison code.
I didn't really sense him taking a position or setting out to "prove" a pre-determined conclusion. Instead, I really felt he was trying to sort it out to gain an understanding of what conclusions might be supported by what evidence. Instead of a thesis, I think we're seeing something more like lab notes.
Rick
Juan Salvador replied on Thu, 2008/03/27 - 2:14pm
It's twice slower in my environment,
I'm working in an pollution mathematical model -that sort of things where you need to run handreds of millons of math cals- In a multicore workstation with OpenSuse Linux 10.3, the groovy version is twice slower for calculation, but when saving results (JDBC connection to MySql) Groovy is as faster as Java. Twice slower for math calcs is not bad while I get better, compact and mantainable code.
Andres Almiray replied on Thu, 2008/03/27 - 4:59pm
in response to: jsalvador
Ahmet A. Akin replied on Thu, 2008/03/27 - 9:40pm
in response to: hohonuuli