Over the past couple of nights, I tried out MagLev on my Mac laptop (Mac OS X Leopard, 2.2GHz, 2GB Ram). While the shootout will provide us with a detailed comparison between MagLev and the other major Ruby implementations (on many tests) I thought I’d share my first impressions.
MagLev is going to be a fast implementation of Ruby. How many times faster? Well, it depends on the test, and I don’t have systematic numbers yet to come up with a geometric mean of the ratios (the shootout will do just that). What I can tell you now though, is that it’s fast, significantly faster than MRI. You know that slowness that you’ve almost come to expect when running a Ruby script? It’s simply not there with MagLev. And let’s not forget, that MagLev’s added value is not limited to performance, in the same way that JRuby’s integration with Java is a strong selling point.
It’s not mature, of course, and there are a few things which haven’t been implemented yet. But it’s complete enough to run WEBrick and several other small programs I tried. A truly impressive accomplishment if you think about what was achieved in just three months. So let me reassure you that it’s not vaporware, it’s real and it’s fast.
I want to leave you with an example of a non-trivial benchmark. Running the binary-trees test from the Computer Language Benchmarks Game (chosen because it’s the first test), I obtained the following results:
| PHP 5.2.5 | 100.603s |
| Perl 5.8.8 | 70.885s |
| Ruby 1.8.6 | 60.089s |
| Python 2.5.1 | 29.908s |
| MagLev | 7.673s |
| C++ (gcc 4.0.1) | 6.265s |
Aside from being much faster than Ruby 1.8.6, MagLev’s speed is pretty darn close to that of the extremely fast C++ (complied with g++ and all the optimizations), in this specific case.
You can’t read too much from a single test, especially when we are not comparing different VMs from the same identical script like we’ll do for the shootout. It’s impressive nevertheless and it should give you a hint about MagLev’s speed even when facing non-trivial, non-ad-hoc benchmarks.
Get more stuff like this
Subscribe to my mailing list to receive similar updates about programming.
Thank you for subscribing. Please check your email to confirm your subscription.
Something went wrong.
wicked!
I’m surprised this article’s title is not prefixed with “Holy Shmoly” 🙂
Could you add Psyco and LuaJIT to the list? I guess that would be more fair for an advanced VM like GemStone’s.
… With this benchmark you are measuring the performance of the memory management subsystem. malloc-(free/garbage collection) is more expensive than “interpreting” some bytecode. This benchmark says nearly nothing about the performance of the “MagLev-Engine”. But it says something about the mm-system.
Wow.
Is there word on when MagLev will be generally available?
(I recognize it isn’t quite done yet, but I’d be very interesting in experimenting with it some!)
Very nice numbers, but it’s definitely a GC/object allocation benchmark. Ruby doesn’t do as well because it has a fairly primitive GC. JRuby doesn’t do as well because we’re allocating way too many objects, mostly for compatibility reasons.
And Java is the winner on the current shootout results.
This is amazingly good news for Ruby!
@ralf: I know nothing about mm systems and the like, so please do not take this the wrong way, but how were you able to recognize that this was about MM systems instead of bytecode interpreting?
@Ralf, @Charles: Yes, it’s testing memory allocation and management, but this is interesting nevertheless, especially because Ruby’s main implementation has an issue with this.
Well, that certainly sounds promising. Can’t wait for more detailed info and benchmarks!
Impressive results!
Could you also run some of the other tests for example fannkuch or fasta?
These are the tests where Java shines and I suppose, that is because less code is executed in C in Ruby.
There is no way Maglev is going to be “pretty darn close to that of the extremely fast C++” when it is finally released.
Articles like this are just setting expectations it cannot meet which will cause noise from the haters that Ruby isn’t up to the job and make it harder for Ruby developers to justify using it for real projects.
mtkd please don’t change the meaning of my words by selectively quoting me.
I wrote “Aside from being much faster than Ruby 1.8.6, MagLev’s speed is pretty darn close to that of the extremely fast C++ (complied with g++ and all the optimizations), in this specific case.”
I never claimed that MagLev is going to be as fast as C++ in general (it’s not going to).
Fantastic performance numbers. MagLev could really be a game changer.
@Markus: I couldn’t run Fasta, but I was able to run, by implementing a method of the Array class on my own, Fannkuch.
These are the results that I obtained:
I think benchmarking MagLev against anything is a terrible idea at this stage. I think releasing numbers as though they mean anything is an even worse idea.
Awesome numbers Antonio. And I hope people understand what you are saying. In no way you stated that this is the ‘real’ number for the general case but a good ‘reference number’.
I wonder, is there something like a test suite not focused only on computational performance, but that also mix and match I/O and such?
For instance, you said Webrick runs. I don’t recall if Webrick has a test suite. Running that one could give us a better understanding of the overall behavior of Maglev.
What do you think?
Don’t forget, the current winner of this benchmark is Java. Yes, Java is beating plain old C in this benchmark.
@Jon: There is no harm in discussing early results. They are framed as such, with no claim of being conclusive.
@AkitaOnRails: That’s an interesting idea, Fabio. Please feel free to join the Ruby Benchmark Suite’s Google Group to discuss this further.
Long life to MagLev!
This is promissing…
It’s surprising how in this test 1.86 is much faster than PHP.
Umm isn’t anybody worried about the fact that it isn’t a free implementation? Or is it?
php 100s
ruby 1.8.6 60s
is that possible?
Nice job, still.
It was interesting being in the audience at RailsConf where MagLev was first revealed. I think I speak for many when I say that I wanted to dislike it; but it was an impressive demo. Avi nailed it. The speed looks impressive (though it is very early), but I’m really interested with the idea of having a persisted, shared memory space at the VM level; ie the global memory space is shared and persisted. Faster speed is nice, but I think this feature is fascinating and I can’t wait to play around with it.
How did you get a copy of Maglev, and where can we get one? 😛
@Robert Hulme++.
Where can we get one?
@Robert, @Luis: Unfortunately the VM is not publicly available at this stage. I only managed to get mine under non-disclosure terms, thanks to GemStone’s interest in an open, unbiased and independent performance test which I’ll carry out with my Ruby VMs shootout.
Can you describe how MagLev will potentially change the way we write Rails apps? No more ActiveRecord? Maybe some other kind of framework altogether (like a Seaside for Ruby)?
Is there some way to prepare for development under MagLev? Thank you.
Kyle – take a look at this video: http://www
it should answer your first question.
Frankly, if MagLev can’t run Rails as it exists right now, then MagLev isn’t Ruby, it’s MagLev, a fast language that happens to look a good bit like Ruby.
Still waiting for some actual code, some actual substance to back up these numbers. From what I know about JRuby and MRI, and reading blogs about performance and Ruby VM implementation, I am going to remain highly skeptical about MagLev.