I’m not at RailsConf, which is being held in Portland, but it doesn’t take a great leap of faith for me to believe that the session on MagLev was the star of the conference. Avi’s demo created quite an echo and it’s currently the most discussed topic in the Ruby and Rails communities, despite the announcements of IronRuby on Rails and today’s release of Rails 2.1.
I had no doubt that this would be the case. For the past few days, and before RailsConf, I’ve been in touch with the nice people at Gemstone and witnessed, before most, the speed of the current implementation. My first comment truly was “Holy Shit!” and I don’t like to swear. 😉 In a side-by-side comparison, it was like watching a race between an eight week old chubby bulldog (Ruby MRI) and a full grown jaguar (MagLev). Ruby MRI looked cute, I was almost cheering for it (“come on Ruby, you can do it”). It was so spectacular that you couldn’t help but think that they were pulling your leg.
MagLev is not complete yet, but it’s very remarkable that it accomplished so much, both in terms of tests passed and its speed – despite being only three months old. It speaks volume about the capabilities of the Gemstone platform (on which MagLev is based upon) and MagLev’s team work.
Let’s be clear about one point, this announcement changes everything; it has the potential to revolutionize the Ruby community. It’s not just a matter of a substantial speed increase, Ruby’s main weakness, it’s also a matter of scalability, paradigm shift and Enterprise perception.
When you get to use an object persistence model that can hold up to 17 Petabytes (that means 17 Million Gigabytes for you), with the ability to scale by simply adding instances, the whole “Ruby doesn’t scale” FUD starts to fade real fast. Also, aside from all the technical advantages brought by MagLev, selling a Gemstone based solution to environments that are typically harder for Ruby to penetrate into, becomes a no-brainer. It’s a newfound ticket for Ruby and Rails into many sectors of the Enterprise, and an easy entrance in the financial world as well.
MagLev is unconventional, but that’s because most impressive innovations tend to be so. That’s what the future looks like when seen through the eyes of the present. The reality is that MagLev’s OODB paradigm and architecture are quite a good fit for common Web development too, so if properly pulled off, MagLev could change the way we write Rails applications.
A few people objected that it won’t fly, because it’s going to be a commercial product. I don’t think that’s the case. The Rails community adopts Macs, TextMate, and other commercial software without blinking, so why would they have any issues with paying for a truly superior Ruby VM (if it proves to be so)? The terms of release for MagLev have not been finalized yet, but it’s clear that some part of it will be open and others (e.g. the actual VM written in C) closed source. It’s likely that there will be different pricing levels, including a free version with some limits in place. But price will mostly be irrelevant.
Planning the next Great Ruby Shootout
The great news is that the next edition of my shootout will include MagLev, so we’ll get to test it in a fair and controlled environment, against all the other major implementations. The results of the shootout will be published sometime in June. These are the implementations that I’ll most likely test:
- Ruby MRI
- Ruby 1.9
- JRuby
- Rubinius
- MagLev
- Ruby Enterprise Edition
- IronRuby
- IronRuby on Mono
I’ll probably test them on Ubuntu 8.04 (32 and 64 bit), Mac OS X Leopard and Windows Vista (32 bit). Please note that not all eight of them can be tested on each of the four platforms, so you won’t see MagLev on Windows or Linux 32 bit, for example. I’d like to test all four operating systems from the same machine (my MacBook Pro Core 2 Duo 2GHz SR, with 2GB of RAM) so that we can also compare how these major implementations behave on different operating systems, but I may have to opt for some native installations and carrying out others from VMware Fusion; I’ve yet to sort all the details out. 😉
A criticism that I’ve often heard is that these micro-benchmarks are well known and specifically targeted for speed by the implementations’ authors. The new shootout will also include a set of tests that no one has seen before (I just started to write/put them together). They may still be synthetic benchmarks, but they’ll help us form a better idea of the actual speed of these VMs.
Get more stuff like this
Subscribe to my mailing list to receive similar updates about programming.
Thank you for subscribing. Please check your email to confirm your subscription.
Something went wrong.
Perhaps, include the newly release Ruby 1.8.7 ?
It will be interesting to see with all the changes and backport they put into this release affects performance.
Yup, the demo was really something. But Charles Nutter of JRuby raised a point during the Q&A that since the implementation isn’t complete yet, the benchmarks that were presented might be drastically less impressive once the implementation is complete and Maglev is RubySpec compliant.
Don’t know how true that it is, but I’m hoping it’s not! It looks and sounds really cool.
Does the Gemstone OO database technology support replication, hot backups, and failovers?
I would think that these matter more than the ability to manage more data than 99% of the user base requires — especially within the enterprise.
@Taylor: Yes, Ruby 1.8.7 will be tested.
@Arun: Charles’ concern is justified, but MagLev is unlikely to get worse by an order of magnitude. Given the current results, even if it were to slow down a lot, it would still be much faster than Ruby MRI.
Avi says that they’re leveraging 30+ years of development on Smalltalk, where the problems that the current Ruby implementations are facing were already solved years ago.
It will be a pity if MagLev’s VM remains closed source, because no other VM would be able to benefit from that experience.
Unless the recipe is documented somewhere.
In the other hand, if MagLev goes open source, I wonder what would that mean to Rubinius.
Ugh. Sorry but having done micro-benchmarks of C vs Java vs Smalltalk before, now that there is an advanced VM directly running Ruby (not counting Java since it is advanced but not designed to run dynamic languages) you need to beef up your tests.
Mostly, no part of the test should ever do nothing. For instance, bm_vm2_method.rb:
“m” should have some side-effect that ultimately gets printed out to the console. This test in Java (since at least 1.3 jdk) takes ZERO time because the entire program is optimized out.
Many of your other tests contains similar micro benchmark flaws. Also, please test using OpenJDK to run JRuby/XRuby, if they run with it. Fedora Core currently has better support for OpenJDK, but it is in the ubuntu repo also.
“The results of the shootout will be published sometime in June.”
We already are in June, maybe today then? 😉
Kidding! Keep on the good work!
@Anon: Please note that these tests were not implemented by me and are considered “standard” in the Ruby community. However, you have a valid point and I’ll definitively consider manually modifying them, so that each of them is forced to compute the results no matter how smart the VM is. I’ll also try to see if OpenJDK is a viable option for JRuby.
> The new shootout will also include…
As I’ve mentioned previously, all those so_* tests are programs that once upon a time were part of Doug Bagley’s Great Computer Language Shootout, and were removed back in 2005.
The idea that the Ruby community considers something from the Shootout “standard” is mildly amusing – it’s just a pity that it’s something we abandoned.
I too am excited to see something getting 10 times faster, but I am having hard time visualizing a proprietary, closed-source platform “revolutionizing” something in 2008.
Definitely. Another flawed YARV benchmark is the ‘ensure’ test. The ensure body is empty, so we just optimize it out in Rubinius.
That is a common misconception Antonio. YARV’s tests were not supposed to be used for benchmarking Ruby.
Check this interview: http://blog.grayproductions.net/articles/the_ruby_vm_episode_ii
By the way, how am I supposed to create paragraphs in this blog?
“MagLev could change the way we write Rails applications” – so that they look like Grok applications 😛
Antonio, I use JRuby with OpenJDK on Ubuntu 8.04 every day and it works with no problems.
As for performance/completeness discussion, from my side (working on JRuby), it is VERY frustrating when we have to add/change fixs in JRuby to make it fully 100% compatible and when in some cases that slows down the implementation. Disabling some features, enabling non-safe and not-100%-compatible features, it *is* possible to speed up JRuby at least 2X. So, compatibility does come with a price.
Take a look at the latest headius blog post on that topic as well.
Is this new OODB thing the one that will make working with exisiting legacy databases less frustrating than now?
Or is it totally alternative to traditional RDBMS?
Also, why the choice to go closed source?
@Santana, perhaps I shouldn’t have said “standard”. What I meant was that Yarv is not the only implementation that adopts them (e.g. they are in Rubinius’ repository as well).
In my new shootout I’ll introduce more general benchmark tests that should remove the advantages from any VM that specifically targets these micro-tests.
You can separate paragraphs like this (remove the spaces from br):
Unfinished? Commercial? I’m sorry, why did I come to this page again?
@Isaac, “standard” was the wrong word to use. They are commonly used for benchmarking, but many of them are admittedly useless when testing multiple implementations.
@Ev: If MagLev was able to run Ruby 10x faster for real, many people wouldn’t care too much about its proprietary nature. It won’t become the most popular choice perhaps, but it’d be fairly common. It would also mean that a fast Ruby based on a Smalltalk-like VM would be possible, and that there’s hope that Rubinius could achieve similar results.
@Wilson, @Santana: Useless tests that can be optimized out, will be removed from the next shootout and new “general benchmarks” will be introduced.
@Vladmir, I’ve no doubt that becoming fully compatible with Ruby will slow down things. However, if MagLev were to be THAT fast in my new independent tests, then perhaps, at the end of the day, they’ll still come out ahead by a good margin. I read Headius’ post and I think that he raised a lot of good points, but he wasn’t exactly fair in a few of them.
@Francesco, OODB is an alternative to RDBMS. They are closed source, because that’s how they make money. 😛
@Greg, you came here to read about a promising commercial software. It doesn’t have to be everyone’s cup of tea, of course.
You should include MacRuby in your benchmark as well:
http://ruby.macosforge.org/trac/wiki/MacRuby
@jp_tix: Good point. On Mac OS X, I will.
@Antonio, just pitching in to suggest a type of performance test known in some circles as “Processing a Shitload of Logs”. That would see object instantiation and gc in action under heavy load.
Can’t wait to see your next shootout. Having a better performance test suite is going to be refreshing, actually 🙂
Also, do you plan on checking memory consumption on these tests?
Quote: “Your comment is awaiting moderation.”
Get a defensio.com account, already! We have a great WP plugin 🙂
Your ‘Holy Shit’ comment made it into their presentation; nice big slide with just those two words.
Thanks for the benchmarks and keeping us informed on the changes going on!
@Scott asked, “Does the Gemstone OO database technology support replication, hot backups, and failovers?”, the answer is yes. If you followed, Avi Bryant’s presentation:
http://www
One can see that MagLev supports replication and will have support for failovers very soon. For example, he started two VM in the video where each VM knows about a shared object instance. If one or both of the MagLev VMs goes away after a commit, he can simply start at least one MagLev VM to get back where he left off. I say that because this is how Gemstone/S works now and have been working like this for well over 15+ years.
I’m pretty sure “17 petabytes” is just marketing speak for “this compiles on an OS that runs on x86-64 CPUs.”