Over the years the inadequacy of Ruby’s main implementation has led to the creation of several alternatives. The greatest common divisor between these is an attempt to improve the performance of Ruby, both in terms of time and space. But every Ruby implementation has another, deeper reason for being. For example, Ruby 1.9.1 is a refactoring of the language that provides the chance to incorporate several much needed features into a relatively fast virtual machine, whereas JRuby’s truest value lies in its ability to interact with the Java ecosystem. Likewise IronRuby (which is admittedly at an earlier stage in its development) is attempting to plug a dynamic language like Ruby into the .NET world (as per its predecessor IronPython).
While MacRuby is a younger, lesser known implementation, it has the potential to become a game changer – at least for Mac developers. Based on Ruby 1.9, MacRuby’s main aim is to provide programmers with the ability to write Mac OS X applications in Ruby, making Ruby a first class Cocoa programming language. In what may sound like an utopistic effort, MacRuby strives to provide the high level abstractions, power and syntax sugar of Ruby, without the characteristic performance hit of its main implementation.
Rather than relying on a mix of Objective-C and Ruby (through a bridge like RubyCocoa), developers can use MacRuby which integrates with Mac OS X core technologies and acts as an alternative language to Objective-C. To be exact, Objective-C’s runtime and generational garbage collector are at the heart of MacRuby, but from an API standpoint, programmers can write code in Ruby, instead of in the more verbose and low level Objective-C. MacRuby maintains the ability to integrate with Objective-C code, but doing so is often unnecessary thanks to a framework known as HotCocoa, which is a Ruby wrapper around the Cocoa API.
MacRuby applications end up being succinct, easy to write and straightforward to maintain. Their look and feel is exactly the same as those of applications written in Objective-C, because they are actually native Cocoa applications. In fact, MacRuby objects are Objective-C objects which use Core Foundation data types and services. The current version, MacRuby 0.4, even allows you to package applications as self-contained .app, without having to redistribute MacRuby itself.
Sponsored and developed by Apple, MacRuby 0.4 is a stable release that can be – and already is – used to write desktop applications. But MacRuby’s real promise lies in its experimental branch. This rewrite will become MacRuby 0.5, as announced by the MacRuby team earlier today (along with a new, nice looking site). This future version of MacRuby is freakishly fast and uses the LLVM to generate code for the Objective-C runtime. The layer composed of LLVM, ObjC Runtime, Generational Garbage Collector and Core Foundation make this specific Mac OS X release possible. To this layer, add in Ruby 1.9’s AST parser and Standard Library, and you get MacRuby 0.5 as it stands today. In the future it’s likely that applications built with MacRuby will be compiled into binary code, like Objective-C ones, thus removing the issue of protecting one’s source code in commercial applications.
To really understand the value of MacRuby, consider the following Hello World program that uses RubyCocoa:
require 'osx/cocoa'; include OSX
app = NSApplication.sharedApplication
win = NSWindow.alloc.initWithContentRect_styleMask_backing_defer(
[0, 0, 200, 60],
NSTitledWindowMask|NSClosableWindowMask|NSMiniaturizableWindowMask|NSResizableWindowMask,
NSBackingStoreBuffered, false)
win.title = 'Hello World'
button = NSButton.alloc.initWithFrame(NSZeroRect)
win.contentView.addSubview(button)
button.bezelStyle = NSRoundedBezelStyle
button.title = 'Hello!'
button.sizeToFit
button.frameOrigin = NSMakePoint((win.contentView.frameSize.width / 2.0) - (button.frameSize.width / 2.0),
(win.contentView.frameSize.height / 2.0) - (button.frameSize.height / 2.0))
button_controller = Object.new
def button_controller.sayHello(sender)
puts "Hello World!"
end
button.target = button_controller
button.action = 'sayHello:'
win.display
win.orderFrontRegardless
app.run
And compared to the equivalent MacRuby and HotCocoa program:
require 'hotcocoa'
include HotCocoa
application do |app|
win = window :size => [100,50]
b = button :title => 'Hello'
b.on_action { puts 'World!' }
win << b
end
The first approach reminds me of Objective-C, the second is a pure Ruby DSL.
Benchmarking the experimental branch
The question on many readers minds is probably, how fast is it? To begin with, the start-up time is negligible which is a good quality for desktop applications to have. Using the YARV tests that ship with the experimental branch, let me show you the numbers I obtained on a Mac Pro with two Quad-core Intel Xeon 2.8 GhZ and 18 GB of RAM. The tests were also run on much less “beefier” hardware with analogous results.
All the usual disclaimers apply here. These are just a few very basic micro-benchmarks that should give you a “feel” for how two VMs stack up against each other; but don’t read too much into this and don’t expect it to be a scientific report on the performance of the implementations that were tested. Also keep in mind that MacRuby 0.5 is currently an experimental release, and while it’s able to pass RubySpec’s language specifications, it is not a complete implementation so far. The team is taking incompatibility issues seriously though and will make sure MacRuby will be able to run any Ruby code.
The following table summarizes the results of these benchmarks for Ruby 1.8.6, Ruby 1.9.1 and MacRuby 0.5:
This table shows the ratios between Ruby 1.9.1 and MacRuby 0.5, respectively, against the Ruby 1.8.6 baseline. When you see a number 4, for example, that means that the given implementation was four times faster than Ruby 1.8.6.
And here is a direct comparison between Ruby 1.9.1 and MacRuby 0.5. In this case, a number 4 (for example) would mean that MacRuby was four times faster than Ruby 1.9.1 for the given test:
The following chart should help you better visualize the results shown in the first table (click on it to enlarge the picture):
Even when you consider the disclaimer above and the trivial nature of the benchmarks themselves, it’s clear that at this stage of the game, MacRuby 0.5 is built for speed. the fastest Ruby implementation around. MacRuby literally dominates Ruby 1.9.1. On “average”, according to these limited tests the experimental branch of MacRuby appears to be roughly 3 times faster than Ruby 1.9.1 (YARV), and in some cases even faster than that. You should definitely find this impressive.
For full disclosure, I’d like to explain the most likely reasons behind the four tests in which MacRuby is slower than Ruby 1.9.1:
- bm_app_raise: MacRuby opts for cost free IA64 exceptions. What this means is that begin/rescue clauses don’t require a setjmp() like YARV does, but in case of exceptions at runtime, raising an error is more expensive. Of course, exceptions are… well, exceptional, so this has a trivial impact on real world programs.
- bm_app_mergesort: Array operations are currently suboptimal. Slight improvements are expected.
- bm_so_object: MacRuby’s object allocation tends to be relatively expensive. If you are allocating a zillion objects in a test, MacRuby will pay a hefty price for it.
- bm_so_random: The performance of Fixnum has been optimized in this early release, but both Bignum and floating point operations are still suboptimal. Work is in progress and major improvements are expected to occur in future versions. In the case of Bignum, vectorization will do the trick.
The take home lesson
MacRuby is a serious project that fits in, and serves the flourishing market of Cocoa applications well. Its experimental branch is a major rewrite that, grants this implementation the title of fastest Ruby in the West that, much like Unladen Swallow for CPython, could become a very fast alternative to Ruby 1.9.x and JRuby. It’s mainly aimed at the desktop world, and as such the question of when it will work with Rails, is less pressing than it was in other early Ruby implementations. What’s certain is that Ruby is going to become a first class “scripting” language and a common choice for desktop applications on Mac. And should the difference in performance still remain in future, stable versions, it’s not hard to imagine that Apple’s server segment could also benefit from this, when Rails support becomes available.
Update (2009-04-04): I’ve modified a controversial statement since, technically speaking, while it is true that a very fast, incomplete implementation is promising and worth getting excited about, it cannot be considered the fastest Ruby implementation until a great degree of compatibility has been reached. This doesn’t diminish the value of MacRuby in any way, but rather draws a more accurate conclusion about the data that’s available today. It’s also worth noting that the few benchmarks that have been mentioned here are only part of the story. Speed aside, MacRuby’s aim and potential for Mac development still stand.
The image at the top of the post was generated by James Reynolds with HotCocoa::Graphics, a Processing-like library that uses Mac OS X’s graphics capabilities and makes them available to MacRuby.
Get more stuff like this
Subscribe to my mailing list to receive similar updates about programming.
Thank you for subscribing. Please check your email to confirm your subscription.
Something went wrong.
Great article Antonio!
To be fair even tho MacRuby mainly targets native OSX application developers, it will eventually be able to run Rails and should be portable on any posix systems. Deploying a Rails app that runs virtually 3x faster than when running on yarv and use less memory is something that might happen sooner than later.
Also one can dream of compiling iphone apps written in Ruby!
-Matt
You do a disservice to your readers and to other implementations with your knee-jerk blog posts about every partially completed implementation that comes around the corner. Weren’t you crowing about MagLev a little less than a year ago, before they had even started running specs? Wasn’t it a little premature?
I think the improvements in MacRuby are fantastic, but there’s a lot more work to do. About half of the benchmarks I tried to run simply broken, and I found many key core behaviors have not yet been implemented, including several that will make some of the current optimizations difficult to maintain.
I have no doubt MacRuby will continue to post excellent perf numbers in many areas, but declaring them the “fastest Ruby implementation” before they can even run applications on the new branch is absurd. MacRuby is not a Ruby implementation until it can run RubySpecs and Ruby applications. It’s the last 10% or 1% of compatibility that really makes Ruby hard to optimize, and they’re just getting to that now.
So take a breath, calm down, and let these guys continue their excellent work without you hyping them all out of proportion.
Wow! Good read! Excellent job!
Ok…I’m interested, I’m interested! 🙂
Gotta look for the best place to start gathering relevant docs now.
Charles: Without a doubt, we have a very long way to go before we can call ourselves wholly compatible with Ruby 1.9. However, I don’t think Antonio’s post is hyperbole or fanboyism – a quick glance at our SVN commit logs will prove that we are progressing at rates that other Ruby implementations simply cannot match. Add in the true innovations like HotCocoa, and we have a platform that, in my opinion, deserves the hype it’s getting.
Antonio: Thanks for a fantastic article! It’s great to see MacRuby bounce around the blogosphere.
I use a Mac, and am very excited by MacRuby and it’s potential for creating Cocoa apps, Mac scripting (command line and Applescript) and iPhone apps.
However, I feel your graph it somewhat misleading:
There are three test where MacRuby is 5 times, 4 times or twice as slow as 1.8.6 and yet these are effectively hidden by your presentation. Compare them with the YARV stats that are in the same ballpark but positive.
But let me state that I’m excited by MacRuby, I think it’s good for the Mac and good for Ruby!
Will the work done on the LLVM based VM be portable, or tied closely with the MacRuby implementation? Basically, I’m asking how portable this new VM will be to other platforms that don’t have CoreFoundation. Anybody have an idea?
It’s funny this is being announced just about the same time as Google’s “Unladen Swallow” project, which aims put Pythom on LLVM.
> may sound like an utopistic effort
The word you’re looking for is “Utopian”.
Just started to play with MacRuby. Very cool stuff. Would love to see this used in a SDK for iPhone development… from what I understand the hang up here is mostly about the garbage collector.
@Craig, both adjectives are correct.
I think everyone needs to realize that MacRuby is based on Ruby 1.9’s C code base/stdlib. This is not an attempt to rewrite the thing in Java like JRuby did, or rewrite the whole thing in Ruby/C++ like Rubinius did. Also, this effort should not be compared to Maglev, which, again, was a from scratch implementation.
There are substantial structural changes with merging the object models and using the new GC, etc. These are not minor, but at the same time its not a from scratch implementation. The goals of MacRuby are to enjoy the power of Ruby, REAL RUBY for OS X development. OS X is an exciting platform and this enables writing full blown desktop applications with Ruby. Performance is often more critical for desktop apps than server apps since you cannot scale out hardware for lack of a performing runtime.
The new VM replaces YARV but its working off the 1.9 AST, not some attempt to recreate the AST with some untested parser. The VM project was necessary to achieve the level of performance that desktop applications demand and to increase universe of applications that can be built with it.
I am not trying to minimize the challenges that Laurent, et al have faced and will face to achieve compatibility with 1.9 but I have full faith that they will get there and soon. Don’t think that Laurent is naive in his thinking on how hard things are, he understands C-Ruby’s runtime very, very well.
I do not expect performance to go down with MacRuby I expect to to rapidly improve. I say with with firsthand experience as Laurent has developed the new VM over the last 3 months (yes, this whole VM has been done in just 3 months). He has sent me the micro-benchmarks as he has discovered new ways to optimize the VM. I think these benchmarks, while not representing real-world performance, are important indicators of troubled areas of the runtime or advantages of the runtime that can be exploited. This post about MacRuby’s performance is real and Antonio’s excitement is well founded (thank you Antonio for writing it). I look forward to MacRuby’s future and I will strive with others to make it the best Ruby runtime in the world.
In my experience object allocation is one of the areas where Objective-C is very slow – you just cannot get away with writing the same kind of code you would in Java (where object allocation is very cheap) as the performance will be horrible.
So given the Ruby -> Objective-C object mapping in MacRuby I was not very surprised to see the slow object benchmark times. I also suspect that the scope for improvement in this area will be quite limited without improvements to the Objective-C runtime.
This seems like a pretty fundamental performance metric for a language where “everything is an object” – so it will be interesting to see how this works out for “real life” Ruby code. Developers may have to change their programming idioms to fit MacRuby’s performance characteristics.
That said, this is a very interesting development especially for those of use with an interest in Cocoa application development. Interfacing scripting languages with Cocoa is a pretty hot area right now, and there is some great stuff going on with Javascript too – which will take full advantage of the effort Apple is putting into Javascript performance.
Mark: object allocation in Objective-C is a bit slow when used in the garbage collected environment, because it’s no longer allocating memory from the default malloc zone but from a special zone handled by the collector. However there are ways to work around the problem by implementing a per-thread allocation cache and this is very likely something we will investigate soon. However, unless you are allocating lots of objects in a loop this should not I think impact the typical Cocoa application that you will write with MacRuby.
Charles, there is no misrepresentation of the facts. You’re taking issue with my remark about the fastest implementation because MacRuby is incomplete. Yes, I’m sure this will have an impact, but I disagree with you on the basis of the drastic improvement over YARV on the tests that pass.
It’s like bashing YARV when it wasn’t complete, and claiming that it wasn’t faster than Ruby 1.8, just because it couldn’t run many tests and didn’t have full coverage of the language. Performances may decline in an attempt to cover the whole language, but I suspect that it will still be faster than the current versions of YARV and JRuby, based on these numbers. You disagree, and that’s fine.
But please understand that all I’m doing is bringing attention to lesser known, truly promising projects. I’ve been following MacRuby for months before I made a post about it, so there wasn’t much of a knee-jerk reaction on my part.
Also keep in mind that the speed of the experimental branch compared to other VMs is not the only subject of this article. My piece talks about the current version of MacRuby, what this is good for, and what the next version will bring to the table.
I’ll follow up in the mailing list in regards to my strategy for more realistic benchmarks.
Great read Antonio, thanks for writing about this implementation. I’ve been following the work Laurent has done these last couple of months very closely and have had the privilege to be able to discuss some of the optimization ideas that he was having. From these conversations, it’s without a doubt that I’m convinced that Laurent is on to something. Sure, running the RubySpecs is a target that will likely affect performance a bit, but both Laurent and Antonio aren’t people who don’t realize this. From discussions with Laurent, I know for a fact that this guy will eat up computer architecture documents just to see if there are some mechanisms to optimize the code, and it is for this reason that I believe that being optimistic about this project wouldn’t be a disservice to any reader at all. In fact, doing the contrary would imo be a disservice. Further more, despite all the magnificent achievements Laurent has made, he’s a very humble guy so I don’t think cheering him on will affect anything in a negative sense. If anything, it’ll probably just raise the bar for him and make the challenge a bit more interesting.
“Its experimental branch is a major rewrite that grants this implementation the title of fastest Ruby in the West.”
That’s the issue Antonio. MacRuby isn’t a fully qualified Ruby yet. And don’t get me wrong, i’m really jazzed about MacRuby, and really hope they do kick ass.
But, I don’t know what the performance to completeness landscape looks like, and how rocky and difficult the climb to 99% completeness is going to be.
In that regard i do think it’s worth heeding headius’s caveats, both in terms of interpretation of the numbers, and conclusions we want to draw about the ruby landscape.
I think it’d be really interesting to do a comparative analysis of how performance changed over the history of each implementation’s repository (guess maglev will have to sit that one out) in relationship to an implementation’s completeness. But until we do, i really am hesitant to ascribe what the future performance of a Ruby is going to be, based on the numbers that they produce now.
I wonder if MacRuby and HotCocoa already can be used for rapid GUI prototyping.
I needed a small app with a GUI that just did some http requests. I tried it in Shoes, but decided Cocoa was a better fit becasue I needed Interface Builder for some quick alignment.
I wasn’t happy with Objective-C at that point though, I needed to do some string concatenation and I would much rather use “#{url}&#{var}&#{var2}” instead of [url stringByAppendingString [@”&” stringByAppendingString var]] …
Huzzah, yet another Mac-only open source project. So while Linux and Windows developers target all platforms, Mac developers continue to write software that works nowhere else. Nice job.
Ruby is way to cryptic for me, thinks like “win = window :size => [100,50]” doesn’t make sense 😀
But then again, I am used to C, C++ and Java.
Then ruby just looks weird…
Zizo
@Richard that’s the prerogative of each developer. I know plenty of developers who do Windows only programming (to include Vista only beyond that). Not to mention iPhone apps vs whatever else is out there. Not everything has to be cross-platform.
@Richard Wekson, MacRuby is built on top of CoreFoundation http://en.wikipedia.org/wiki/Core_Foundation which that can be used to write cross-platform applications for Mac OS X, Linux, and Windows(via Cygwin).
MacRuby can be compiled on Linux and available on Windows via Cygwin. However because cocoa isn’t available on other platforms, you can’t develop a cross platform GUI, the same way you can’t build a cross-platform .NET GUI app.
– Matt
Sweet graph. It very clearly shows that a benchmark made of half the code from bm_app_fib and half from bm_app_raise will run at 10x the speed on MacRuby! Oh, wait…
Joining lines between apples and aardvarks on a graph is not going to improve people’s understanding of the relative performance of the two implementations.
Boring old two bars per benchmark for the two implementations (Ruby 1.8 unnecessary, it’s the default y=1.0 line) would be easier to read. You could also sort the benchmarks by the MacRuby multiplier so that it becomes clearer for how many cases MacRuby is giving a speedup vs. a slowdown, which might answer @dave’s misgivings about the graph.
-Jason
IA64 != X86_64
@Zizo the line “win = window :size => [100,50]” can be written (as it is Ruby 1.9) as
win = window(size: [100,50])
I prefer this style and suspect that you might as well.
-Robert
You know what would look better than the current HotCocoa example?
require 'hotcocoa' include HotCocoa application do window( :size => [100,50] ) do |win| win < 'Hello' ) do |b| b.on_action{ puts 'World!' } end end endOr, with a nice instance_eval, something like:
require 'hotcocoa' include HotCocoa application do window( :size => [100,50] ) do add button( :title => 'Hello' ) do on_action{ puts 'World!' } end end endA line graph doesn’t make sense here; use a bar graph.
I’m going to wager that object creation under Ruby already suffers the same issues as object creation in Obj-C vs Java – in much the same way as Smalltalk vs C++.
How about porting it to OpenStep? I don’t have the same appreciation for shiny hardware as Steve Jobs.
Ugh I meant “porting it to GnuStep”.
Having taken the Python fork at the Ruby/Python fork in the road a while back – I just wish someone was doing something similar for Python 🙁
Maybe I’ll be looking at Ruby again in a couple years – I’m trying to reduce the constant juggling between different languages…
If you want to be taken seriously, learn to draw graphs right. Line graphs should always have two continuous axes. Your graph there should be a bar chart.
@Thomas, I’m familiar with the concept. The equivalent bar/column chart was very crowded and hard to read at 600px. I will create a small bar chart version that links to a larger one.
I REALLY hope this compiles on Linux. Any tutorials out there? Is it stable? Does it run rails? what’s the memory usage like?
🙂
-=r
how does the speed compare to maglev? 🙂
Nice for simple application development, but I often find myself needing libraries for functions that aren’t available in either Objective-C nor Ruby such as Boost or specialized mathematics, simulations, etc. So this doesn’t help me for most of my work. But for a nice quick GUI app that I only need on OS X and not Windows or Linux it puts my previous favorite wxPython as well PyQt, PyObjC (and of course Cocoa-Ruby) to shame. I wish somebody would do this for Python given my years of Python experience but I guess I’ll just have to learn Ruby. It will be worth it.
Hi Antonio,
I hope that you do not mind this long post, and are willing to read through it completely.
And possibly give me some good advice, and point me in the right direction.
I am a i5/System i ILERPG developer.
A craft that is getting less and less in demand here in The Netherlands. (One year without a contract already.)
This graph (only US based I believe) plots the situation more or less accurately : http://www.indeed.com/jobtrends?q=ruby%2C+rails%2C+rpg&l=&relative=1
So I need and want to learn new “tricks”.
I got interested in Ruby and On Rails somewhere in 2007 through this article : http://www.ibm.com/developerworks/data/library/techarticle/dm-0705chun/ and this website : http://midrange-ruby.org/.
But the impossibility/inability of companies offering IBM midrange timesharing, to provide a working Ruby (On Rails) environment on i5/OS PASE, and support it also, made it unsuitable for selfstudy.
You can sdd to that the additional premium of timesharing on i5 instead of older System i hardware.
I’ve just started to add the Mac OS X platform to my skill sets. Starting with OO.
After feeling comfortable with OO, the next planned step is MacRuby. (After reading your excellent article about it here.)
But your other articles here about DB2 on Mac OS X Snow Leopard, and about DB2 RoR performance, over at IBM’s website, resparked the idea of using Ruby on i5 again.
It makes me wonder if I could kill 2 birds (possibly more) with one stone.
Learn OO, Ruby, On Rails and be able to use it on Mac OS X and also i5 (using my existing skills) and other platforms.
But MacRuby is locked solid into the Mac platform. So I doubt if that is the way to go.
Also, I find it hard to find out if Ruby on i5 is in demand at all. The fact that the agencies I’m registered with do not even know about Ruby on i5, should probably give me a hint.
What would be a good approach to achieve this ?
Please, when possible, could you mention the names of specific resources, products, and development tools (used by large companies). And I’ll take it from there.
(Also, I cannot afford products like “IBM Rational Developer for Power Systems Software” and “IBM Rational Developer for i”). 😉
I hope that you have got the time for this.
Thanks in advance.
Whoops! Correction.
Please replace i5 with AIX in this sentence :
“Also, I find it hard to find out if Ruby on i5 is in demand at all. The fact that the agencies I’m registered with do not even know about Ruby on i5, should probably give me a hint.”