In yesterday’s post I compared IronRuby 0.9, Ruby 1.8.6 (from the One-Click Installer) and Ruby 1.9.1 (downloaded from the official site) against one another. IronRuby did great, but the discussion in the comment section quickly veered towards what version of the One-Click Ruby Installer should have been used.
I justified my choice of using the “old” One-Click Installer, by the fact that I wasn’t aware of official releases of the new installer. As well as that the old One-Click Installer is the most widely downloaded version. Very few people are familiar with the upcoming version of the project. This point is about to change.
Luis Lavena took over the One-Click Installer project and has been working on the next version (RubyInstaller from now on), the aim of which is to replace the One-Click Installer by building Ruby 1.8 and 1.9 with MinGW and GCC. In theory, this brings performance gains on Windows to the table, and gets rid of having to use Visual C++ 6 (a “ten year old compiler”) to build Ruby and other native gems. The project also strives to be lighter by bundling fewer (unnecessary) gems for Windows users.
There’s no doubt that in the long run, this new project will become the de facto standard for Windows, but the questions on everyone’s mind are, should I bother with it now? How much of a performance boost are we talking about here? 10%? 20%? Let’s find out.
In this follow up article I’m going to compare the performance of Ruby 1.8.6 from the One-Click Installer (mswin32), Ruby 1.8.6 from RubyInstaller (mingw32), Ruby 1.9 (mswin32) downloaded from the Ruby-Lang.com site, and Ruby 1.9 from the RubyInstaller project (mingw32) against one another. I’ll copy and paste part of the setup and disclaimer from yesterday’s post, for those who haven’t read it. Feel free to skip this part if you wish.
The table/image below shows the times for each benchmark, for Ruby 1.8.6 (mswin32), Ruby 1.8.6 (mingw32), Ruby 1.9.1 (mswin32), and Ruby 1.9.1 (mingw32). In the table I used (RI) as shorthand for RubyInstaller to indicate mingw32 versions.
Red values are errors, timeouts and inapplicable tests. Green, bold values indicate better times than what Ruby 1.8.6 (mswin32) delivered. A pale yellow background indicates the best time for a given benchmark. Total time is the run-time for the subset of benchmarks that were successfully executed by all four implementations. Timeouts have been included this time around. Each timeout has been counted as an additional 300 seconds.
The total runtime (including timeouts) is summarized in the chart below:
Wow! Ruby 1.8.6 (mingw32) improves from 3% to 664% (depending on the test), over the current One-Click Installer. The geometric mean of the ratios (read “on average”) tells us that it was about 283% faster. The Ruby 1.9.1 version provided by the RubyInstaller was slower than the mswin32 version in a couple of tests, but faster everywhere else. How much faster? Up to 342% faster, with an average (again calculated through the geometric mean of the ratios) of a 77% increase in speed.
These finds prompt me to ask, what are we waiting for? You know how unresponsive Ruby is on Windows, and how tests take forever to execute? These mingw32-based releases may very well solve this. And incidentally the bar has been raised for IronRuby as well. I formally invite the Ruby on Windows community to embrace these two projects.
I sincerely welcome and appreciate your comments, whether in agreement or dissenting with my article. However, trolling will not be tolerated. Comments are automatically closed 15 days after the publication of each article.