The Great Ruby Shootout (Windows Edition)

This post contains the results of a Ruby shootout on Windows that I recently conducted. You can find the Mac edition, published last month, here. I was planning to have this one ready much sooner, but a couple of serious events in personal life prevented that from happening. Be sure to grab my feed or join the newsletter to avoid missing the upcoming Linux shootout.

The setup

For this shootout I included a subset of the Ruby Benchmark Suite. I opted to primarily exclude tests that were executed in fractions of a second in most VMs, focusing instead of more substantial benchmarks (several of which come from the Computer Language Benchmarks Game). The best times out of five runs are reported here for each benchmark.

All tests were run on Windows 7 x64, on an Intel Core 2 Quad Q6600 2.40 GHz, 8 GB DDR2 RAM, with two 500 GB 7200 rpm disks.

The implementations tested were:

  • Ruby 1.8.7 (2010-01-10 patchlevel 249) [i386-mingw32] (RubyInstaller)
  • Ruby 1.9.1 p378 (2010-01-10 revision 26273) [i386-mingw32] (RubyInstaller)
  • Ruby 1.9.2 dev (2010-05-31) [i386-mingw32] (experimental)
  • JRuby 1.5.1 (ruby 1.8.7 patchlevel 249) (2010-06-06 f3a3480) (Java HotSpot(TM) 64-Bit Server VM 1.6.0_20) [amd64-java]
  • IronRuby 1.0 x64 for .NET 4.0

JRuby was run with the --fast and --server optimization flags.

Disclaimer

Synthetic benchmarks cannot predict how fast your programs will be when dealing with a particular implementation. They provide an (entertaining) educated guess, but you shouldn’t draw overly definitive conclusions from them. The values reported here should be assumed to be characteristic of server-side – and long running – processes and should be taken with a grain of salt.

The results

Please find below the execution times for the selected tests. Timeouts indicate that the execution of a single iteration for a given test took more than 60 seconds and had to be interrupted. Bold values indicate the best performance for each test.

RBS Windows Shootout

Conclusions

Despite a couple of errors and a few timeouts, JRuby was the fastest of the lot, which can be seen as impressive if we consider that this is Windows we are talking about after all.

Ruby 1.9.1 and 1.9.2 were almost as fast as JRuby on these tests. With a few exceptions, the performances of the two 1.9 implementations were, expectedly, very similar.

JRuby, 1.9.1 and 1.9.2 were all faster than the current MRI implementation, which can be seen as a prerequisite as we move, as a community, away from Ruby 1.8. Finally, it’s worth noting that IronRuby’s performance was however in line with that of Ruby 1.8.7.

Update (July 3, 2010): The following box plot compares the various implementations for the tests for which all the implementations were successful. Only times for the largest successful input number were used in those tests where multiple input numbers were tested.

Windows Shootout Boxplot

Get more stuff like this

Subscribe to my mailing list to receive similar updates about programming.

Thank you for subscribing. Please check your email to confirm your subscription.

Something went wrong.

22 Comments

  1. Luis Lavena June 28, 2010
  2. Isaac Gouy June 28, 2010
  3. Isaac Gouy June 28, 2010
  4. Luis Lavena June 28, 2010
    • Isaac Gouy June 28, 2010
      • Luis Lavena June 29, 2010
  5. Jimmy Schementi June 28, 2010
    • Antonio Cangiano June 28, 2010
      • Jimmy Schementi June 28, 2010
        • Antonio Cangiano June 28, 2010
  6. Brazen June 28, 2010
    • Antonio Cangiano June 28, 2010
      • Brazen June 29, 2010
        • Antonio Cangiano June 29, 2010
  7. Jim Gay June 29, 2010
  8. Earle Clubb June 29, 2010
  9. Michael Campbell June 30, 2010
    • Antonio Cangiano June 30, 2010
  10. raggi July 1, 2010
    • Antonio Cangiano July 1, 2010
  11. Pingback: The Great Ruby Shootout (July 2010) July 19, 2010

Leave a Reply

This site uses Akismet to reduce spam. Learn how your comment data is processed.