Comments on: Reflections on the Ruby shootout

By: sync ruby gems between different installed ruby versions | Screenage

sync ruby gems between different installed ruby versions | Screenage — Sun, 07 Jun 2009 14:22:35 +0000

[…] If you are in the Ruby business (which probably means “in the Ruby on Rails business” nowadays) sooner or later you’ll have to play around with different Ruby versions on the same machine as you might run into crashing ruby processes or performance issues . At least you’ll notice that running the standard Debian/Ubuntu Ruby versions might get you into serious trouble as it is several times slower than a manually compiled version (for reference see this launchpad bug and this blog entry.). […]

By: Brian Takita

Brian Takita — Tue, 23 Dec 2008 18:18:57 +0000

I filed a bug report on Ubuntu’s bug tracking site.

https://bugs.launchpad.net/ubuntu/+source/ruby-defaults/+bug/307462

By: roger pack

roger pack — Sat, 13 Dec 2008 12:14:13 +0000

You should totally test Lavena’s new “mingw for windows” build, in these tests. It helps windows be a faster 🙂
How to get:
git clone http://github.com/luislavena/rubyinstaller/tree/master
then
rake # create a 186p286 build
or
rake CHECKOUT=1 # to create a 186 SVN trunk build
these will create mingw build in sandbox/ruby_mingw
for use

Also it would be interesting to see the RAM usage per test [as the alioth benchmark shows, along with speed].

Some other comments:
testing against 186 [linux] would also be interesting.

Thanks for doing this!

By: Antonio Cangiano

Antonio Cangiano — Fri, 12 Dec 2008 22:54:45 +0000

No Sam, MagLev was tested against Ruby 1.8.6 built from source. The only VMs affected were those within the “main shootout”.

By: Sam Livingston-Gray

Sam Livingston-Gray — Fri, 12 Dec 2008 22:41:38 +0000

Interesting. With that adjustment, then, it looks as though MagLev is then comparable to 1.8.7 in performance (for the subset of benchmarks it’s able to complete)?

By: Isaac Gouy

Isaac Gouy — Thu, 11 Dec 2008 18:41:42 +0000

In that case, let me suggest that you re-measure just those “Timeout” benchmarks, just for Rubinius, using a wildly high cutoff.

As a practical matter that shouldn’t be too bad – back in the day Rebol nbody program needed 4 hours to finish at the largest input size, but Rubinius should be much faster on the smaller input sizes used in Ruby Shootout.

It’s such a shame to exclude benchmarks that actually require a tiny amount of work because of Rubinius.

By: Antonio Cangiano

Antonio Cangiano — Thu, 11 Dec 2008 18:17:17 +0000

Isaac, they are not. Only the 101 benchmarks that were successful for all the VMs (in blue within the tables) contribute to the totals.

By: Isaac Gouy

Isaac Gouy — Thu, 11 Dec 2008 18:13:33 +0000

Rubinius shows “Timeout” for several benchmarks – are measurements for the other implementations on those benchmarks where Rubinius shows “Timeout” included in the “Geometric Mean” headline chart?

By: Isaac Gouy

Isaac Gouy — Thu, 11 Dec 2008 17:48:44 +0000

antonio > Ruby 1.9 and JRuby are very close, respectively 2.5 and 1.9 faster than Ruby 1.8.7 (from source) on these benchmarks.

1) similar to the benchmarks game ruby 1.9.0 :: ruby 1.8.6

http://shootout.alioth.debian.org/gp4/benchmark.php?test=all&lang=yarv&lang2=ruby

2) similar to the benchmarks game jruby 1.1.5 :: ruby 1.8.7

http://shootout.alioth.debian.org/u32/benchmark.php?test=all&lang=jruby&lang2=ruby

3) Unlike the Ruby Shootout, the benchmarks game /does/ include startup time and /does/ include the time required to parse and compile classes and method for the first time.

By: mbailey

mbailey — Thu, 11 Dec 2008 16:11:09 +0000

Hmm. Now I get it:

compiled_ruby-1.8.6p287 is what we use in production, and it is build from scratch (simple ./configure && make && make install)
compiled_ruby-1.8.7p72 is the latest stable version compiled in the same way (./configure && make)
deb_ruby-1.8.7p72 is the binary from ubuntu 8.10
rpm_ruby-1.8.5p0 is the latest version in CentOS 5.2 yum repo

$ ls -shw1
total 5.3M
2.6M compiled_ruby-1.8.6p287
8.0K compiled_ruby-1.8.7p72
2.7M deb_ruby-1.8.7p72
12K rpm_ruby-1.8.5p0

$ for i in *; do echo -n "$i: "; strings $i | wc -l ; done
compiled_ruby-1.8.6p287: 7533
compiled_ruby-1.8.7p72: 27
deb_ruby-1.8.7p72: 7642
rpm_ruby-1.8.5p0: 33

$ for i in *; do echo "$i: "; ldd $i ; echo "" ; done
compiled_ruby-1.8.6p287: 
	linux-vdso.so.1 =>  (0x00007fff569fe000)
	libdl.so.2 => /lib/libdl.so.2 (0x00007fe54e58e000)
	libcrypt.so.1 => /lib/libcrypt.so.1 (0x00007fe54e356000)
	libm.so.6 => /lib/libm.so.6 (0x00007fe54e0d1000)
	libc.so.6 => /lib/libc.so.6 (0x00007fe54dd5f000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fe54e792000)

compiled_ruby-1.8.7p72: 
	linux-vdso.so.1 =>  (0x00007fffc5fff000)
	libruby1.8.so.1.8 => /usr/lib/libruby1.8.so.1.8 (0x00007f3ebd9e5000)
	libpthread.so.0 => /lib/libpthread.so.0 (0x00007f3ebd7c9000)
	libdl.so.2 => /lib/libdl.so.2 (0x00007f3ebd5c5000)
	libcrypt.so.1 => /lib/libcrypt.so.1 (0x00007f3ebd38d000)
	libm.so.6 => /lib/libm.so.6 (0x00007f3ebd108000)
	libc.so.6 => /lib/libc.so.6 (0x00007f3ebcd96000)
	/lib64/ld-linux-x86-64.so.2 (0x00007f3ebdceb000)

deb_ruby-1.8.7p72: 
	linux-vdso.so.1 =>  (0x00007fff309fe000)
	libdl.so.2 => /lib/libdl.so.2 (0x00007ffd2840a000)
	libcrypt.so.1 => /lib/libcrypt.so.1 (0x00007ffd281d2000)
	libm.so.6 => /lib/libm.so.6 (0x00007ffd27f4d000)
	libc.so.6 => /lib/libc.so.6 (0x00007ffd27bdb000)
	/lib64/ld-linux-x86-64.so.2 (0x00007ffd2860e000)

rpm_ruby-1.8.5p0: 
	linux-vdso.so.1 =>  (0x00007fffa8bfe000)
	libruby.so.1.8 => not found
	libpthread.so.0 => /lib/libpthread.so.0 (0x00007fa7a063f000)
	libdl.so.2 => /lib/libdl.so.2 (0x00007fa7a043b000)
	libcrypt.so.1 => /lib/libcrypt.so.1 (0x00007fa7a0203000)
	libm.so.6 => /lib/libm.so.6 (0x00007fa79ff7e000)
	libc.so.6 => /lib/libc.so.6 (0x00007fa79fc0c000)
	/lib64/ld-linux-x86-64.so.2 (0x00007fa7a085b000)