Get weekly emails about new Programming books

Today I’m announcing a pet project of mine that I think may interest some of my readers. Any new books? is a free notification service which enables you to subscribe to a series of subjects that interest you and receive weekly emails about new book releases in those categories.

These books are hand-selected, to filter out obvious duds, and include only books that appear to be promising/interesting. Naturally I have included programming among the 42 available categories.

And before you ask, I’ll clear this up from the get-go. This differs from Amazon’s newsletters, because it’s far more systematic. With this service you can be in the loop when it comes to subjects you really care about. I believe Amazon’s occasional newsletters are more targeted to what you’ve recently purchased/viewed (useful, but not the same type of service).

I think this is a useful service, which by the way is absolutely free to use. Try it out.

The Great Ruby Shootout (July 2010)

The Great Ruby Shootout measures the performance of several Ruby implementations by testing them against a series of synthetic benchmarks. Recently I ran Mac and Windows shootouts as well, which tested a handful of implementations. However this article reports on the results of extensive benchmark testing of eight different Ruby implementations on Linux.

The setup

For this shootout I included a subset of the Ruby Benchmark Suite. I opted to primarily exclude tests that were executed in fractions of a second in most VMs, focusing instead of more substantial benchmarks (several of which came from the Computer Language Benchmarks Game). The best times and least memory allocations out of five runs are reported here for each benchmark.

All tests were run on Ubuntu 10.4 LTS x86_64, on an Intel Core 2 Quad Q6600 2.40 GHz, 8 GB DDR2 RAM, with two 500 GB 7200 rpm disks.

8 implementations

The implementations tested were:

• Ruby 1.8.7 p299
• Ruby 1.9.1 p378
• Ruby 1.9.2 RC2
• IronRuby 1.0 (Mono 2.4.4)
• JRuby 1.5.1 (Java HotSpot(TM) 64-Bit Server VM 1.6.0_20)
• MagLev (rev 23832)
• Ruby Enterprise Edition 2010.02
• Rubinius 1.0.1

JRuby was run with the –fast and –server optimization flags.

Disclaimer

Synthetic benchmarks cannot predict how fast your programs will be when dealing with a particular implementation. They provide an (entertaining) educated guess, but you shouldn’t draw overly definitive conclusions from them. The values reported here should be assumed to be characteristic of server-side — and long running — processes; they should be taken with a grain of salt.

Time Results

Please find below the execution times for the selected tests. Timeouts indicate that the execution of a single iteration for a given test took more than 300 seconds and had to be interrupted. Bold, green values indicate the best performer out of each test.

Warning: The bm_primes.rb benchmark was originally written to aid the development of the Prime class. As such in 1.9.2 it was rewritten in C, which makes it a poor representation of implementation performance. This benchmark will removed in the future.

If you are not interested in the individual test results, the information presented in the table above is summarized directly below:

  Ruby 1.9.2         JRuby
Min.   : 0.013   Min.   : 0.382
1st Qu.: 3.258   1st Qu.: 3.051
Median : 4.543   Median : 4.997
Mean   : 9.262   Mean   : 9.180
3rd Qu.: 8.573   3rd Qu.: 8.969
Max.   :45.009   Max.   :48.850

    MagLev         Ruby 1.9.1
Min.   : 0.351   Min.   : 0.015
1st Qu.: 2.140   1st Qu.: 3.387
Median : 6.069   Median : 6.205
Mean   : 9.100   Mean   :10.860
3rd Qu.: 9.266   3rd Qu.:11.559
Max.   :51.221   Max.   :46.849

 Ruby 1.8.7         IronRuby
Min.   : 0.708   Min.   :  3.601
1st Qu.: 5.102   1st Qu.: 10.505
Median : 8.380   Median : 12.912
Mean   :18.785   Mean   : 26.539
3rd Qu.:24.793   3rd Qu.: 36.115
Max.   :75.653   Max.   :135.204

   Rubinius           REE
Min.   : 0.484   Min.   : 0.584
1st Qu.: 3.087   1st Qu.: 4.343
Median : 9.636   Median : 6.660
Mean   :13.232   Mean   :15.036
3rd Qu.:17.674   3rd Qu.:21.336
Max.   :73.050   Max.   :61.960

For the sake of convenience, I also produced a box plot from the successful data points:

There are a few considerations based on these results that I feel are worth mentioning:

• As you can see Ruby 1.9, JRuby and MagLev converge towards a similar performance level according to these tests.
• Ruby 1.9.2 manages to squeeze in a bit of extra speed when compared to Ruby 1.9.1 (which is a welcome improvement).
• Ruby 1.9 seems to be either much faster than Ruby 1.8 or roughly as fast, depending on the test. This appears to be in line with what I’ve seen in real world programs. There are programs that will only receive a 10-20% boost from 1.9, while others improve drastically. The results really depends on what a program spends its time doing.
• Performance wise, Rubinius is really starting to be a much more serious contender.
• Ruby Enterprise Edition is slightly faster than Ruby 1.8.7, to the extent where this is clearly visible in almost all of the tests.
• IronRuby running on Mono was the worse of the lot.

Memory Results

The following table shows the approximate memory consumption for each implementation when running each benchmark:

Summarized:

  Ruby 1.9.2        Ruby 1.9.1
Min.   :  4.320   Min.   :  4.580
1st Qu.:  4.378   1st Qu.:  4.695
Median :  6.285   Median :  6.920
Mean   : 20.795   Mean   : 15.669
3rd Qu.: 10.162   3rd Qu.: 11.383
Max.   :171.500   Max.   :100.570

   Ruby 1.8            REE
Min.   :  3.040   Min.   :  8.220
1st Qu.:  4.290   1st Qu.:  9.682
Median :  7.745   Median : 15.565
Mean   : 20.698   Mean   : 27.014
3rd Qu.: 11.273   3rd Qu.: 38.620
Max.   :103.520   Max.   :125.910

  Rubinius           MagLev
Min.   : 37.63   Min.   : 81.74
1st Qu.: 39.78   1st Qu.: 82.52
Median : 45.48   Median : 83.53
Mean   : 65.70   Mean   : 96.29
3rd Qu.: 58.22   3rd Qu.: 98.10
Max.   :215.33   Max.   :175.85   

    JRuby
Min.   : 49.04
1st Qu.: 71.23
Median :176.72
Mean   :169.41
3rd Qu.:209.04
Max.   :404.06

And finally, in graph form:

A few considerations on memory:

• Memory readings for IronRuby were not available, so they were not included.
• Ruby 1.9.2 uses the least amount of memory (as one might expect).
• JRuby was by far the most memory intensive of the group.
• Ruby Enterprise Edition used less memory than 1.8.7 in a few tests, but overall was more memory hungry than 1.8.7. This is really odd and entirely unexpected.

Linux Vs. Windows

This shootout and the Windows one were both performed on the same machine, thus we can compare how the same implementation perform under different operating systems. The only adjustment that’s required is the timeout. Every result longer than 60 seconds from this shootout needs to be considered a timeout, because the previous shootout did so as well.

It is commonly believed that Ruby performs much better on Linux than on Windows (with the exception of IronRuby). Let’s find out if these test results confirm that notion.

Ruby 1.8.7:

Ruby 1.9.2:

JRuby:

Finally, in chart form (yellow entries are on Windows as indicated by the labels containing W):

To use a beloved MythBusters expression, this myth is confirmed.

Note: As requested by a few commenters, here is a comparison of IronRuby as well (.NET 4.0 Vs. Mono 2.4.4):

Conclusion

In conclusion, let me just state that it’s nice to see several implementations getting faster. Ruby 1.9.2, JRuby, MagLev and Rubinius are all becoming serious competitors and working their respective ways closer to a similar performance level. If you think these benchmark shootouts are becoming boring, then the results are becoming more stable and predictable. I suspect that as time goes on, performance will not be the real distinguishing factor when choosing a Ruby implementation, other features will be.

In-Depth Book Review: Practical Clojure

Title: Practical Clojure
Authors: Luke VanderHart and Stuart Sierra
True pp.: 198
Publisher: Apress
Published on: June 2010
ISBN-13: 978-1430272311
Rating: 6.5/10

Published in June 2010, Practical Clojure by Luke VanderHart and Stuart Sierra is the latest Clojure book to hit stores. Despite the Clojure 1.0 jar shown at the beginning of the book, this title tries to cover the current version of the language, including references to concepts that will be introduced by the upcoming 1.2 version.

The target audience of this book is programmers who are absolutely new to Clojure. It didn’t strike me as being particularly aimed at developers who are coming from the Java camp, or the Lisp camp; in this regard, the book is rather “background agnostic”, even though Lisp programmers will feel much more at home than Java programmers will, due to the nature of the language itself.

The authors of the book are clearly well versed in this new language (Sierra is part of Clojure/core, the equivalent of the A-Team in Clojureland) and their confidence with the concepts presented is demonstrated throughout the book. Their explanations tend to be clear and to the point. Longer discussions are occasionally included when required to introduce concepts that are novel to most programmers, like the Software Transactional Memory (STM), refs, atoms and agents.

The book starts out by presenting a short but well-argued case for why Clojure is a worthwhile language, and then focuses almost exclusively on the core of the language. I’m afraid they do so to the detriment of the ecosystem surrounding Clojure. The authors don’t talk about how to install Clojure, recommend editors and IDEs (albeit a few are casually mentioned), or how to use build tools like Ant, Maven or Leiningen.

clojure.contrib, a fundamental extension library, is barely mentioned and there is no coverage of other important libraries or emerging frameworks. For instance, perhaps expectedly, Compojure (a web framework) and Incanter (a statistical and graphical environment) are only mentioned as examples of DSLs, however examples of their usage are not provided. (I believe the authors mistakenly refer to Compojure as Enclojure, which is a different project).

Despite the narrow focus, Practical Clojure doesn’t shy away from complex subjects and manages to include a chapter on Java interoperability, parallel programming, metaprogramming, and performance considerations. It does so briefly however, favoring a cursory presentation of the fundamental concepts rather than in-depth coverage, which would provide the reader with the degree of confidence required to approach real world problems.

The core language is covered in a manner that acts as both a tutorial and a reference. Major concepts, data structures, and common functions are presented to the reader with an endless supply of tiny examples. It’s easy to fly through them, but typing along in the REPL will be a far more valuable exercise for readers who hope to retain the information presented.

This leads us to another shortcoming of this book, which is the lack of more structured and complex examples. When I define their examples as “tiny”, with very few exceptions, I really mean it. For the first few chapters of the book the examples don’t get much larger than calculating the square root of a number through Newton’s method or adding contacts to an address book. Most of the other examples do a good job of illustrating the point they are trying to make with one, two, or just a handful of lines of code.

This is an actual sample of the kind of examples you’ll find throughout the book to illustrate many core API functions:

user=> (reduce + [1 2 3 4 5])
15

Note that this approach is didactically valid, because it isolates the function to show exactly how it works. After dozens of these functions though, you may expect larger examples to show how to integrate the use of some of these functions and data structures you’ve learned about. Such examples are seldom included. Furthermore, the book lacks any exercise for the reader. Foundational books that fail to offer many articulated examples and that lack exercises, tend to make it hard for the reader to retain the information and get some hands-on practice.

I have lots of respect for short books that get to the point and avoid wasting the reader’s time. K&R is notoriously acclaimed thanks to its clear and concise nature. However, Clojure is not C, and I feel that the 198 pages fall a little short when it comes to introducing this wonderful language to new readers. There is more to Clojure than simply surveying the language itself, even though I suspect that certain readers may appreciate this extremely narrow focus.

Overall the book is well-edited, despite the presence of minor issues. Aside from a few typos (e.g., “becauseall” on page 79), readers may find the formatting to be slightly inconsistent at times. For example in chapter 5 when presenting sequences, after the map function has been introduced, the font for the subsequent functions is substantially decreased for no apparent reason. Readers may be misled into thinking that the functions presented afterwards are somehow different from the previous ones, when in fact they’re all defined in clojure.core. In Listing 6-3, at page 103, the authors present their first “complex” example (the address book) and they do so by using, among others, doseq. This macro was not introduced before that page nor is it really explained within the example.

From a physical standpoint, this book is a rather thin and wide paperback. A small font, coupled with small margins and a wide layout, imply that the readability of the book suffers a little. The paper itself is off-white, fairly thick and slightly textured, not as pleasant to the touch as other books by Apress or most other technical publishers, even though I recognize that this is a matter of taste (some people may actually love it because of these characteristics).

With two introductory Clojure books on the market, drawing comparisons is unavoidable. Stuart Halloway’s Programming Clojure is a slightly older book (published in May 2009), which grants Practical Clojure a distinct advantage. This is not to say that Programming Clojure is obsolete, on the contrary it’s still a valid choice, but it doesn’t illustrate some of the new features that are available today. For example, in chapter 13 Practical Clojure introduces protocols and datatypes that will be available in Clojure 1.2 for the first time. Given that Halloway’s book was published more than a year ago, there was no possible way he could have included such powerful abstractions at the time.

Despite being older and less methodical than Practical Clojure, Programming Clojure tends to offer more complex examples. In the introduction of Programming Clojure you’ll see examples which Practical Clojure fails to include until much later in the book. Practical Clojure, the subject of this review, may leave you wanting for more practical examples of how all the language features fit together. Whereas Programming Clojure may leave you longing for more consistent explanations of how each part of the language works on its own.

Practical Clojure and Programming Clojure are competitors in the marketplace, but it wouldn’t be a bad idea to get ahold of both, because they complement each other quite well, in my opinion. Having to pick just one, I would probably recommend Practical Clojure, given its more consistent and up to date presentation. The sizzle offered by Programming Clojure, can be found to a much greater degree in upcoming and less introductory books, such as The Joy of Clojure. In this sense, reading Practical Clojure first followed by The Joy of Clojure, would be a solid learning path (Clojure in Action is another worthy addition, but it doesn’t replace The Joy of Clojure, which is a real gem).

In conclusion, Practical Clojure is not the Clojure equivalent of the highly praised Practical Common Lisp, from the same publisher. Reading it cover to cover and typing all the snippets included within, will not give you enough knowledge to start writing complex, idiomatic Clojure programs out of the gate.

However, if you are learning Clojure today, I do recommend this book. It’s a clear, well thought-out, concise introduction to the language that will give you a solid foundation as you go on to learn more about Clojure and Lisp in general.