Introducing Redis: a fast key-value database
One of the many advantages of having remarkable friends is learning quite early on about their most ambitious and interesting projects. Today, I’m going to talk about Redis, one such project that my friend Salvatore “antirez” Sanfilippo started.
Redis (REmote DIctionary Server) is a key-value database written in C. It can be used like memcached, in front of a traditional database, or on its own thanks to the fact that the in-memory datasets are not volatile but instead persisted on disk. As such it’s also very similar to memcachedb, though unlike the latter, Redis provides you with the ability to define keys that are more than mere strings (as well as being able to handle multiple databases). At this early stage (beta 6), lists, sets and even basic master-slave replication are supported, but more features are in the works (including compression).
Despite being a very young project, it already has client libraries for several languages: Python and PHP (by my friend Ludovico Magnocavallo), Erlang (by my friend Valentino Volonghi of Adroll.com), and Ruby by Ezra Zygmuntowicz. Except for Ezra, who should no doubt be a familiar name to most, Redis is pretty much an Italian product; and like other Italian products such as Lamborghini and Ferrari, this schema-less database is amazingly fast.
On an entry level Linux box, Redis has been benchmarked performing 110,000 SET operations, and 81,000 GETs, per second. As you can imagine, fast performance is one of the major goals of this project, and having chosen linked lists to have at the core of Redis’ implementation allows it to perform PUSH operations in O(1).
Salvatore has implemented a Twitter clone known as Retwis to showcase how you can use Redis and PHP to build applications without the need for a database like MySQL or any SQL query. He used PHP in order to reach a wide audience, but of course you can do the same with Python, Ruby or Erlang. The remarkable thing is how fast this clone is. According to Apache’s benchmark data, Salvatore’s commodity server (a Pentium D which is also running several large sites) could handle 150 pageviews per second (6 milliseconds each) for each of the 50 concurrent users. This was possible while using the grand total of 1 MB of RAM for the database. Of course, this is just a quick benchmark and there wasn’t a huge deal of data in the database either, but the responsiveness was very impressive nevertheless.
Salvatore will be publishing a beginner’s article based on the PHP Twitter clone he wrote, soon. It should appear on this wiki page where the code is already available, within the next couple of days. You can follow Salvatore and the evolutions of this project through his Twitter account. So check Redis out and (especially if you have experience with key-value databases) don’t forget to provide your feedback and/or contribute to the project.
The Rise of the Functional Paradigm
In yesterday’s address to the Ruby community, Dave Thomas invited Rubyists to fork Ruby, to freely research and experiment with new and interesting features. If this process is successful, many of these features will inevitably see their way back into Ruby’s core, thus improving the language in leaps and bounds. And I feel he couldn’t have been any more right. In fact, the whole industry is experiencing the trend of incorporating features developed in less common languages, research languages, “toy languages” if you prefer, within mainstream ones.
Experimenting with these alternative languages is important because occasionally they themselves become widely used, and even when they fail to do so, they lend their insight to the world of software development, finding their way into other languages. This approach greatly accelerates the development of common languages for the good of their large user bases and the improvement of the software industry. It’s a win-win situation for everyone involved and for the development community as a whole.
Pay attention to the development community online, and you’ll quickly notice a few non-mainstream programming languages appear over and over again. I’m referring to languages like F#, Erlang, Haskell, Scala and Clojure. I’ll admit to a certain selection bias, given that I tend to hang out in communities where hackers and developers actively pursue the betterment of their programming skills, beyond the stereotypical 9 to 5 requirements. But nevertheless, three or four years ago the average developer probably wouldn’t have heard about any of them (at least the ones that existed at the time). And today all of these languages have active communities, books being published about them, and most programmers have at least encountered some of these names.
They are all different languages, but their common denominator is the functional paradigm. Notice that I titled this post “The Rise of the Functional Paradigm” and not “The Rise of Functional Languages”. In a sense the latter is true as well, since there’s been much more attention towards functional programming languages lately. But there is a subtle difference. I don’t expect purely functional languages to become the most used programming languages anytime soon. For the foreseeable future, I don’t predict US companies to outsource Haskell jobs to India or China, like they do today for Java or .NET projects.
Yet these functional languages serve a higher purpose. Not only do they satisfy the needs of intellectually curious developers and companies looking for a competitive advantage, but they also have a great deal of influence on the rest of the development world.
We are seeing a convergence between these two groups of languages. Functional languages will strive to become as useful as possible, with libraries and tools that are more adequate for mainstream developers, while conserving their functional purity (I’m looking at you almighty Haskell). Meanwhile, mainstream languages will slowly adopt powerful features found in these functional and other research languages, adding further expressiveness and capabilities to their largely adopted foundations. F#, the evolution of C# and the addition of LINQ should be enough evidence that this is the case at least for the .NET platform. And even C++0x and D are leaning towards the incorporation of some functional features (e.g. lambda expressions and closures). The two types of languages come from different directions but will reach a similar destination.
The ever increasingly popular Ruby, Python and JavaScript owe their success to several factors. And while they are considered multi-paradigm and were mostly aided in their popularity by their immediacy, simplicity, usefulness and a set of historical circumstances, they’re all hybrid languages that adopt functional features. The functional paradigm is becoming so common, that it’s hard to imagine seeing any new programming language rise to fame without including at least a subset of the features available in other functional programming languages. As developers, we’ve grown to expect the elegance of functional features in a language. No lambda, no party.
If the 90s were characterized by the rise of the Object Oriented paradigm, and this decade can be considered as a transition phase, then the future belongs to the functional paradigm. Whether developers prefer to mix this with other paradigms (e.g. in languages like Ruby, Python, C#, etc…), like a powerful cocktail, or shoot it straight down (e.g. in purely functional languages like Haskell), the functional paradigm is here to stay.
Review of the screencast series The Ruby Object Model and Metaprogramming (episodes 1-3)
Even if you’re new to the Ruby community, you are unlikely not to have heard about the Pragmatic Programmers, who are well reputed for publishing great programming books. You may not have noticed that lately though, they’ve also been releasing several series of screencasts.
Each episode lasts about half an hour and sells for a mere $5. This strategy will work well in their favor because screencasts are a great way to teach programming. So far they’ve put out 5 series: Expression Engine, Core Animation, Erlang by Example, Everyday Active Record and, their latest entry, The Ruby Object Model and Metaprogramming.
If you are reading this blog, I’ll assume you’re mostly interested in the last two topics. I haven’t had a chance yet to watch the Active Record ones, so I can’t say much about them, other than that I have faith in their author, thanks to his magnificent work on the Railscasts.
Today after work however, I spent about an hour and a half watching the first three available episodes of The Ruby Object Model and Metaprogramming. This is a brief review of my thoughts on them. In short: they’re fantastic. But let me provide you with a bit more of a in-depth analysis than that.
These screencasts shine for the simple reason that you feel as though you’re sitting right there, next to a master programmer like Dave Thomas, as he explains Ruby concepts in detail to you. Whenever you’re not very clear about a concept, you can go back and listen to the explanation as many times as you’d like. You can pause, experiment with irb, and then go back to the screencast. After watching the first episode, I though that the $5 price was a steal.
Currently the following three episodes are available:
- Objects and Classes (29 mins)
- Sharing Behavior (39 mins)
- Dynamic Code (31 mins)
The first episode lays the groundwork for the series and introduces a useful diagram that enables you to easily visualize the Ruby object model correctly. Concepts like singleton methods and ghost classes are really well explained. The first episode also covers ‘self’ and the concept of current class, which is often confusing to beginners.
The second episode explores what constitutes good object oriented design, as well as the usage of prototype programming, inheritance (and how not to abuse it), and finally an excellent explanation of modules and mixins. If you are a beginner you may not know the difference between including and extending a module, or the true power of mixins. Fear not, the last part of second episode really covers those subject well.
The third episode starts by defining blocks and their usage, including the difference between lamdba and Proc.new. It then moves on to explore the concept of bindings capturing the execution context, and the importance of closures in metaprogramming. The last 12 minutes or so are spent dealing with the subject of dynamically defining methods, which is at the heart of metaprogramming, and the video includes a nice example of how to create your own version of attr_accessor by defining, in a module, a method that dynamically defines getter and setter methods.
I’d say that aside from the feeling that I was enjoying a “private lesson”, what I liked the most was the coding style recommendations about when to employ a given concept and when to opt for something else. The examples were clear and easy to follow as well.
I can’t recommend this series highly enough to any Ruby programmer with a basic grasp of the language. I feel that intermediate developers will get the most out of these videos, but they’ll also definitely be very beneficial for beginners. Real experts shouldn’t find any alien concepts, at least in the first three episodes; I know I didn’t. Yet, they’ll appreciate, just like I did, the fresh slant when it comes to reviewing some of the illustrated concepts and examples. There will also be a few “ah-ah!” moments here and there, I guarantee you.
Five stars, and I’m certainly looking forward to future episodes.
Welcome to IBM, Damien Katz
It looks like the new year has started off on the right foot. I just learned that Damien Katz, the mastermind behind CouchDB, is now an IBM employee. Not only that, but he’s being hired specifically to work on CouchDB full time. This is great news, in his own words:
All the code will be Apache licensed and donated to the Apache Software Foundation, with the plan CouchDB will eventually become an official Apache project. A big plus here is the Apache license allows anyone to do pretty much anything with the code, so everything remains truly open source. I wouldn’t have done this without IBM’s commitment to keeping CouchDB open.
A warm welcome to Damien, who is truly a great addition to the IBM family. I’m planning on getting in touch with him through SameTime.
Come to think of it, the fact that CouchDB is a document-oriented database written in Erlang reminds me that with the beginning of the new year, now might just be the right time for me to get into Erlang. Multicores are here, and all that…
