The pursuit of excellence in programming

As I write a series of thoughts on the pursuit of excellence in programming, I must preface my essay by asking you to ignore that I wrote these words. I invite you to evaluate the opinions and ideas presented here not ad hominem, but rather on the basis of their own merits. It would be easy to otherwise mistakenly dismiss them with the infamous question posed by Steve Jobs to a blogger: “What have you done that’s so great?”.

This is to say that I talk about the ambitious and noble goal of achieving excellence in programming, fully conscious of not having achieved said excellence. For the time being, I don’t feel like I can point my finger at something that would impress Steve Jobs (or less critical observers). I’m just a traveler on the journey of learning, with a desire to share his experiences and plans.

Two visions of intelligence

Mastering a complex discipline such as programming requires a great amount of learning over the course of several years, perhaps even decades. Maximizing one’s ability to learn is therefore an early investment that can quickly repay itself.

The biggest impact on my ability to learn was caused by a shift in the way I considered the matter of intelligence. There are mainly two ways to think about it. You can either consider intelligence to be a static, intrinsic ability or a more dynamic, cultivable characteristic of human beings.

Cognitive scientists and psychologists conclusively determined that people who perceived intelligence as a dynamic characteristic, outperformed and were more successful than people who internalized intelligence as an intrinsic, static ability.

It’s worth noting that it’s not really important whether intelligence can actually be developed through application. It’s the perception of it that forges students’ approach to learning.

This difference in perception is often conditioned by early parenting. Kids who are encouraged to work hard to achieve results and are praised on the basis of their effort, tend to develop a perception of intelligence and results as something they can work on. Other kids are conditioned to think that they are doing well because they are “smart” and that their intelligence alone will most likely lead them to success.

Society has a fascination with genius, and parents like to fancy their little ones to be several standard deviations better than the norm, but conditioning children this way has dangerous and counterproductive consequences.

Kids who are labeled and praised because of their “innate capabilities”, will often suffer from an overconfidence that will affect their ability to challenge themselves through the depths of the unknown, because they feel it would threaten their status. What if they fail? It would mean, in their eyes, that perhaps they are not the smart person they have been assumed to be all along. We all have seen such kids failing here and there, and quickly making excuses such as, “Oh, I wasn’t trying at all”.

A parent who is cultivating a kid’s interest in hard work, may be more likely to encourage their child with words such as, “It’s OK. Keep studying, and you’ll definitely do better next time”. A parent proposing a model of static intelligence, may justify their child’s failure in a given subject by concluding that “maybe you are not cut out for subject X”. [1]

When facing failure, the “static intelligence” child may crumble under the weight of his own demise, as if failure was a reflection of their intrinsic value rather than a temporary speed bump and occasion for growth. A “dynamic intelligence” child will simply try harder next time. Genius or not, excellence and mastery of any subject requires hard work and many “smart” kids fall short when the bar is raised high enough so that “smartness” alone won’t cut it anymore. This usually corresponds with the switch from high school to college.

I’m very familiar with all of this, because I was one of those kids. I was labeled by my parents and science teachers as a “genius”. Even psychologists at school, who came to help us figure out what careers we were better suited for, ended up telling me that I could pursue virtually any career (at the time I was interested in nuclear physics) and that according to their (virtually meaningless) IQ tests I would be classified as a “genius”.

Please note that the problem wasn’t so much the label. Most smart kids figure out that they are smart on their own rather quickly. The real problem was that I wasn’t taught the value of long-term intellectual effort. Effort itself was considered as being somewhat detrimental to my status. Not only was I supposed to succeed, but I supposedly had to do so without putting forth any effort (a “utopic” ambition).

One of the first example of this that I can remember is when my father “caught me studying” for a few hours the same book before a test in middle school. He told me something along the lines of “Why do you need to study? A genius like you should figure out the test without studying.”. It’s absurd, I know, and probably one of the dumbest things my — otherwise bright — father has ever told me. As a young kid though, such a statement can have a strong impact on you.

Another example has to do with Latin. My teacher was a palm reading, crazy cat lady and I had no respect for her from the start. So I didn’t pay attention in class, on top of not studying Latin at home, I set myself up for failure. When the first translation assignment came around, I got a mildly negative score. My less than professional teacher told me, “Oh I thought you were good, but I guess you are not”. Afterwards my father added to this by saying something along the lines of, “Well, don’t worry, I guess languages are not your strength.”.

Boom. That was enough for me to stop having any interest in Latin and completely ignore a subject at which I wasn’t excelling. Nobody could tell me, “You are stupid because you don’t understand Latin,” if I didn’t try at all. So I was the high school kid who did advanced Calculus stuff on his own for fun, when my classmates where struggling with Algebra, yet I pretty much sucked at Latin. (In retrospective, the thought patterns required to excel at Latin where not very different from those required to excel at math or mastering English as a second language. I suspect that had I put in some effort I could have been very good at, and actually enjoyed, the subject.)

Over the years I had to readjust this perception entirely. By falling on my face more than once, I learned that excellence is only achieved through a combination of talent and effort. The real genius may lie in the ability to put in thousands of hours of focused study and practice, in the pursuit of whatever one person is trying to learn and understand.

If people knew how hard I worked to get my mastery, it wouldn’t seem so wonderful at all. — Michelangelo

By this new definition, I was a complete idiot who had to entirely learn from scratch how to appreciate the value of effort to hone and develop his “talent” (which is nothing but a seed on its own).

Learning to learn

Being very interested in programming, computer science, mathematics, and science in general, I decided at some point that I had to entirely change my attitude towards learning if I were to master any of those disciplines. Effort was now more important than intelligence on its own, and I would feel satisfied only when doing a really good job in the pursuit of something challenging, that couldn’t be achieved by sheer “talent”.

The saddest thing in life is wasted talent, and the choices that you make will shape your life forever. — Calogero ‘C’ Anello from A Bronx Tale

In the process, I started to internalize a few principles and I dealt with issues related to both the art of learning and the art of programming.

An awful feeling

There is a cognitive bias known as the Dunning-Kruger effect [2], in which subjects who are inexperienced or less competent within a given discipline, tend to overestimate their abilities (they are in other words affected by illusory superiority [3]).

The other side of this coin is that the more you study, the more you realize how little you know and how much there is to know (a concept put forward by Socrates back in the days of Ancient Greece). This is both a pleasure and a discomfort. There is a huge amount of satisfaction in finding things out. Yet, being in doubt and fully aware of how little you know tends to be an unpleasant side effect all learners have to live with. Doubt truly is the water that’s fundamental for the growth of the flower of intellectual curiosity. [4]

My approach in this case is to embrace and dominate my ignorance and fears. Whenever there is a concept that I feel particularly ignorant about or that is way over my head, I try to tackle it as if my life depended on it. There are still countless things I’m ignorant about, but this approach as really paid off for me over the years.

If you are trying to learn a whole branch of computer science or mathematics, it will take a long time, so you may want to start first with smaller “fears” that can be mastered, at least at an introductory level, in a short amount of time. Rather than thinking, “Oh yeah, I should really learn Git”, for months, act on the thought. The essential knowledge to work with Git or Hg doesn’t take months to learn (assuming you have a need for either of these particular tools).

But you are in this for the long run, so don’t be afraid of improving your craft by studying advanced topics that require a bigger commitment in terms of time as well. There is no royal shortcut, mastery of our craft will require thousands of hours of dedicated study and practice.

Theory and practice

Masters of any intellectual discipline tend to have good working knowledge of both theoretical and practical aspects. Pursuing excellence in programming requires the study of many insightful books that will widen your view of the field and as a result necessarily improve your craft.

Working with books alone is not enough though. Programming requires writing and reading programs every day for years. That’s why I have a rule: I refuse to go to sleep if I haven’t read and written some code on a given day (this doesn’t include the code I write for work, of course). So far, this rule has had a positive impact on my ability to code.

It is a mistake to think that the practice of my art has become easy to me. I assure you, dear friend, no one has given so much care to the study of composition as I. There is scarcely a famous master in music whose works I have not frequently and diligently studied. — Mozart

Breadth Vs. Depth

As we progress in our journey towards the pursuit of excellence in programming, a question that will no doubt pester people’s minds is whether one should go for breadth of knowledge, or depth. There are countless programming languages, paradigms, methodologies, technologies, et cetera.

The truth of the matter is that if we are to become GrandMaster programmers, we cannot ignore either of them. In practice, depth has a much stronger impact in the way we construct software. It is through deep understanding that we can see the whole through the parts. There is therefore value in specializing in just a few languages and technologies, and really mastering them in-depth.

Getting things done in software development requires a certain pragmatism and proficiency with the tools at hand. There is no escaping it. Depth is therefore necessary, but not sufficient.

I find that the web is particularly good at covering the breadth aspect of things. There are always new and interesting areas I can learn about and experiment with. I don’t need a whole 600 page book to get a feeling for — or to better understand — certain technologies that are not crucial to my area of specialization.

While there are quite a few exceptions I could mention, I’d say that I tend to use the Internet as an aid for horizontal scaling of my knowledge, and books for vertical scaling. And again, more often than not, the depth level is mostly determined by the amount of time, practice and effort I put into it, rather than the media I’m using.

Where to find time

One of the objections I hear often, particularly when it comes to reading books is, “I don’t have time!”. In a few extreme cases, that may actually be true, but I think that most people severely underestimate how much time is “wasted” on a daily basis (even just surfing online).

Breaks are extremely important, and I’m not advocating any regime of incessant study. I simply know how crucial it is to be constant. It’s a marathon, not a sprint.

My second rule is: I refuse to got sleep if I haven’t read at least a chapter of a book that day. Very often I get caught up in the book I’m reading or working through, and end up getting through much more than just one chapter (it really depends on the book, of course). But for me the rule is clear: no sleep allowed until one chapter has been read every day.

Try this approach and you’ll see that reading doesn’t have to take up much time, yet doing so you can still read several books (including technical ones) every month.

We are what we repeatedly do. Excellence, then, is not an act but a habit. — Aristotle

For certain books, it is convenient to have the book in PDF format on your computer, as you switch from the book to the editor/console and back. However, generally speaking the computer tends to be quite distracting and thoughts like “I’ll just check my email quickly” can easily lead to hours spent doing something else.

For this reason I prefer to read from a paper book which is also easier on the eyes for extensive reading after a day in front of a computer screen. Even if I’m using the computer at the same time to input code, the physical presence of a book next to my laptop is enough to remind me that I shouldn’t get distracted online. (For me the depth and focus required by books is also an antidote to the re-wiring that the web tends to do to our brains. [5])

By the way, a few days ago Amazon announced a gorgeous, brand new graphite color Kindle DX [6]. I think I may pull the trigger and get it for my upcoming 30th birthday. Buying numerous paper books is expensive, and given the price of Kindle books, this move would end up being cheaper in the long run. The large e-ink display almost looks like paper and the device is not as distracting as an iPad (you read on a Kindle, and that’s it). Plus, it’s lighter than most technical books and surely takes up less space in your home.

Achieving focus

With so much going on within the programming world, distractions are easy to come by. My approach is to focus only on the given macro-task at hand. If I’m trying to learn about process calculi for example, then for the next few months my “learning time” will be ruthlessly dedicated to that subject, as if the rest of the programming ecosystem stopped in time.

Then there is focusing at a micro-task level. Learning about a given subject can always be divided into a long series of smaller steps. When I’m focusing on one such, tiny step, then everything else ceases to exist (or at least in theory).

One trick I use to achieve solemn focus on micro-tasks, whether reading code, writing code, or reading a technical book, is the use of the Pomodoro Technique [7]. In short, I use timer software [8] which alerts me when 25 minutes (a pomodoro) have passed, and gives me a 5 minute break for each pomodoro. Every 4 pomodoros, I can take a longer break.

When I first started using this technique I thought it was mostly a gimmick and I had a hard time taking breaks. I just wanted to keep going and the “tracking” seemed silly. However, I must say that it strikes an elegant balance between the desire to focus for extensive periods of time and the importance of taking regular mini-breaks.

This approach has become routine for my mind now, so even if it is just a gimmick, it’s still a good way for me to get focused and “in the zone”.

Aiming for sprezzatura

The pursuit of excellence requires a huge drive from within, and a fundamental dissatisfaction with just being good at a given discipline. I believe this is true regardless of the profession at hand.

As I progress in my journey, I’m discovering how the key is to make the pursuit of excellence a habit. This goes against my nature of being an intellect sprinter, but I’m in for the long run and I’m really learning to enjoy the, marathon like, process.

My long-term goal is to program with sprezzatura [9], where the process is so internalized and part of my subconscious that it almost looks effortless (as if the act of programming was committed to muscle memory). It will be an overnight success, 15 years in the making.

Regardless of the improvement level achieved, I will always have the joy, privilege and need to continue to learn for the betterment of myself and my craft. When there is no set destination, the journey is what really matters.

Ancora imparo. (I’m still learning.) — Michelangelo

Notes

[1] These concepts are explained, in a much more eloquent manner, in the early chapters of the excellent book, The Art of Learning by Josh Waitzkin.

[2] Dunning-Kruger Effect on Wikipedia.

[3] Illusory Superiority on Wikipedia.

[4] For more on the importance of doubt in science, check out the beautiful epilogue in What do you care what other people think? by Richard P. Feynman.

[5] For more on this phenomenon, read The Shallows by Nicholas Carr.

[6] The new Kindle DX on Amazon.

[7] The Pomodoro Technique.

[8] Pomodoro for Mac OS X.

[9] Sprezzatura on Wikipedia.

Translations

Carlos Marcelo Cabrera translated this article into Spanish: La búsqueda de la excelencia en la programación.

A new domain name for Zen and the Art of Programming

For years Zen and the Art of Programming has been hosted on my domain antoniocangiano.com. However, I’ve decided to switch it over to a new one, programmingzen.com. This move was done for two main reasons: first, it is sometimes hard for native English speakers to communicate and remember my name. Second, my blog isn’t so much about me and my life, as it is about the world of programming.

As I look towards the steps that are required to bring my blog to the next level, I decided it was time to make the switch to a new domain. I’d like to transform Zen and the Art of Programming into a blog that every programmer could potentially be interested in subscribing to, not just people who are into Ruby or following me.

The good news for you is that you don’t need to do a thing. Every internal link has been permanently redirected to a corresponding programmingzen.com link, with the exception of the homepage. I’m now using antoniocangiano.com as a sort of minimalist virtual business card, from which I link to my email, Twitter, Linkedin, my three blogs (this, my personal blog, and my math blog), as well as my book and ThinkCode.TV.

Stay tuned for lots of interesting new content, which I plan to post more frequently. Meanwhile please feel free to let me know what you think about this new change; I’d really welcome your thoughts.

Follow up to my Gmail third-party access post

My previous post about a possible intrusion by non-authorized parties on my Gmail account has received a lot of attention on Hacker News, and was even linked to from LifeHacker. There were a lot of questions, concerns, and critics that quickly surfaced, and in this post I’ll address most of them. Given the volume of heterogeneous points at hand, I will present this post in an informal FAQ manner.

Image © Sooperkuh.

Did the IP 173.203.211.51 belong to a malicious user?

No. It belongs to etacts (a social plugin for Gmail), who has confirmed that the IP is one of theirs. They were authorized by me to access my inbox via IMAP for “analytics” purposes.

Why did you originally think that it could have been a malicious user?

A reverse-lookup of the IP address did not show an etacts hostname, as it had in the past, but instead a generic Slicehost server. A search quickly revealed that other people had complained online about the same IP accessing their inbox, too. This was a red flag, but contrary to some peoples’ interpretation, I didn’t freak out about it.

Oh come on, you freaked out, admit it.

Not quite, but I was reasonably concerned.

Were you a bit paranoid about it?

No.

But they are after you.

Really? Who?

Everyone. Everyone is after you.

Oh.

What about Zoho Discussions and Trendly?

Zoho and Trendly are the only two services I granted access to my Google account. In practice, this means I simply logged in using Google as a single sign-on, instead of creating a new account for each of those services. This type of OAuth-like authentication does not provide third party companies with your password, as explained by Google in this help page.

Contrary to my initial post, Zoho and Trendly could not have been the culprits. etacts on the other hand required a username and password in order to work. This is likely to change when Google introduces OAuth for IMAP.

Shouldn’t you — or anyone — know better than give your password to third parties?

Generally speaking, yes. You shouldn’t share your password with anybody. In this case, I assessed the risks and benefits of doing so, and opted for the convenience of the service, feeling that etacts was/is trustworthy.

etacts is a YCombinator-funded startup and its entire business model depends, for the time being, on people trusting them with their login credentials. Until Google launches OAuth for IMAP, there isn’t really a way around this scenario, if you want to use such a service.

I fully understand that some people may feel that it’s never wise to trust a third party regardless of the benefits involved into the equation.

Shouldn’t etacts attempt to prevent such false alarms?

Yes, and the team at etacts was very proactive in terms of proposing possible solutions to prevent those users who trust them from needlessly worrying when an usual IP is recorded. Publishing a list of their IPs and ensuring that reverse lookups of such IPs lead to actual etacts hostnames, were two of the potential solutions that they suggested.

So was it much ado about nothing?

Yes and no. Without a doubt, the event that led me to write the article was the false alarm, but the real motivation behind it was to bring Google’s access log to people’s attention, regardless of whether they shared their password or not. In fact, following my post there were reports of people who’d discovered there had been illicit accesses to their accounts.

I also published the article to share some suggestions on what to do when dealing with an intrusion.

Why did you say that you changed the password on a wired computer?

As I mention in the post, it was unlikely that someone managed to sniff my password by whatever means. Occam’s Razor did point towards the service I’d given my password to, which I made clear in my initial post.

I happened to have a newly formatted Linux desktop, so I used that one. It didn’t take any extra effort on my part. In my case, using that or using my laptop would have been no different (in fact, I used my WiFi throughout the whole ordeal, and I haven’t bothered to change my WiFi password either).

Since those were general suggestions in a checklist of sort, it didn’t hurt to include it for people who may have had their password sniffed (even though such scenario would require further action).

You don’t need DBAN to remove possible keyloggers

That’s true, you don’t. I’d been planning to do a clean install on my MacBook Pro for a long time, so I thought I’d take the opportunity and go ahead with it now. As well, formatting one’s hard drive to remove keyloggers is still a valid suggestion for those who did have their password keylogged.

You don’t need DBAN for a new installation either

These days it’s rather rare that I do a brand new installation, so when I do, I like to start with a clean slate. Booting with DBAN and running a quick erase to zero in every bit is not an overly lengthy process. It’s an unnecessary step, that is unless you have unicorn riding midget pr0n to hide.

Do you have unicorn pr0n to hide?

Only a few gigabytes.

Is there anything else you’d like to add?

One thing I forgot to mention in my previous post is that, after changing your password, you may want to remove the app authorization from your list of authorized third parties for your Google Account.

BTW, what’s up with the server dying on us?

When I setup my server (a long time ago), NGINX wasn’t as established as it is today. So I went with Apache which is a slow and heavy beast. If you add that WordPress is resource hungry as well, you get a fine tuned 1GB of RAM slice that struggled with several thousand visitors in the span of a few hours. An strace of the Apache processes revealed that despite APC and the WordPress SuperCache plugin, PHP was still hogging resources like doing so is going out of style. (Despite my server issues more than 20,000 visitors managed to read the article from my old server.)

I have now moved this blog to a 4GB, 2 CPUs server in the cloud, using NGINX as a web server. I will slowly port my other blogs as well as I migrate away from Slicehost. This setup should be able to handle much heavier loads (at least in theory). If you notice any problems, please don’t hesitate to let me know. (I officially declare my current deployment as an alpha “release”).

That should wrap things up in regards to this Gmail investigation, folks. Time to move on.

Who is accessing your Gmail account?

The Gmail team recently introduced a new feature (in the footer) that enables account holders to verify the latest login activities on their account. I routinely check mine and the results are usually boring, reminding me I check my email way too often (and I do so mostly via browser, through my Canadian IP).

An unwelcome surprise

If you don’t check yours regularly, you should (my version of Google Apps doesn’t have this feature though). In fact, tonight during a routine check, I discovered an unwelcome surprise: an entry that didn’t belong. The following screenshot shows the recent activity on my account (with some information blanked out):

See that US, IMAP line? That wasn’t me. So did someone manage to access my account? Or was it a web application that I authorized? Before panicking, I decided to look into whatever information I could gather about that IP.

It turns out that it’s the IP of a server hosted by Slicehost (RackSpace), but I couldn’t find any website running on that IP address (173.203.211.51). To make things more interesting, I found two people (one German, one Japanese) complaining online about the same IP address and IMAP access to their Gmail accounts.

Was my account hacked into? I have a hard time believing that someone actually managed to login by guessing my password which was as secure as a password can be. I haven’t used my laptop on an unsecured WiFi. I use a Mac and am very cautious about what I install, so I doubt I have a keylogger installed or anything of that nature. Using 1Password I’m even immune to the so-called “tab napping” attacks.

Possible culprits

Assuming that this is not a misunderstanding and some SaaS application I authorized is not in fact using that server to perform a legitimate action, I think it’s likely that someone managed to get in through a vulnerability or backdoor in one such application.

I’m not pointing fingers here, nor accusing anyone, but it is interesting to find such an occurence happening so shortly after granting the aforementioned authorizations. The websites I granted access to were:

• Zoho Discussions (24 hours before the suspected intrusion happened)
• Trendly (3 days before the intrusion)
• Etacts (a few weeks before the intrusion)

It’s worth mentioning that in the past Etacts had scared the crap out of me with their American IP showing up in the recent activity list. However a lookup has always shown the questionable IP to belong to them.

Do any of these services intentionally use the server with IP 173.203.211.51? Since I’m not the only one who suspects a violation from this IP, it would be interesting to hear what Slicehost has to say about it? Perhaps they know if it’s a legitimate or illegitimate use of their server.

How to deal with an email intrusion

The perception of being intruded upon, whether it’s real or just a scare, is definitely not pleasant. Just in case the same happens to you, here is what I did to deal with the situation:

• I verified that there were no messages sent on my behalf.
• I checked that there weren’t any new filters that would forward emails to a possible malicious user.
• I verified that there weren’t any forwards and ensured that forwarding was disabled.
• POP3 was already disabled, and I have now disabled IMAP as well.
• I revoked access to my Google account for all listed web applications.
• I changed my password to another humongous one on a different computer, with a brand new installation of Linux, directly wired to my DSL modem (bypassing the whole wireless infrastructure I set up at home).
• I will, soon enough, format my Mac (I’ve been planning a DBAN wipe, plus a brand new installation for a while either way).
• I will continue to monitor my account activity.

This is the kind of information I felt necessary to share even if this turns out to be a false alarm. I highly suggest that you keep an eye on your Gmail account activity and if you find something suspicious, act accordingly.

UPDATE (June 17, 2010): Please read my follow up post.

The world according to Android

Apple is receiving copious amounts of bad press due to their somewhat arbitrary and restrictive App Store policies. This isn’t going to change a thing at Apple, but the complaints are warranted. Android on the other hand is touted as being “the open platform”, and developers are invited to develop and sell their apps on the Android Market, rather than investing in a proprietary platform like the iPhone OS and the App Store.

So much for being open

When discussing the main emerging alternative platform, Android, we often hear about fragmentation issues, but critics tend to ignore the elephant in the room. Very few people can buy and sell apps on the Android Market. In fact, only developers from nine countries are allowed to sell applications on the Android Market. Take a look at the following map I prepared:

Fine folks from Canada, Australia, New Zealand, Denmark, Sweden, Ireland, Russia, China, India, etc, are all excluded from this platform. Sure, they can sell apps on their own sites, but the real advantage of the App Store and the Android Market is in their ability to show and sell apps to a wide audience. If you lose that, you are pretty much out of the game or you’ll be starting off with a humongous disadvantage.

Is this the best Google can do?

Before you jump in and cut Google some slack, remember that the Android Market has been around for almost two years. As much as I want to cheer for the underdog, one has to ask if Google really thinks that this half-assed attempt will challenge the absolute supremacy of the App Store in the marketplace. How much innovation, money, and good will are they leaving on the table? Keep in mind that there is no stevejobs@google.com. Google is notoriously bad when it comes to supporting and listening to their customers, because trust me, I’m not the only one complaining about this.

I want to believe in the future of Android and the prospect of developing for it, but like many other developers I don’t even have the option. If I wanted to develop in the emerging mobile market and perhaps make some money out of it, it’s the iPhone OS way or the highway.

Upgrading to Rails 3

Rails 3 is a major upgrade; using it almost feels like working with an entirely new framework. Porting existing applications and acquiring the skills required to build new ones entails a significant amount of effort. You could scout the net for bits and pieces of information, but that would be time consuming and possibly frustrating. Thankfully there are resources available that have done the work for you, so you don’t have to waste time trying to figure out what’s new.

In this post, I’d like to point out a couple of resources that I think compliment each other well, and focus on how to upgrade applications, as opposed to simply providing you with a shopping list of features.

The first one is Upgrading applications to Rails 3, a screencast that was just released by ThinkCode.TV. This screencast is almost an hour-long and shows you how to port a real world web application from Rails 2 to Rails 3. As such, it can be very useful if you have existing code that you’d like to port over to Rails 3. The author ported a few large applications to Rails 3, as he has solid experience with it. I’m biased of course, but I feel it’s well worth $8.99. (Today only, use the coupon RAILS3 to purchase this Rails screencast for just $5.99.)

The second resource is the Rails 3 Upgrade Handbook by Jeremy McAnally. It’s a beautiful PDF that succinctly explains what’s new in Rails 3, as well as how to upgrade your applications to the new edition of the framework. At 10c per page ($12 for 120 pages), it too is worth the money in my opinion.

Regardless of whether you end up buying these resources or not, I sure hope you have extensive test coverage for your existing Rails 2 applications. In my experience this is a must, because porting complex applications to Rails 3 without solid test support is a definite challenge. Nevertheless, I feel that this major upgrade is truly worth it. Rails 3 really brings Rails to a whole new level and we, as a community, should be proud and excited about what lies ahead of us.

Web or iPhone OS applications to make money?

If you are a developer who’s interested in starting a new business or even just earning extra cash on the side, you have a few options that can lead to scalable profits and passive income. The most popular choices these days are SaaS (Software as a Service) and iPhone/iPad development.

Choosing web application or iPhone OS development is a matter of personal taste, skills, goals, technical requirements, and so on. If we are looking at things solely from an economical perspective though, it’s interesting to consider for a moment which of the two is most likely to be profitable all things being equal.

In my opinion, despite the horror stories, it is far easier to make money with iPhone/iPad development than with charging for a service provided by a web application. The reason for this is very simple. Users don’t expect to pay for websites. They occasionally do for the useful ones, but it’s a relatively new approach and people are still adjusting to it.

Due to the hosted nature of SaaS offerings, they often require a monthly or yearly subscription as well, which is harder to sell than a single one time purchase. $9.99 here, $19.99 there, all adds up very quickly to an uncomfortable monthly bill.

iPhone and iPad applications on the other hand have a major advantage. Apple created an incredible ecosystem where millions of users are ready to pay for tiny applications that solve one small problem or are amusing enough to install. Low prices without the need for a subscription encourage impulse buyers to purchase applications without giving it a second thought.

You’ve probably been on the web for many years now, yet how many web applications do you pay for? However if you have an iPod Touch, an iPhone, or an iPad, how many applications have you bought? I own an iPod Touch and rarely use it, yet even I’ve paid for some applications.

It’s a matter of expectations and Apple did a marvelous job in that respect. When you buy an iPhone, you are almost expected to get cool apps for it. That, along with web access, is the whole point of having an advanced smart phone device. There are no such expectations for web applications.

Take a look at this list of applications and their sales. Most of them could be easily ported to the web, yet their limited scope and functionality wouldn’t even justify calling the resulting sites web applications. Few people would pay for them, even if one were to price them at $0.99. (And I’m not talking about apps for which being on a phone is the key feature). Conversely, porting a free Flash game from the web to the iPhone, for example, would almost certainly lead to sales.

Android is an option too, but I feel that Google has done a much poorer job at setting expectations. Furthermore their marketplace is a lackluster version of the App Store, users from several countries are unable to purchase applications or have to jump through hoops to do so, and developers from countries such as Canada, cannot publish and sell their apps on the Android Market.

The question is, will Google do a better job with the Chrome Web Store that was announced yesterday at Google I/O? If so, it may become a good extra channel through which to sell web applications; an App Store for the web if you will. The distinction between web apps and offline mobile apps may even fade away at some point. That would be a great thing for developers. For the time being however, it is my opinion that developing for the iPhone OS is a safer bet in terms of achieving profitability.

Benchmarking MacRuby 0.6

Recently MacRuby 0.6 was released. The development team put a lot of emphasis on improving compatibility with Ruby 1.9, and the viability of MacRuby as a tool for developing Mac OS X applications. Focus on these aspects took precedence over performance, but I was still curious to see how well it performed when compared to Ruby 1.8.7 and Ruby 1.9, respectively.

This article showcases the results of a small Ruby shootout for Mac. I plan to publish a Windows one by next week, and then a week or two after that, a complete Linux shootout that will have many more implementations. Grab my feed or join the newsletter to avoid missing upcoming shootout posts.

The setup

The tests are a large subset of the Ruby Benchmark Suite. Each test was run 10 times, five to detect the best execution time, and five to detect the minimal memory consumption. All of the tests were run on Mac OS X 10.6.3, on my MacBook Pro 2.66 GHz Intel Core 2 Duo, 4 GB 1067 MHz DDR3 RAM, 320 GB 7200 rpm disk.

Stable implementations tested:

• MacRuby 0.6
• Ruby 1.8.7
• Ruby 1.9.1

Disclaimer

Synthetic benchmarks cannot predict how fast your programs will be with one implementation or another. They provide an (entertaining) educated guess, but you shouldn’t draw overly definitive conclusions from them. Furthermore, the Ruby Benchmark Suite has many tests that don’t provide much insight when it comes to comparing implementations. They are there for legacy reasons and will probably be removed in the future. For the time being, take them with a grain of salt.

The results

Without further hesitation, here are the execution times for the tests (divided in A-L, M-Z). Timeouts indicate that the execution of a single run took more than 60 seconds and had to be interrupted. Bold values indicate the best performance for each test.

And here is the estimated memory usage:

Conclusions

MacRuby 0.6 is faster than Ruby 1.9.1 at times, but it can also be significantly slower. Overall, as things stand now, its performance appears to be between that of Ruby 1.9.1 and Ruby 1.8.7, with several outliers and a greater variance compared to those two implementations. Memory wise, MacRuby appears to be significantly more “memory hungry” than the other implementations (even though this wasn’t all that much of a surprise to me).

I’m interested in seeing how future releases that will be focused more on performance will affect these preliminary results. For the time being however, don’t let this outcome discourage you from using MacRuby 0.6, which is the first release that’s considered stable for Mac OS X development.

Download: CSV Files

PS: If you are looking for a fun and easy way to get started with MacRuby, check out ThinkCode.TV’s screencast on the subject.

Update (July 3, 2010): The following box plot compares the various implementations for the tests for which all the implementations were successful. Only times for the largest successful input number were used in those tests where multiple input numbers were tested.

The most important programming language today

“What programming language should I study next? What framework?” I occasionally receive emails from younger — and not so young — readers alike asking me for guidance about such matters. “Use the right tool for the job” is the correct answer, but it’s cheap advice when there are a plethora of tools seemingly right for the job. For most people these days the job at hand is of course web application development.

Should they study Ruby and Ruby on Rails? Or Python and Django? How about C# 4.0 and ASP.NET MVC? Maybe CakePHP? Java and Stripes? And how about more exotic choices like Clojure and Compojure or Scala and Lift?

With very few exceptions, in 2010, it’s hard to choose a combination of semi-popular technologies that couldn’t do the job. Does it really make a huge difference if you choose to study Ruby on Rails or Django? In all honesty, despite all the existing differences, it doesn’t really matter. As long as you become proficient with one of these tools, you will be adequately equipped to approach most web development tasks. Your experience as a server-side developer will be the bottleneck, not your framework of choice.

The real reason why I get asked these questions though, is that these people are mostly looking for a silver bullet, a language-framework combo that will magically allow them to create fantastic web applications in a matter of weeks. They are often after a shortcut, but there is no royal path to web programming.

When I think about the future of programming languages, I envision Babel not people talking Esperanto. We are destined to live in a technological world were there will be many valid server-side options, which are similar yet different enough to justify their own existence and that of their respective communities.

There won’t be a programming language to rule them all, but I believe one language will continue to be the lingua franca of the web. In that sense, it’s the most important programming language today and I think its relevance will only continue to grow in the future. I’m talking of course about JavaScript.

Today JavaScript is king when it comes to client-side web programming. It took us a while to reach this point. In the collective mind, JavaScript was considered a poor language used by amateurs to create annoying web pages. Today (thanks to AJAX amongst others factors) it’s a language that’s appreciated by many professionals and used by the vast majority of web developers. Whether you program web applications in Ruby, Python, Perl, PHP, C#, or something else, you’ll deal with JavaScript (it’s the greatest common denominator of the web development community). I know of very few professional web developers who lack a cursory knowledge of the language (or its cousin, ActionScript).

Over the past few years the browser has become the single most important application on users’ computers. This in turn, sealed JavaScript’s fate for the foreseeable future. Despite its many flaws, JavaScript is a powerful and elegant language that has advanced features which are blatantly missing from “full blown” languages like Java. Programmers have come to realize its power and usefulness within the browser. Beautiful JavaScript frameworks like jQuery, YUI, and more recently SproutCore and Cappuccino (Objective-J), showcased the art of what it’s possible to accomplish with this language. And with HTML5 becoming closer to reality, there will be an ever greater emphasis on DOM scripting and less reliance, when feasible, on RIA plugins.

If generally speaking, JavaScript is a solid and powerful language that most web developers need to know anyway, why can’t we develop in JavaScript server-side as well? And while we’re at it, maybe use it for desktop applications too? It would seem rational to capitalize on the benefits of having a huge percentage of programmers use the same language for both client and server-side programming. (If an update to the language is required to clean it up a little, let’s do that.) Why shouldn’t we be able to run js myscript.js, outside of a browser, and obtain the result of the computation in output? There is no inherent reason why JavaScript needs to be tied to the browser.

Thankfully times are changing and concrete answers to those rhetorical questions are emerging. The V8 JavaScript Engine is a project that was started by Google which provides us with a standalone shell to execute scripts and try out code in a basic REPL (Read-eval-print-loop). It’s the same engine embedded into Google Chrome, and as such, it’s a fast implementation as well.

Another great effort that’s headed in the same direction, and builds on top of V8 is Node.js, an evented I/O framework. You can think of it as Tornado, Twisted or EventMachine, simplified for server-side JavaScript. Node doesn’t require as much knowledge about event loops and non-blocking I/O, and the look and feel of such callbacks is very reminiscent of the type of AJAX code we’ve all seen before. Node can easily be used as a basic, ultra fast web server, to which one can delegate I/O callbacks for scalability and efficiency.

Recently Heroku announced beta support for Node¹. It’s a risk on their part, but one worth taking in my opinion. If nothing else, at the very least, Rails developers deploying on Heroku will have the option to integrate Node to increase scalability and performance.

But Node (which embeds the V8 engine) has a lot more potential than just that. The ultimate goal is to become a self-contained solution which would allow one to develop and deploy server-side JavaScript code in production mode.

Node is just a prominent example of the impact of the CommonJS project/movement, which is aimed at making JavaScript available outside of the browser (on severs and desktops). There is in fact an ecosystem of new .js libraries that are meant to be used with server-side JavaScript (this is likely to grow over time).

What we really need is a lightweight web framework that well integrates server and client-side JavaScript. This would have game changing potential (think Rails back in 2004). Developers have grown accustomed to a high level of abstraction when it comes to web development though, so there are a couple of possibilities here: either Node will become that framework or someone will create such a framework (perhaps on top of Node). Whoever does that will hold a piece of future and a golden ticket in their hands.

[1] For a terrific demo of a Cappuccino + Node application deployed on Heroku, check out GitHub Issues.

On the iPhone, iPad and Android Market Share

There has been quite a bit of discussion over the market share of mobile devices today (arguably, for the past few years). It all started with a link on TechCrunch, claiming that Android overtook the iPhone in terms of US traffic (according to AdMob). This being a clear case of selection bias, I set about to figure out the mobile devices used by the visitors of some of my sites during the past month.

As these figures will obviously depend on the type of audience that a site attracts, I put the following three sites on the table as samples (the first two are mine, the last one is my wife’s). All three sites have a predominantly North American audience, but they aren’t lacking for international readers either. This is not meant to be a scientific survey, just a quick way to gather some empirical data and satisfy my (and perhaps your) curiosity. It’s also important to understand that the “market share” here is not actually the market share of all mobile devices sold, but rather limited to those normally used for web browsing.

Let’s start with the data for this site which obviously has a very technical audience:

Next are the results for Math Blog, with has a geeky, but less technical audience.

Finally, my wife’s blog which is devoted to all things vintage. Her audience is generally non-technical and prominently comprised of female visitors.

A few thoughts on this data:

• No matter how we look at it, the iPhone is still much more popular than Android, particularly among technical people. If we consider all the devices running iPhone OS, the outlook is even less encouraging for Android OS.
• These numbers suggest that Android probably has a 10 to 20% “web usage share” amongst mobile devices.
• In one month, the iPad managed to grab a 5 to 10% slice of the mobile pie. Assuming we are OK with grouping it together with smart phones and PDAs, I suspect its share will become much larger in the coming months.

Android offers you a great degree of freedom, compared to the iPhone OS due to Apple’s draconian policies. But the market share for Android still seems to be somewhat small (at least according to the, admittedly limited, figures above). This may change in the future, if and when the number of popular devices that run Android continues to grow (which would include alternatives to the iPad).

With Android still having a long way to go before it catches up with the iPhone OS, which of the two would you develop for, assuming you could only pick one?

Rails, DB2 and the Enterprise

Recently Matt Aimonetti wrote an insightful article about Rails and the Enterprise. In it he identifies five core Enterprise application needs:

• Reliability
• Support
• Performance
• Advantage over the competition
• Integration and transition path

Matt then proceeds to illustrate how Rails does a good job in regards to most of these points, despite a few existing challenges.

Among these challenges, I can clearly see the following:

• There isn’t a 1-800-RAILS number; the community may be great, but there are not exactly yearly contracts in place. Furthermore, the author of the Ruby driver for, say, Oracle doesn’t owe you a thing. He may or may not be there for you when you need something to be fixed quickly.

• XML support is less than ideal (but it is improving).

• Integration with the Enterprise world is not easy, due in part to less than stellar SOAP support (but that is also improving).

• If you’re taking advantage of ActiveRecord and an Enterprise database like Oracle, your DBA isn’t likely to be happy that you aren’t using prepared statements.

You may think these are small points, and in the startup world they generally are. However, in the Enterprise world they do make the difference between adoption and niche.

One thing that Matt forgot to mention is DB2, which should be the poster child for how Rails can be Enterprise ready. And as a bonus, you get to throw around IBM’s name (which is synonymous with Enterprise) because Rails is both supported by, and used within, IBM.

Let’s address each point above with DB2 on Rails in mind.

IBM is the only database vendor to provide a Ruby driver and ActiveRecord/Rails adapter for its databases. This means that you have a team that’s accountable when things don’t work as they’re supposed to. This team is accountable, regardless of whether you have a contract with IBM or not; it’s their job, not a hobby. This involvement with Rails dates back to 2006, with continuing releases and improvements ever since. Our IBM’s optional yearly 24/7 support contracts (e.g., for less than $3000 a year per server with DB2 Express-C) include support for DB2 on Rails as well.

DB2 supports native storage and querying of XML documents and data (plus it’s fast). This technology is known as pureXML.

DB2 can both consume and serve SOAP web services. This lets DB2 do the integration for you.

DB2 on Rails supports parameterized queries. (While on the subject of queries, if you are using JRuby you can take advantage of pureQuery, which is an IBM created Enterprise solution that’s aimed at making your queries fast, reliable, manageable and easy to debug.)

If you are trying to introduce Rails into your Enterprise job, chances are that DB2 will already be present within the company infrastructure. If not, you can use DB2 Express-C which is entirely free — thus making it easier to introduce than an expensive solution.

IBM is one of the most trusted brands on the market today, as it has been for decades now. Banks and Enterprise companies the world over trust DB2 with their most critical data. One way for the Rails community to increase the adoption of Rails in the Enterprise, is to acknowledge and embrace the great pair that is Rails and DB2.

A tale of two search engines

Remember when Altavista seemed good enough? Then along came Google and seemingly overnight everything changed. We didn’t even know that it was possible to receive such good link suggestions from a search engine. Yet there, right before our very eyes, it happened.

These days, highly popular search engines are worth billions of dollars, mainly thanks to the massive advertisement businesses that can be built on top of them. The incentive to get a slice of that huge pie is clearly there. But can we do better than Google? And will such innovation necessarily arrive from the research labs of giants like Apple or Microsoft?

The answer to the first question is obviously yes. There is always room for improvement. The latter question may appear equally obvious as well, but let’s take the tale of two different search engines into consideration.

One of them was started by xooglers (ex-Google employees) with plenty of experience in the field of search engines; it had a team dedicated to its development and could afford to have a VP of communications. They received plenty of funding ($33M) to get the ball rolling, and garnered a fair bit of press coverage when they first launched.

A very different search engine, however, was started by just one person. It was bootstrapped (no external investors), came wrapped in a silly name, and virtually no one paid attention to its launch.

The first is Cuil (pronounced ‘cool’), which as many know, has become something of a running joke online. It’s the perfect example of how not to create a startup and of everything that could possibly go wrong with an ambitious software project.

From day one the results were so incomplete and irrelevant, that one has to wonder if the $33M they received was spent on developing the technology needed to clone Samuel Becket and place him in charge as the chief architect of the project. The absurdity of Cuil’s search results even led to the development of a highly entertaining Cuil Theory.

A few days ago the Cuil team launched an automated Wikipedia of sort, called cpedia. The end results were so terrible, that they will serve as an eternal cautionary tale against the indiscriminate use of Markov chains.

Unless Cuil/Cpedia is a practical joke aimed at the tech community, they may as well shut them both down. At this point they really are just wasting their time (and ours).

The second search engine, the one with a rather wacky name, was created by a single person (Gabriel Weinberg) and is called Duck Duck Go (a play on the name of that old childhood favorite, ‘Duck, duck, goose’). However, much to the surprise of many, there is real innovation going on here (particularly presentation wise).

Duck Duck Go’s zero click information is very useful – as is dividing the results by topic (for example, the word “ruby” can have different meanings in different contexts). Presenting all the search results on a single page (via AJAX) was a smart and handy feature as well. With this search engine your privacy is respected, particularly since your IP is not even logged.

This site is still as niche as it gets, but it’s reaching a tipping point amongst the ultra-geeks — not to mention that more and more people (myself included) are adding Duck Duck Go as their default search engine within their browsers. Is it better than Google? No, not always. It depends on the type of query. Sometimes it’s better, sometimes it’s worse, but it’s usually quite usable and is a concrete attempt to innovate the search engine realm.

Duck Duck Go serves as a poignant reminder to the software world that David can still strike Goliath.

My kingdom for an iPad

Tomorrow the iPad goes on sale in the States. Announced in January, the iPad sits squarely between a laptop and an iPod Touch. Large lines are expected to form in front of Apple Stores across America; ants scurrying to grab their crumbs.

What is uncertain is whether this release is going to be much ado about nothing or more an event that will revolutionize the computer market.

Among the iPad shortcomings are the following:

• A somewhat embarrassing name;
• Lack of Flash support;
• Inability to multitask (exception made for some Apple built-in apps);
• Software restrictions due to DRM;
• Lack of webcam;
• Lack of USB ports;
• Not as portable as an iPod, iPhone or a tablets a la Nokia N900;
• Not ideal for long typing sessions, due to a virtual keyboard (even though an external keyboard can be purchased as an optional accessory).

Your first impression might be that we are dealing with a flop like the Apple TV or similar niche products that are popular with Apple fans, but lack the transformational power and impact on society that have been shown by the iPod or the iPhone.

Much of the iPad’s criticism comes from a fundamental misunderstanding of the target use of this product. Those who consider the iPad a replacement for their laptop will no doubt be disappointed by the performance and restrictions of this device.

One also needs to take into consideration the iPad’s target audience. Many assumed that the target audience was primarily composed of geeky early adopters, programmers, and more in general, people with a technical mindset.

Far from telling my readers that they shouldn’t indulge in the prohibited pleasure of possessing an iPad, it seems clear to me that the Gaussian function has its maximum elsewhere, amongst students, casual users, and the general public, who want a device from which to check their email, surf, and play from their couch, kitchen or local coffee shop. A computing device that is small enough to carry in a purse, but large enough (with its 9.7" diagonal) to easily display websites and applications, without causing one to squint their eyes.

Seen in this light, the iPad has a solid reason for being, despite all of its limitations. Imagine a computer that is accessible and easy to use, and doesn’t require IT support from your nephew to fix (or remove a virus). In other words, a portable device that simply works. It only does a few things, but it does them in a manner that provides a pleasant experience to the average user.

The iPad’s field of application isn’t very restricted either. It can be seen as a portable console for casual gaming, a digital frame to show photos set to a soundtrack, or a quick presentation tool at a small business meeting. It’s a multimedia tool for listening to music and watching videos and lectures. Finally, the iPad is also a magazine and ebook reader. Some may rightfully argue that the e-ink technology is easier on the eyes for extensive reading, but the iPad has a vibrant color screen, and is able to display complex PDFs as well as ePub books sold directly from the brand new iBookstore.

The design, as is customary for products designed by Jonatahn Ive, is minimalistic, sleek and easy on the eyes. Starting at $499 (as shown in the figure below), the price point is rather competitive, so as to be able to reach a wide, international audience.

All things considered, it’s easy to imagine that the iPad will be a commercial success with the potential to transform how millions of consumers approach the Internet, gaming and book reading. This is far from certain, but I suspect that the iPad will be the iPod of the laptop world.

My suspicion is further supported by the ecosystem that surrounds this form of lightweight computing. Teachers will love the possibilities that such a device opens up, where students can now have a more interactive and multimedia-driven experience (particularly if ad hoc applications are created for this).

Book and magazine publishers already love the idea of selling books through the iBookstore, a refuge from the totalitarian price policies imposed by Amazon. This could in turn, really increase publishers’ investment in the digital world.

Programmers will be able to explore new ideas and create applications that are specifically tailored to the iPad user interface and user experience. The many advantages, and few disadvantages, of this approach are well known thanks to the iPod Touch and iPhone experience.

While restrictions are obviously limiting, they can also foster creativity. Among a sea of silly gag applications, there were also truly innovative apps designed for the iPhone. I would expect nothing less from applications developed using the same tools and distributed through the same channels, but targeted to a device that has much larger screen and processing capabilities.

Personally, I don’t know if I will purchase an iPad or not, after all I spend far too many hours in front of a traditional laptop already. But I clearly see a brilliant future for this new Apple product, despite its limitations and the closed approach to hardware and software that has become typical of Cupertino’s company.

What about you, are you headed to the Apple Store?

Switching from Feedburner Email to AWeber for my newsletter

This is just a brief post to let my readers know that I’m switching from Feedburner Email’s service to AWeber to handle sending out my posts via email. Why am I switching from a free service to a commercial one? Because AWeber offers countless features that I plan to use, plus it’s a lot more flexible than Feedburner. This change will only affect email subscribers. My Atom feed will still be handled by Feedburner, of course.

With Feedburner’s email service for example, I have no control over the emails that are sent out. With AWeber I’ll be able to create a newsletter service that not only sends out the posts I publish on this blog, but it will also allow me to send out custom written emails. I want to cultivate my newsletter and doing so starts with a service that allows me to do just that.

If you currently subscribe to my email newsletter via Feedburner you’re subscription will automatically be imported to AWeber and you will receive an email asking you to confirm the subscription. Unless you click on the activation link in that email, you will not be added to the new newsletter service. This is because the new newsletter has a double opt-in confirmation feature enabled. Starting tomorrow Feedburner’s email service will be disabled, as AWeber steps in to take over.

I invite you to join my newsletter by providing your first name and email address in the email sign-up on the left hand side bar, or by clicking here.

Thank you to all those who sign up; I look forward to bringing my posts to your inbox via AWeber.

Heads up: IBM is looking for top notch student hackers

As a thank you for following my blog, I’d like to introduce you to what I think is a great opportunity for the right students. My team is looking for two bright students for a 16 month, full-time internship opportunity with IBM.

Aside from being a bright and ambitious student, you should currently be working towards a Computer Science degree at any recognized University in the world (and have completed at least 2 years of your degree). You read that right, this opportunity is not limited to Canadian students. We are looking for the most talented students in the world, so as long as you’re ready to relocate to Toronto for 16 months, and are able to start the internship program in May 2010, we have the means to get you here. The location for this internship will be the Toronto Software Lab, in Markham, Ontario.

In particular, we are looking for students who are highly skilled in the area of web technologies and who are enthusiastic about cloud computing. This is NOT your run-of-the-mill, boring student job. You’ll be working with cutting edge technology on high visibility, fast paced projects that involve web technologies, cloud computing, Xp/Agile methodologies, and other related fields. You’ll be granted ample freedom to express yourself and your programming creativity. Should you be accepted for this well paid (in my opinion) internship program, you will report directly to my manager (a cool, down to earth guy) and I’ll be your mentor.

If you fit the criteria described above, please get in touch (cangiano at ca dot ibm dot com) and tell me a bit about yourself. You don’t need to attach your résumé at this stage, but if you drop me a line, I’ll be able to provide you with info on how to apply through the official IBM channels. We’ll take it from there.

Please note that I do NOT receive any form of compensation for referring you. I’m just hoping to provide a tiny subset of my readership with what could be a really great opportunity.

Setup Ruby Enterprise Edition, nginx and Passenger (aka mod_rails) on Ubuntu

The following is a very short guide on setting up Ruby Enterprise Edition (REE), nginx and Passenger, for serving Ruby on Rails applications on Ubuntu. It also includes a few quick and easy optimization tips.

We start with setting up REE (x64), using the .deb file provided by Phusion:

wget http://rubyforge.org/frs/download.php/66163/ruby-enterprise_1.8.7-2009.10_amd64.deb
sudo dpkg -i ruby-enterprise_1.8.7-2009.10_amd64.deb
ruby -v

In output you should see “ruby 1.8.7 (2009-06-12 patchlevel 174)…” or similar. If this is the case, good; while you are there, update RubyGems and the installed gems:

sudo gem update --system
sudo gem update

Next, you’ll need to install nginx, which is a really fast web server. The Phusion team has made it very easy to install, but if you simply follow most instructions found elsewhere, you’ll get the following error:

checking for system md library ... not found
checking for system md5 library ... not found
checking for OpenSSL md5 crypto library ... not found

./configure: error: the HTTP cache module requires md5 functions
from OpenSSL library.  You can either disable the module by using
--without-http-cache option, or install the OpenSSL library in the
system,
or build the OpenSSL library statically from the source with nginx by
using
--with-http_ssl_module --with-openssl=
 options.

Instead, we are going to install libssl-dev first and then nginx and its Passenger module:

sudo aptitude install libssl-dev
sudo passenger-install-nginx-module

Follow the prompt and accept all the defaults (when prompted to chose between 1 and 2, pick 1).

Before I proceed with the configuration, I like to create an init script and have it boot at startup (the script itself is adapted from one provided by the excellent articles at slicehost.com):

sudo vim /etc/init.d/nginx

The content of which needs to be:

#! /bin/sh

### BEGIN INIT INFO
# Provides:          nginx
# Required-Start:    $all
# Required-Stop:     $all
# Default-Start:     2 3 4 5
# Default-Stop:      0 1 6
# Short-Description: starts the nginx web server
# Description:       starts nginx using start-stop-daemon
### END INIT INFO

PATH=/opt/nginx/sbin:/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
DAEMON=/opt/nginx/sbin/nginx
NAME=nginx
DESC=nginx

test -x $DAEMON || exit 0

# Include nginx defaults if available
if [ -f /etc/default/nginx ] ; then
    . /etc/default/nginx
fi

set -e

. /lib/lsb/init-functions

case "$1" in
  start)
    echo -n "Starting $DESC: "
    start-stop-daemon --start --quiet --pidfile /opt/nginx/logs/$NAME.pid \
        --exec $DAEMON -- $DAEMON_OPTS || true
    echo "$NAME."
    ;;
  stop)
    echo -n "Stopping $DESC: "
    start-stop-daemon --stop --quiet --pidfile /opt/nginx/logs/$NAME.pid \
        --exec $DAEMON || true
    echo "$NAME."
    ;;
  restart|force-reload)
    echo -n "Restarting $DESC: "
    start-stop-daemon --stop --quiet --pidfile \
        /opt/nginx/logs/$NAME.pid --exec $DAEMON || true
    sleep 1
    start-stop-daemon --start --quiet --pidfile \
        /opt/nginx/logs/$NAME.pid --exec $DAEMON -- $DAEMON_OPTS || true
    echo "$NAME."
    ;;
  reload)
      echo -n "Reloading $DESC configuration: "
      start-stop-daemon --stop --signal HUP --quiet --pidfile /opt/nginx/logs/$NAME.pid \
          --exec $DAEMON || true
      echo "$NAME."
      ;;
  status)
      status_of_proc -p /opt/nginx/logs/$NAME.pid "$DAEMON" nginx && exit 0 || exit $?
      ;;
  *)
    N=/etc/init.d/$NAME
    echo "Usage: $N {start|stop|restart|reload|force-reload|status}" >&2
    exit 1
    ;;
esac

exit 0

Change its permission and have it startup at boot:

sudo chmod +x /etc/init.d/nginx
sudo /usr/sbin/update-rc.d -f nginx defaults

From now on, you’ll be able to start, stop and restart nginx with it. Start the server as follows:

sudo /etc/init.d/nginx start

Heading over to your server IP with your browser, you should see “Welcome to nginx!”. If you do, great, we can move on with the configuration of nginx for your Rails app.

Edit nginx’ configuration file:

sudo vim /opt/nginx/conf/nginx.conf

Adding a server section within the http section, as follows:

    server {
        listen 80;
        server_name example.com;
        root /somewhere/my_rails_app/public;
        passenger_enabled on;
        rails_spawn_method smart;
    }

The server name can also be a subdomain if you wish (e.g., blog.example.com). It’s important that you point the root to your Rails’ app public directory.

The rails_spawn_method directive is very efficient, allowing Passenger to consume less memory per process and speed up the spawning process, whenever your Rails application is not affected by its limitations (for a discussion about this you can read the proper section in the official guide).

If you have lots of RAM (e.g., more than 512 MB) on your server, you may want to consider increasing you maximum pool size, with the directive passenger_max_pool_size from its default size of 6. Conversely, if you want to limit the number of processes running at any time and consume less memory on a small VPS (e.g., 128 to 256MB), you can decrease that number down to 2 (or something in that range). (Always test a bunch of configurations to find one that works for you). You can read more about this directive, in the official guide.

While you are modifying nginx’ configuration, you may also want to increase the worker processes (e.g., to 4, on a typical VPS) and add a few more tweaks (such as enabling gzip compression):

# ...
http {
    passenger_root /usr/local/lib/ruby/gems/1.8/gems/passenger-2.2.5;
    passenger_ruby /usr/local/bin/ruby;

    include       mime.types;
    default_type  application/octet-stream;

    access_log  logs/access.log;

    sendfile        on;
    keepalive_timeout  65;
    tcp_nodelay on;

    gzip on;
    gzip_comp_level 2;
    gzip_proxied any;   

    server {
    #...

When you are happy with the changes, save the file, and restart nginx:

sudo /etc/init.d/nginx restart

If you wish to restart Passenger in the future, without having to restart the whole web server, you can simply run the following command:

touch /somewhere/my_rails_app/tmp/restart.txt

Passenger also provides a few handy monitoring tools. Check them out:

sudo passenger-status
sudo passenger-memory-stats

That’s it, you are ready to go! I hope that you find these few notes useful.

Add code highlighting to your Google Waves

Google Wave is still rough around the edges, but it has a lot of potential in terms of becoming a great collaboration tool. As a developer, your first question will probably be: “How do I add code highlighting to my waves?”. The answer is straightforward, however not very easy to find if you google it. I hope this post will help fellow developers who are experimenting with Google Wave.

The following steps are required to obtain syntax highlighting for your code:

1. Create a new wave and add the Syntaxy robot to your wave. Use the wave address: kasyntaxy@appspot.com.
2. Reply to your first message or within it, thereby creating a reply (called “blip” in Google lingo).
3. Specify your code’s language, prefixing the name with a hash and exclamation mark, like #!python or #!ruby.

At this point, as you type the code in your blip it will be highlighted by the Syntaxy bot as shown in the picture below:

More advanced automatic syntax highlighting bots will probably appear as Google Wave progresses, but this one should do the trick for now. On a side note, if you copy and paste code from XCode, the code formatting will be kept in your waves and blips without the need for bots.

Getting MacRuby’s compiler to work

There is major news in Rubyland today. MacRuby’s team just released their fist beta of version 0.5 (an experimental, still incomplete version of Ruby), which brings JIT, removal of the dreaded GIL (Global Interpreter Lock), native threads, GCD (Grand Central Dispatch) for multicore computing, and a whole new set of features found in the release announcement to the table.

The most important new feature is the presence of a compiler. That’s right, thanks to this release, Ruby code can now become highly optimized executable code. How awesome is that? I can sense that you’re pumped by this news, so why not head over to MacRuby.com and download the installation file for yourself? After you’ve done that, the next thing you’re going to want to do is run a small test like the following:

$ macrubyc world_domination.rb -o world_domination
Can't locate program `llc'

Oh noes! llc is a tool that ships with the LLVM (upon which MacRuby is built), however it’s not included with MacRuby’s installer (it will be in the future). But fear not my friends, there is a solution:

$ svn co -r 82747 https://llvm.org/svn/llvm-project/llvm/trunk llvm-trunk
$ cd llvm-trunk
$ ./configure
$ UNIVERSAL=1 UNIVERSAL_ARCH="i386 x86_64" ENABLE_OPTIMIZED=1 make -j2
$ sudo env UNIVERSAL=1 UNIVERSAL_ARCH="i386 x86_64" ENABLE_OPTIMIZED=1 make install

If your machine does not have 2 cores, remove the -j2 option from the fourth line or adjust the number accordingly.

The compilation phase may take a couple of centuries, depending on your machine’s speed, but it should eventually build the LLVM. llc will be placed in your PATH, and you’ll finally be able to compile Ruby code and obtain an executable to help you carry out your world domination plans.

$ macrubyc world_domination.rb -o world_domination
$ ./world_domination
MUAHAHAHAHA!

The CRTC tries to eliminate ISP competition in Canada

The CRTC is at it again. This time they want to allow Canadian phone companies to cut off their wholesale offering, essentially killing ISP competition in Canada. Please express your concerns here, where you can quickly email Industry Minister Tony Clement, Prime Minister Harper, the Opposition Leader (Michael Ignatieff) and your local MP. Let them know that you care about this important issue.

For more details, I’m attaching an informative email I received from TekSavvy:

Hello,

We need your help — we need you to lend us your voice and state your opinion on a recent CRTC decision that could allow the incumbent phone companies (such as Bell and Telus) to discontinue offering services (Internet and phone) to wholesale providers which in effect would eliminate competition in Canada.

With no competition to help fuel the market; Internet innovation could come to stand still and prices for broadband access could skyrocket out of control. The incumbents would also be free to do with the lines as they please. This would open up future privacy invading Deep Packet Inspection (DPI) technologies and direct marketing/advertising based on *you* and *your families* internet activities and habits.

They could also use this same technology to turn the Internet into a pay-per-website service the same way that TV is a pay-per-channel service. Can you imagine a world where you have to pay to use the Google website to search for information? Then, if you find what you are looking for you may be restricted to the content because your ISP doesn’t have a “peering” agreement with said company.

Please help us stop this major potential catastrophe.

All concerned consumers please visit:
http://www.competitivebroadband.com/consumer/

All concerned businesses please visit:
http://www.competitivebroadband.com

Read the information presented and decide for yourself.

Once you’ve decided what you think is right, you can click the “Make a Difference” link, fill out the form with your details and send a pre-written message to Minister of Industry Tony Clement, Prime Minister Harper, the Opposition Leader, and your local MP or choose to write your own.

Together we can make a difference.
Thank you,
TekSavvy Solutions, Inc.

Cell Phone Cost Calculator Killed In Canada. Enough Is Enough!

Having been born and raised in Europe, I find the Canadian Telco sector appalling. In what is an otherwise outstanding country, the monopolistic tendencies and de facto cartels of the phone companies are screwing over Canadian residents, and there is very little being done to counteract this.

Recently Bell got its way again, and UBB (Usage Based Billing) was introduced regardless of what thousands upon thousands of taxpayers had to say about it. Countless complaints were forwarded to the CRTC (Canadian Radio-television and Telecommunications Commission), and these were promptly disregarded.

On Slashdot today there’s a story entitled Cell Phone Cost Calculator Killed In Canada. Upon further inspection it’s revealed that cell phone carriers have successfully lobbied public officials to stop a taxpayer funded initiative that would publish an online cellphone cost calculator. (“OMG competition! Think of the shareholders!” As someone pointed out on Slashdot).

Stopping this service is a waste of our money, but more importantly, it clearly highlights the fear of transparency and competition that is typical of companies like Bell and Rogers. It shows the power that phone companies have over the government. And it demonstrates how elected officials like Tony Clement (Industry Minister) are far more concerned with the bottom line of public companies than the interest of Canadian citizens and residents alike.

This is outrageous. How can we fight back? For once, I’m a believer in voting with my dollars. I do not currently own a cell phone, and I surf the web on a (factual) 3 Mbps connection from TekSavvy, having switched from Rogers’ 10 Mbps plan more than a year ago.

The two are an impediment, because I need a cell phone and a fast Internet connection. But the lack of competition doesn’t leave you with many alternative options if you want to avoid giving your hard earned money to the types of carriers mentioned above. And in the case of the calculator, my tax money ended up being wasted to protect these companies’ obsolete pricing models.

The (always excellent) Michael Geist makes a worthwhile suggestion:

With public dollars having funded the mothballed project, the government should now consider releasing the calculator’s source code and enable other groups to pick up where the OCA (Office of Consumer Affairs) left off.

Today I registered MyPhoneBill.ca. Should they ever release the source code, I will host it and deploy it at this handy URL. After all, we’ve already paid for the code.

In truth, I don’t see this happening. An open source solution that companies and individuals could build upon would frighten cell phone carriers even more than a closed source project deployed by the government.

If they won’t open source the code of the cell phone cost calculator, as I suspect the case will be, I’ll take a stand. I’ll implement and offer the service myself at MyPhoneBill.ca (site not active yet).

Such a site may very well have a solid business model, and I’m not opposed to that idea. But that’s not the main reason why I’d create such a project. Canadians have the right to learn about what the most advantageous phone plan for their usage pattern is, without having to spend days on end doing research. And companies have no right whatsoever to prevent this from happening. Similar sites exist in almost every other industrialized country and cell phone companies usually collaborate with them.

Enough is enough!