How to setup Clojure from scratch
My previous post about Clojure generated quite a bit of interest, so I thought I’d follow it up with something a bit more concrete. I primarily wrote this article for a friend who asked me for guidance on how to set it all up; and while this isn’t he only way to setup Clojure, I hope it will help other people who are also getting started with this great language.
As some people pointed out, setting up your Clojure environment can be slightly challenging. For example, when you install Ruby or Python, you can expect a Read-Eval-Print Loop (REPL) such as IRB or IDLE to be ready for you to use right out of the box, whereas the current version of Clojure (version 1.1) does not include an interactive top-level.
This post explains how to setup a Clojure environment step-by-step, including a working clj script (the common name for Clojure’s REPL). For this post I used the clj script contained within the Getting Started guide on Wikibooks as a base, but I built on top of it , so as to customize and improve it. In writing this post, I’ve made the assumption that you’re using a *nix environment. My instructions will be Mac OS X specific, even though it should be a breeze to adapt them for Linux, if required.
Step 1: Download and Build Clojure
The first step is to download and build the latest version of Clojure. (Please note, I used bit.ly below for code formatting reasons.)
$ cd /tmp
$ wget http://bit.ly/clojure
$ unzip clojure-1.1.0.zip
$ cd clojure-1.1.0
$ ant
This will generate several jars for us, including clojure.jar, which we’re interested in. Aside from clojure.core, we are also interested in clojure.contrib, a collection of useful libraries and functions that are often used in Clojure programs. To download and build clojure.contrib, follow these instructions:
$ cd /tmp
$ wget http://bit.ly/clojure-contrib
$ unzip clojure-contrib-1.1.0.zip
$ cd clojure-contrib-1.1.0
$ ant
This should generate a clojure-contrib.jar file. We can now proceed to copy both the clojure.jar and clojure-contrib.jar files to a convenient location. (You can copy the files with a single line, but for formatting reasons I split the operation into two copy operations).
$ mkdir -p ~/Library/Clojure/lib
$ cd ~/Library/Clojure/lib
$ cp /tmp/clojure-1.1.0/clojure.jar .
$ cp /tmp/clojure-contrib-1.1.0/clojure-contrib.jar .
Note that on Linux, you’d choose a different location (e.g., /opt/clojure/lib).
Step 2: Install rlwrap
Before we can proceed with creating a fancy clj script, we need rlwrap, which will make our interactive prompt much more friendly. On Mac OS X, if you are using MacPorts, you can simply run the following:
$ sudo port install rlwrap
Step 3: Create a clj script
Now we can finally finally create a clj script and save it as ~/Library/Clojure/clj:
#!/bin/sh
breakchars="(){}[],^%$#@\"\";:''|\\"
CLOJURE_DIR=$HOME/Library/Clojure/lib
CLOJURE_JAR=$CLOJURE_DIR/clojure.jar
CONTRIB_JAR=$CLOJURE_DIR/clojure-contrib.jar
CLOJURE_CP=$CLOJURE_JAR:$CONTRIB_JAR:$PWD
if [ -f .clojure ]; then
CLOJURE_CP=$CLOJURE_CP:`cat .clojure`
fi
if [ $# -eq 0 ]; then
exec rlwrap --remember -c -b $breakchars \
-f $HOME/.clj_completions \
java -cp $CLOJURE_CP clojure.main -i $HOME/.clojure.clj --repl
else
exec java -cp $CLOJURE_CP clojure.main -i $HOME/.clojure.clj $1 -- $@
fi
Make it executable and symlink it to a convenient place on your path (e.g., ~/bin):
$ chmod a+x ~/Library/Clojure/clj
$ ln -s ~/Library/Clojure/clj ~/bin/.
At this point we almost have a working script. Before declaring it fully functional, let’s take a look at three files this script references:
.clojure: This file is optional and is located in the same folder as the scripts you want to execute. It can be used to include further jars in your classpath.~/.clj_completions: Includes a list of Clojure functions for tab completion (courtesy of rlwrap).~/.clojure.clj: Your Clojure startup file. It’s the first Clojure file that’s run when we start the REPL or execute a script. We’ll use this to add a couple of nice functions to the defaultcljREPL, but you can add anything you’d like to load at startup.
Step 4: Create a startup file
By default, clj will not have an exit function. You can either CTRL+D or enter a call to System/exit. This is not a big deal, but I like the idea of exiting with a simple (exit) call (as one does with Ruby and Python). Fortunately, we can add this function to the startup file. Furthermore, for learning purposes, few things beat having the source function which reveals the source code of built-in Clojure functions. This too can be added to ~/.clojure.clj:
(use 'clojure.contrib.repl-utils)
(defn exit [] (System/exit 0))
(exit) will now exit the REPL and source will reveal the source code as shown below:
user=> (source peek)
(defn peek
"For a list or queue, same as first, for a vector, same as, but much
more efficient than, last. If the collection is empty, returns nil."
[coll] (. clojure.lang.RT (peek coll)))
nil
Step 5: Create a ~/.clj_completions file
As explained in the Clojure Wikibook referenced above, you can create the .clj_completions file by running the following from the REPL or as a script (e.g., clj myscript.clj):
(def completions
(reduce concat (map (fn [p] (keys (ns-publics (find-ns p))))
'(clojure.core clojure.set clojure.xml clojure.zip))))
(with-open [f (java.io.BufferedWriter. (java.io.FileWriter. (str (System/getenv "HOME") "/.clj_completions")))]
(.write f (apply str (interleave completions (repeat "\n")))))
This will populate the file .clj_completions with a list of 500+ symbols. You can now run clj and try tab completion by inserting a few characters and then pressing tab. For example:
mbp:~ acangiano$ clj
Clojure 1.1.0
user=> (re<tab>
re-find ref-min-history repeatedly
re-groups ref-set replace
re-matcher refer replicate
re-matches refer-clojure require
re-pattern release-pending-sends reset!
re-seq rem reset-meta!
read remove resolve
read-line remove-method rest
read-string remove-ns resultset-seq
reduce remove-watch reverse
ref rename reversible?
ref-history-count rename-keys
ref-max-history repeat
There you have it, a complete setup of Clojure from source to clj. You can find information about setting up your favorite editor/IDE on the Assembla Wiki.
Note: As pointed out by Phil and other commenters elsewhere, the easiest way to get started these days is probably to install cljr, a standalone REPL and package manager. My post outlines how to do it from scratch (as you can deduct from the title), just like we used to before these tools were released. You may however consider the pre-packaged solution instead.
Thoughts on Clojure
Lisp has had a tremendous impact on the world of programming. Even though Common Lisp and Scheme — the two main Lisp dialects — may not be considered mainstream today, several popular languages have been influenced by one or both of them.
It isn’t stretching things too much to say that both Ruby and Python can be seen as slower, easier (for beginners), object-oriented, infix Lisp dialects.
Some may say Ruby is a bad rip-off of Lisp or Smalltalk, and I admit that. But it is nicer to ordinary people. — Yukihiro “Matz” Matsumoto
Ruby and Python aren’t intimidating and remain very approachable for absolute beginners. Furthermore, their approachability is not confined to the language design itself, but transcends into the community and ecosystem that surrounds them.
I’m not here to discuss how languages like Ruby and Python managed to become more popular than major Lisp dialects nowadays. I’d rather focus on how these gentler introductions to functional programming are acting as gateway drugs to Lisp for many developers.
A community that values metaprogramming and is obsessed with the construction of DSLs (Domain Specific Languages) like the Ruby’s is, will no doubt find in Lisp a valuable ally. Plus, if you know Ruby inside and out, you should find Lisp to be easy enough to learn.
To attract Ruby developers though, Lisp has to offer something more than just a set of powerful features. You could say that Rails is enough of a reason to learn and use Ruby. But what is Lisp able to solve all that better than Ruby? I’ll answer that question by focusing on a specific dialect of Lisp, that I and continually more Ruby developers are getting into: Clojure.
It wouldn’t be fair to characterize the Lisp community as stagnant, but Clojure is definitely a welcomed dose of new blood. Clojure is a JVM-based modern Lisp designed for concurrency, which elegantly includes a set of carefully chosen features that are not easily found in mainstream languages.
In my opinion, Clojure has three main advantages over Ruby:
- It’s much faster than Ruby, which makes it a better choice for intensive processing. (FlightCaster for example, uses both Rails and Clojure. Rails for the “front-end” and Clojure for the heavy lifting/forecasting.)
- It greatly simplifies concurrent programming, making the language more future-proof as hardware manufacturers continue to produce processors with more CPU cores.
- Clojure emphasizes functional programming and tries to minimize side effects.
Clojure’s interoperability with Java resolves the issue of only having a few available libraries, which often affects new languages. It also helps in getting people to use the language within the enterprise world where Java still dominates.
Of all the “new” languages out there, I find Clojure to be the most fun, interesting and pragmatic: it’s something worth getting excited about. I don’t really care if it turns out to be the next Ruby or not, it’s a language that’s worth knowing and using. (If you haven’t tried it yet, a decent, short introductory book is the recently published Practical Clojure.)
Clojure’s popularity may even bring more attention to Lisp in general (for example, most must-read literature uses Scheme or Common Lisp). Perhaps then, it may indirectly help introduce more traditional Lisp dialects to a new generation of programmers.
The pursuit of excellence in programming
As I write a series of thoughts on the pursuit of excellence in programming, I must preface my essay by asking you to ignore that I wrote these words. I invite you to evaluate the opinions and ideas presented here not ad hominem, but rather on the basis of their own merits. It would be easy to otherwise mistakenly dismiss them with the infamous question posed by Steve Jobs to a blogger: “What have you done that’s so great?”.
This is to say that I talk about the ambitious and noble goal of achieving excellence in programming, fully conscious of not having achieved said excellence. For the time being, I don’t feel like I can point my finger at something that would impress Steve Jobs (or less critical observers). I’m just a traveler on the journey of learning, with a desire to share his experiences and plans.
Two visions of intelligence
Mastering a complex discipline such as programming requires a great amount of learning over the course of several years, perhaps even decades. Maximizing one’s ability to learn is therefore an early investment that can quickly repay itself.
The biggest impact on my ability to learn was caused by a shift in the way I considered the matter of intelligence. There are mainly two ways to think about it. You can either consider intelligence to be a static, intrinsic ability or a more dynamic, cultivable characteristic of human beings.
Cognitive scientists and psychologists conclusively determined that people who perceived intelligence as a dynamic characteristic, outperformed and were more successful than people who internalized intelligence as an intrinsic, static ability.
It’s worth noting that it’s not really important whether intelligence can actually be developed through application. It’s the perception of it that forges students’ approach to learning.
This difference in perception is often conditioned by early parenting. Kids who are encouraged to work hard to achieve results and are praised on the basis of their effort, tend to develop a perception of intelligence and results as something they can work on. Other kids are conditioned to think that they are doing well because they are “smart” and that their intelligence alone will most likely lead them to success.
Society has a fascination with genius, and parents like to fancy their little ones to be several standard deviations better than the norm, but conditioning children this way has dangerous and counterproductive consequences.
Kids who are labeled and praised because of their “innate capabilities”, will often suffer from an overconfidence that will affect their ability to challenge themselves through the depths of the unknown, because they feel it would threaten their status. What if they fail? It would mean, in their eyes, that perhaps they are not the smart person they have been assumed to be all along. We all have seen such kids failing here and there, and quickly making excuses such as, “Oh, I wasn’t trying at all”.
A parent who is cultivating a kid’s interest in hard work, may be more likely to encourage their child with words such as, “It’s OK. Keep studying, and you’ll definitely do better next time”. A parent proposing a model of static intelligence, may justify their child’s failure in a given subject by concluding that “maybe you are not cut out for subject X”. [1]
When facing failure, the “static intelligence” child may crumble under the weight of his own demise, as if failure was a reflection of their intrinsic value rather than a temporary speed bump and occasion for growth. A “dynamic intelligence” child will simply try harder next time. Genius or not, excellence and mastery of any subject requires hard work and many “smart” kids fall short when the bar is raised high enough so that “smartness” alone won’t cut it anymore. This usually corresponds with the switch from high school to college.
I’m very familiar with all of this, because I was one of those kids. I was labeled by my parents and science teachers as a “genius”. Even psychologists at school, who came to help us figure out what careers we were better suited for, ended up telling me that I could pursue virtually any career (at the time I was interested in nuclear physics) and that according to their (virtually meaningless) IQ tests I would be classified as a “genius”.
Please note that the problem wasn’t so much the label. Most smart kids figure out that they are smart on their own rather quickly. The real problem was that I wasn’t taught the value of long-term intellectual effort. Effort itself was considered as being somewhat detrimental to my status. Not only was I supposed to succeed, but I supposedly had to do so without putting forth any effort (a “utopic” ambition).
One of the first example of this that I can remember is when my father “caught me studying” for a few hours the same book before a test in middle school. He told me something along the lines of “Why do you need to study? A genius like you should figure out the test without studying.”. It’s absurd, I know, and probably one of the dumbest things my — otherwise bright — father has ever told me. As a young kid though, such a statement can have a strong impact on you.
Another example has to do with Latin. My teacher was a palm reading, crazy cat lady and I had no respect for her from the start. So I didn’t pay attention in class, on top of not studying Latin at home, I set myself up for failure. When the first translation assignment came around, I got a mildly negative score. My less than professional teacher told me, “Oh I thought you were good, but I guess you are not”. Afterwards my father added to this by saying something along the lines of, “Well, don’t worry, I guess languages are not your strength.”.
Boom. That was enough for me to stop having any interest in Latin and completely ignore a subject at which I wasn’t excelling. Nobody could tell me, “You are stupid because you don’t understand Latin,” if I didn’t try at all. So I was the high school kid who did advanced Calculus stuff on his own for fun, when my classmates where struggling with Algebra, yet I pretty much sucked at Latin. (In retrospective, the thought patterns required to excel at Latin where not very different from those required to excel at math or mastering English as a second language. I suspect that had I put in some effort I could have been very good at, and actually enjoyed, the subject.)
Over the years I had to readjust this perception entirely. By falling on my face more than once, I learned that excellence is only achieved through a combination of talent and effort. The real genius may lie in the ability to put in thousands of hours of focused study and practice, in the pursuit of whatever one person is trying to learn and understand.
If people knew how hard I worked to get my mastery, it wouldn’t seem so wonderful at all. — Michelangelo
By this new definition, I was a complete idiot who had to entirely learn from scratch how to appreciate the value of effort to hone and develop his “talent” (which is nothing but a seed on its own).
Learning to learn
Being very interested in programming, computer science, mathematics, and science in general, I decided at some point that I had to entirely change my attitude towards learning if I were to master any of those disciplines. Effort was now more important than intelligence on its own, and I would feel satisfied only when doing a really good job in the pursuit of something challenging, that couldn’t be achieved by sheer “talent”.
The saddest thing in life is wasted talent, and the choices that you make will shape your life forever. — Calogero ‘C’ Anello from A Bronx Tale
In the process, I started to internalize a few principles and I dealt with issues related to both the art of learning and the art of programming.
An awful feeling
There is a cognitive bias known as the Dunning-Kruger effect [2], in which subjects who are inexperienced or less competent within a given discipline, tend to overestimate their abilities (they are in other words affected by illusory superiority [3]).
The other side of this coin is that the more you study, the more you realize how little you know and how much there is to know (a concept put forward by Socrates back in the days of Ancient Greece). This is both a pleasure and a discomfort. There is a huge amount of satisfaction in finding things out. Yet, being in doubt and fully aware of how little you know tends to be an unpleasant side effect all learners have to live with. Doubt truly is the water that’s fundamental for the growth of the flower of intellectual curiosity. [4]
My approach in this case is to embrace and dominate my ignorance and fears. Whenever there is a concept that I feel particularly ignorant about or that is way over my head, I try to tackle it as if my life depended on it. There are still countless things I’m ignorant about, but this approach as really paid off for me over the years.
If you are trying to learn a whole branch of computer science or mathematics, it will take a long time, so you may want to start first with smaller “fears” that can be mastered, at least at an introductory level, in a short amount of time. Rather than thinking, “Oh yeah, I should really learn Git”, for months, act on the thought. The essential knowledge to work with Git or Hg doesn’t take months to learn (assuming you have a need for either of these particular tools).
But you are in this for the long run, so don’t be afraid of improving your craft by studying advanced topics that require a bigger commitment in terms of time as well. There is no royal shortcut, mastery of our craft will require thousands of hours of dedicated study and practice.
Theory and practice
Masters of any intellectual discipline tend to have good working knowledge of both theoretical and practical aspects. Pursuing excellence in programming requires the study of many insightful books that will widen your view of the field and as a result necessarily improve your craft.
Working with books alone is not enough though. Programming requires writing and reading programs every day for years. That’s why I have a rule: I refuse to go to sleep if I haven’t read and written some code on a given day (this doesn’t include the code I write for work, of course). So far, this rule has had a positive impact on my ability to code.
It is a mistake to think that the practice of my art has become easy to me. I assure you, dear friend, no one has given so much care to the study of composition as I. There is scarcely a famous master in music whose works I have not frequently and diligently studied. — Mozart
Breadth Vs. Depth
As we progress in our journey towards the pursuit of excellence in programming, a question that will no doubt pester people’s minds is whether one should go for breadth of knowledge, or depth. There are countless programming languages, paradigms, methodologies, technologies, et cetera.
The truth of the matter is that if we are to become GrandMaster programmers, we cannot ignore either of them. In practice, depth has a much stronger impact in the way we construct software. It is through deep understanding that we can see the whole through the parts. There is therefore value in specializing in just a few languages and technologies, and really mastering them in-depth.
Getting things done in software development requires a certain pragmatism and proficiency with the tools at hand. There is no escaping it. Depth is therefore necessary, but not sufficient.
I find that the web is particularly good at covering the breadth aspect of things. There are always new and interesting areas I can learn about and experiment with. I don’t need a whole 600 page book to get a feeling for — or to better understand — certain technologies that are not crucial to my area of specialization.
While there are quite a few exceptions I could mention, I’d say that I tend to use the Internet as an aid for horizontal scaling of my knowledge, and books for vertical scaling. And again, more often than not, the depth level is mostly determined by the amount of time, practice and effort I put into it, rather than the media I’m using.
Where to find time
One of the objections I hear often, particularly when it comes to reading books is, “I don’t have time!”. In a few extreme cases, that may actually be true, but I think that most people severely underestimate how much time is “wasted” on a daily basis (even just surfing online).
Breaks are extremely important, and I’m not advocating any regime of incessant study. I simply know how crucial it is to be constant. It’s a marathon, not a sprint.
My second rule is: I refuse to got sleep if I haven’t read at least a chapter of a book that day. Very often I get caught up in the book I’m reading or working through, and end up getting through much more than just one chapter (it really depends on the book, of course). But for me the rule is clear: no sleep allowed until one chapter has been read every day.
Try this approach and you’ll see that reading doesn’t have to take up much time, yet doing so you can still read several books (including technical ones) every month.
We are what we repeatedly do. Excellence, then, is not an act but a habit. — Aristotle
For certain books, it is convenient to have the book in PDF format on your computer, as you switch from the book to the editor/console and back. However, generally speaking the computer tends to be quite distracting and thoughts like “I’ll just check my email quickly” can easily lead to hours spent doing something else.
For this reason I prefer to read from a paper book which is also easier on the eyes for extensive reading after a day in front of a computer screen. Even if I’m using the computer at the same time to input code, the physical presence of a book next to my laptop is enough to remind me that I shouldn’t get distracted online. (For me the depth and focus required by books is also an antidote to the re-wiring that the web tends to do to our brains. [5])
By the way, a few days ago Amazon announced a gorgeous, brand new graphite color Kindle DX [6]. I think I may pull the trigger and get it for my upcoming 30th birthday. Buying numerous paper books is expensive, and given the price of Kindle books, this move would end up being cheaper in the long run. The large e-ink display almost looks like paper and the device is not as distracting as an iPad (you read on a Kindle, and that’s it). Plus, it’s lighter than most technical books and surely takes up less space in your home.
Achieving focus
With so much going on within the programming world, distractions are easy to come by. My approach is to focus only on the given macro-task at hand. If I’m trying to learn about process calculi for example, then for the next few months my “learning time” will be ruthlessly dedicated to that subject, as if the rest of the programming ecosystem stopped in time.
Then there is focusing at a micro-task level. Learning about a given subject can always be divided into a long series of smaller steps. When I’m focusing on one such, tiny step, then everything else ceases to exist (or at least in theory).
One trick I use to achieve solemn focus on micro-tasks, whether reading code, writing code, or reading a technical book, is the use of the Pomodoro Technique [7]. In short, I use timer software [8] which alerts me when 25 minutes (a pomodoro) have passed, and gives me a 5 minute break for each pomodoro. Every 4 pomodoros, I can take a longer break.
When I first started using this technique I thought it was mostly a gimmick and I had a hard time taking breaks. I just wanted to keep going and the “tracking” seemed silly. However, I must say that it strikes an elegant balance between the desire to focus for extensive periods of time and the importance of taking regular mini-breaks.
This approach has become routine for my mind now, so even if it is just a gimmick, it’s still a good way for me to get focused and “in the zone”.
Aiming for sprezzatura
The pursuit of excellence requires a huge drive from within, and a fundamental dissatisfaction with just being good at a given discipline. I believe this is true regardless of the profession at hand.
As I progress in my journey, I’m discovering how the key is to make the pursuit of excellence a habit. This goes against my nature of being an intellect sprinter, but I’m in for the long run and I’m really learning to enjoy the, marathon like, process.
My long-term goal is to program with sprezzatura [9], where the process is so internalized and part of my subconscious that it almost looks effortless (as if the act of programming was committed to muscle memory). It will be an overnight success, 15 years in the making.
Regardless of the improvement level achieved, I will always have the joy, privilege and need to continue to learn for the betterment of myself and my craft. When there is no set destination, the journey is what really matters.
Ancora imparo. (I’m still learning.) — Michelangelo
Notes
[1] These concepts are explained, in a much more eloquent manner, in the early chapters of the excellent book, The Art of Learning by Josh Waitzkin.
[2] Dunning-Kruger Effect on Wikipedia.
[3] Illusory Superiority on Wikipedia.
[4] For more on the importance of doubt in science, check out the beautiful epilogue in What do you care what other people think? by Richard P. Feynman.
[5] For more on this phenomenon, read The Shallows by Nicholas Carr.
[6] The new Kindle DX on Amazon.
[8] Pomodoro for Mac OS X.
[9] Sprezzatura on Wikipedia.
Translations
Carlos Marcelo Cabrera translated this article into Spanish: La búsqueda de la excelencia en la programación.
