Please note that this article is now obsolete.
The performance of Ruby on Rails is influenced by many factors, particularly the configuration of your deployment server(s). However the application code can make a big difference and determine whether your site is slow or highly responsive. This short article is about some of the tips and best coding practices to improve performances in Rails only, and won’t attempt to cover the server configuration improvements for the various deployments options.
- Optimize your Ruby code: this may seem obvious, but a Rails application is essentially ruby code that will have to be run. Make sure your code is efficient from a Ruby standpoint. Take a look at your code and ask yourself if some refactoring is in order, keeping in mind performance considerations and algorithmic efficiency. Profiling tools are, of course, very helpful in identifying slow code, but the following are some general considerations (some of them may appear admittedly obvious to you):
- When available use the built-in classes and methods, rather than rolling your own;
- Use Regular Expressions rather than costly loops, when you need to parse and process all but the smallest text;
- Use Libxml rather than the slower REXML if you are processing XML documents;
- Sometimes you may want to trade off just a bit of elegance and abstraction for speed (e.g. define_method and yield can be costly);
- The best way to resolve slow loops, is to remove them if possible. Not always, but in a few cases you can avoid loops by restructuring your code;
- Simplify and reduce nested if/unless as much as you can and remember that the operator ||= is your friend;
- Hashes are expensive data structures. Consider storing the value for a given key in a local variable if you need to recall the value a few times. More in general, it’s a good idea to store in a variable (local, instance or class variable) any frequently accessed data structure.
- Caching is good: caching can significantly speed up your application. In particular:
- Cache your models through the plugin acts_as_cached (a pdf presentation is here) or the cached_model gem;
- Use MemCacheStore as Sessions container;
- Cache your pages through fragment cache (take a look at the extended_fragment_cache).
- Use your database to the full extent of the law 🙂: don’t be afraid of using the cool features provided by your database, even if they are not directly supported by Rails and doing so means bypassing ActiveRecord. For example define stored procedures and functions, knowing that you can use them by communicating directly with the database through driver calls, rather than ActiveRecord high level methods. This can hugely improve the performance of a data bound Rails application.
- Finders are great but be careful: finders are very pleasant to use, enable you to write readable code and they don’t require in-depth SQL knowledge. But the nice high level abstraction come with a computational cost. Follow these rules of thumb:
- Retrieve only the information that you need. A lot of execution time can be wasted by running selects for data that is not really needed. When using the various finders make sure to provide the right options to select only the fields required (:select), and if you only need a numbered subset of records from the resultset, opportunely specify a limit (with the :limit and :offset options).
- Don’t kill your database with too many queries, use eager loading of associations through the include option:
# This will generates only one query, # rather than Post.count + 1 queries for post in Post.find(:all, :include => [ :author, :comments ]) # Do something with post end - Avoid dynamic finders like MyModel.find_by_*. While using something like User.find_by_username is very readable and easy, it also can cost you a lot. In fact, ActiveRecord dynamically generates these methods within method_missing and this can be quite slow. In fact, once the method is defined and invoked, the mapping with the model attribute (username in our example) is ultimately achieved through a select query which is built before being sent to the database. Using MyModel.find_by_sql directly, or even MyModel.find, is much more efficient;
- Be sure to use MyModel.find_by_sql whenever you need to run an optimized SQL query. Needless to say, even if the final SQL statement ends up being the same, find_by_sql is more efficient than the equivalent find (no need to build the actual SQL string from the various option passed to the method). If you are building a plugin that needs to be cross-platform though, verify that the SQL queries will run on all Rails supported databases, or just use find instead. In general, using find is more readable and leads to better maintainable code, so before starting to fill your application with find_by_sql, do some profiling and individuate slow queries which may need to be customized and optimized manually.
- Group operations in a transaction: ActiveRecord wraps the creation or update of a record in a single transaction. Multiple inserts will then generate many transactions (one for each insert). Grouping multiple inserts in one single transaction will speed things up.
Insead of:
my_collection.each do |q| Quote.create({:phrase => q}) endUse:
Quote.transaction do my_collection.each do |q| Quote.create({:phrase => q}) end endor for rolling back the whole transaction if any insert fails, use:
Quote.transaction do my_collection.each do |q| quote = Quote.new({:phrase => q}) quote.save! end end - Control your controllers: filters are expensive, don’t abuse them. Also, don’t overuse too many instance variables that are not actually required by your views (they are not light).
- Use HTML for your views: in your view templates don’t overuse helpers. Every time you use form helpers you are introducing an extra step. Do you really need a helper to write the HTML for a link, a textbox or a form for you? (You may even make your designer, who doesn’t know Ruby, happy!)
- Logging: configure your applications so that they log only the information that is absolutely vital to you. Logging is an expensive operation and an inappropriate level (e.g. Logger::DEBUG) can cripple your production application.
- Patch the GC: OK, not really a coding issue, but patching Ruby’s Garbage Collection is strongly advised and will improve the speed of your Ruby and Rails applications significantly.
- A final note:I don’t advocate premature optimization, but if you can, work on your code with these principles in mind (but don’t overdo it either). Last minute changes and tweaks are possible but less desirable than a “performance aware” style of coding. Profile your applications, benchmark them
and have fun experimenting.
Get more stuff like this
Subscribe to my mailing list to receive similar updates about programming.
Thank you for subscribing. Please check your email to confirm your subscription.
Something went wrong.
I have to take issue with most of these tips. Most of them are a lot of work for incremental benefit, many decrease maintainability, and some will affect your stability (e.g., patching the interpreter). Method dispatch is not really that slow. Is there some deployed app you are basing these suggestions on?
I agree with Evan. While you claim not to advocate premature optimization, that’s just what you are doing. “Sometimes you may want to trade off just a bit of elegance and abstraction for speed.” Then it’s just a matter of degree.
Much of your advice consists of saying “don’t use Rails feature X”. If you don’t want to use Rails, then don’t use Rails. I find it poor advice to try to turn Ruby on Rails into Java or PHP.
I think these suggestions are fine and agree with a good number of them. Antonio is simply telling readers about practices that can impact performance on high-volume websites. It doesn’t matter what framework you use – you’re almost assured of needing to sacrifice code elegance for speed if your performance requirements are in the upper percentiles. I’d understand the objection if Antonio had said that all Rails programmers should avoid these features right from the start… but he didn’t say that.
Then the question becomes at what point do you actually need to bother with any of these.
We receive ~12 million pageviews per month and go out of our way to write elegant and readable code at the expense of efficiency.
josh, you are clearly overreacting. He is by no means suggesting that you should stop using Rails. He is just pointing out some tips that might help make things better. But I do agree with the points made about most of these optimizations being “best practices” rather than real optimizations. If MUST measure before optimizing, but it’s good to keep things in the back of your mind as you code, e.g. not using too many ivars.
i think the negative reactions here are, at base, a product of the fact that if you are really in a position to be needing to do these optimizations, you should probably know all these things anyway. put another way: if you’re not aware of most of these issues, maybe you shouldn’t be in charge of optimizing a project that requires this much optimization.
Having said that, I do have some constructive comments:
– The author is dead right about the speed of REXML. REXML (in DOM-style mode, at least) is unforgivably slow if you’re parsing large files (ditto for massive quantities of small ones). If you can guarantee their well-formedness, Libxml is unbelievably fast. If you can’t, consider Hpricot’s XML mode. REXML’s stream parser might be decent, haven’t really used it.
– there’s been some research to indicate using MemCache for sessions when you don’t need it (if your session count is small) might actually be slower than SQLSessionStore.
– the ‘use your database’ comment suggests stored procs but neglects to mention: optimize your indexes. analyze your queries, figure out what keys you’re using for your selections, and add appropriate indexes. this will make a significant difference in a lot of cases.
finally: simplify. Four out of five times I find that inefficiencies in an app are the result of overly complicated design. It might very well be okay to use some crazy finder method like Foo.find_by_bar_and_mother_id_and_age_and_genetic_imperative(),
but it probably shouldn’t be in a loop that executes 1,000 times.
Three cheers for the zen of software development.
-o
Why oh why have we stopped talking about patterns!
Some of the suggestions are common sense (ie- don’t initialize variables you don’t need, besides it makes for ugly code), others are over the top (ie- patching the interpreter).
Your comment about using HTML neglects the fact that there are plugins that add rake tasks to pre-render your views to HTML to gain that optimization only in production mode. There are other plugins to optimize your assets as well (ie- asset_packager).
Using non-standard versions of an interpreter is very unwise. As the language changes and grows you are forced to maintain your patched version as the interpreter evolves or never upgrade.
One thing to keep in mind is that hardware is cheap and if you are driving the kind of traffic that would force you to sacrifice elegance for efficiency then you should look at adding more hardware to your deployment. Using finders and filters are part of what makes Rails so powerful, their cost in performance is justified by the cheapness of hardware and the time saved in development and maintainability of the code.
Leveraging the DB is one optimization suggestion I fully support. DB’s are capable of doing some serious heavy lifting with things like views and stored procedures. If you are writing an application that will always use a specific DB then it makes total sense to leverage the benefits of that DB. However for the sake of maintainable code, it would be wise to keep the custom SQL out of the controller and in your model, with healthy and clear documentation.
Very sensible advice Antoni.
I adore this kind of discussions.
It is always good to remind that the frameworks enable us to work fast but some time don’t produce optimize code. This compromise we made every programming day. Should we think about the speed while we build the application or should we first build than measure and then optimize. Like some philosopher said – it is all meter of equilibrium.
I don’t want to propagate my way of doing things but I feel the need to share some of mine experience in a general meaner. Developing real-time applications is mine domain for the last ten years. Every tick counts. On the other side the systems that we develop are too big with a lot of code, unnumbered classes and far to menu methods. Last time I check the source has approximately 0.5 Gb.
Usually I describe mine position as a flight control, kipping in mind the picture how the system operate and just know where to look.
While developing something you should always kip in mind that request are:
– it should work as supposed to,
– be able to find misbehaviors fast and precise,
– you or somebody else will change this code in a six months time under very big pressure and should be able to understand fast what the class, method or system should work,
– this peace of code could stay forever or could be replaced tomorrow with no implication on a rest of the system.
It is hard to make the right balance to all of these requirements, but finding the perfection is hard by its definition.
Some rules of tomb from experience:
– think twice before you code,
– design patterns are your best friend,
– comment code as much as you could,
– write unit tests for every system you build,
– benchmark you code to find the bottlenecks and deal with them first,
– premature optimization is the root of all evil.
Programming is an engineers discipline – we solve real problems in a real time for the real people and we enjoy doing it.
I am fully aver that I didn’t say anything new, but we oath to remind us from time to time about it.
Sorry but have to agree with Josh and Evan here. Why give advice to adopt workarounds in stead of really solving the problem?
For example ‘don’t use the helpers’. It seems to me the author is bashing the actual concept of helpers. It’s not the concept that’s the problem it’s some of the helpers that generate crappy code. (Many of the javascript helpers).
In my opinion Rails is great but still needs a lot of optimization, but we need to focus on getting Rails to work right, let’s put our energy into that not into work around ‘just to get our project done’.
Thank you everyone for your insightful comments and suggestions, they’re all very much appreciated even when they come in the form of criticism. I’ll try to answer to each of the comments that I’ve read so far.
@evan
Hi Evan, thanks for your comment.
> “Most of them are a lot of work for incremental benefit,”
There is no need to implement all of them or overdo it. I advocate a “performance aware” style of coding but my post wasn’t intended to suggest that a lot of effort should be preemptively done. Keep in mind some general principles when you define and code your database and application, spot slow areas and don’t be afraid to fix them, is the intended message.
> “many decrease maintainability”
You’re probably referring to 1 (about metaprogramming), 4, 6 and 7. All of them can affect performances but you can safely skip them if you feel that maintainability is jeopardized in any way. I mentioned them because, while they are micro improvements, every little bit can help those who require a substantial speed boost. Depending on the application and the usage volume, they may be worth doing or not. That’s why it’s important to measure first, rather than blindly applying each of them (point 10). Often other changes to the application may just provide enough of a boost, without requiring fine tuning at this microscopic level. I don’t think Rails applications are inherently slow.
> “and some will affect your stability (e.g., patching the interpreter).”
I’m not aware of any issues with the patch I indicated. Do you know of any specific problems or are you referring to the possible maintenance headache?
> “Is there some deployed app you are basing these suggestions on?”
Internally we are stress testing a simple Web application. Some of the points that I made had a substantial impact on the load that can be handled by the servers. Some of them were not ultimately included because, while they had an impact, we gave preference to a typical Rails style of programming (plus the app is very simple and very fast anyways). In my article I propose a few possible optimizations, but it’s all about what people actually need when confronted with slow applications. It’s a matter of balance as well.
@josh
Hi Josh, thank you for your comment.
> ““Sometimes you may want to trade off just a bit of elegance and abstraction for speed.” Then it’s just a matter of degree.”
It is a matter of degree. It’s not an XOR situation. That “Just a bit” implies that the code can still be elegant and abstracted. But if through the profiler you find a slow abstraction, you shouldn’t be too worried about changing it to something which is still elegant, readable and maintainable – even if (it is so) to a lesser degree when it speeds up things. It is fundamental to find the real culprit of the slowness rather than apply each tip to your code indiscriminately.
> “Much of your advice consists of saying “don’t use Rails feature X”. If you don’t want to use Rails, then don’t use Rails. I find it poor advice to try to turn Ruby on Rails into Java or PHP.”
I think this is a misinterpretation of my article. That’s not what I meant and I hope it didn’t come across like that for everyone. My suggestion was to spot slow code and fix it by using, at least to some degree, one or more of the points made above.
> “Much of your advice consists of saying “don’t use Rails feature X”.”
“…don’t use Rails feature X” only if that X is the bottleneck and possibly a sufficiently maintainable alternative exists. One could argue that if the framework itself provide suboptimal code (for example, a particularly slow helper or library), then that should be fixed rather than worked around. I tend to agree with this, and I’m not advocating otherwise in principle, but until that improvement happens you may still have to get the job done now for your live application (but still feel free to investigate and work on a patch for Rails, if you can.)
@Tyler
Thank you Tyler. Compromises for performance optimization are NOT always required, but if you find your application too slow, it’s better to individuate the hotspots and solve them before you start to Throw Hardware at It (TM), in my opinion.
@PJ Hyett
Hi PJ, thanks for your comment.
> “Then the question becomes at what point do you actually need to bother with any of these.”
When an application is running on a well configured and optimized server and it’s still too slow.
In that case there are two options available: add more hardware, or try to optimize the application (or do both, depending on the case). It’s fine to leave your application as it is and add more physical resources, but it’s not always an option, and some people prefer the philosophy of optimizing the application before spending extra money. I tend to agree with this philosophy if it doesn’t compromise my application. Also, not all the advice provided significantly affects readability or maintenability, so you can still selectively adopt some of the suggestions without including micro-optimizations that you may dislike because of “elegance needs”. (By the way, I imagine you are referring to http://www.chowhound.com, which seems like a very nice site to me. Congratulations on its success and popularity.)
@Mazdak
Thanks Mazdak. As you said, I’m not advocating the transformation of a Rails application to a PHP level in any way. Profiling before applying certain tweaks is essential, there is no point in sacrificing elegance for the sake of it. A find_by_username called twice in the application, won’t affect performance enough to require a rewrite with a find_by_sql. But if this gets called many times in an application or in a plugin, and a profiler shows that quite a bit of time is spent on these methods, then I’d say there is nothing wrong with optimizing it (perhaps using a simple find instead). The overall notion of not doing premature optimization is an important point of the article, and I put it last as a general – but pertinent – reminder. However things like using stored procedures, indexes, adopting good Ruby libraries, or algorithmically efficient Ruby code is something that should be done from the beginning. “It’s good to keep things in the back of your mind as you code,” as you well put it.
@big o
I didn’t explicitly mention indexes, but they are rightfully part of the “Use your database to the full extent of the law” point. I agree with you in regards to their importance and also agree with your “simplify” point. To quote Einstein: “Everything should be made as simple as possible, but not simpler.”. You expressed very well what I meant in regards to finders when you said: “It might very well be okay to use some crazy finder method like Foo.find_by_bar_and_mother_…, but it probably shouldn’t be in a loop that executes 1,000 times.”. I appreciate your comment.
I agree with evan and josh; many of these tips are premature, sacrificing elegance and maintainability for very marginal gains in performance. If you require such raw speed, I’d suggest ruby isn’t the language to achieve it.
One specific point: if using dynamic finders really is an issue in an application (and I don’t really believe it would be), rather than changing all calls to find_by_foo_and_bar to find_by_sql, why not define a concrete find_by_foo_and_bar method on your model?
Item #10 should be #1, as it is by far the most important tip. The worst case is premature optimization, concentrate on simple elegant code first. Once you see you need more speed, profile and benchmark your app to find the bottlenecks, only then should you start optimizing.
I disagree with Josh & Evan. The writer’s 10th point specify the intended usage for the other points. Good suggestions and some overreactions in an interesting thread!
Note: I’m a different Josh
I think Patrick brings up a good point. If you are writing a guide for “performance aware” style of coding, your first point should be about keeping your code simple and elegant.
Optimizing code isn’t the same thing as creating simple and elegant designs from the start.
Thanks Antonio for the tips! While I do not 100% agree with them all, I appreciate the different perspective and experiences. We should all realize that this is not meant to be a definitive guide, it is just one perspective. How about starting with saying with what you agree with and not start out by attacking. Keep it positive!
If you think you really do know more or can make a better guide, please write one and then post a link.
@Mark Carey: “Hardware is cheap”
While this is somewhat a true statement I feel that it misses a vital point – “money is money”. You mention that the development and maintenance (through nice and elegant code) time saved will justify the amount of money you will spend on new hardware. Yet, money is money and at the end of the day it makes sense (to the economists in all of use) to maximize the amount of money you put in your pocket (or minimize the amount you lose).
Thus, if I can save $10k on hardware by following a couple of Antonio’s suggestions it would be a wise move as long as the associated development and maintenance costs (in coding time) of the code change are less then $10k.
My main point is that sometimes it is a good idea to keep the code elegant even if it means throwing more hardware at your project, while other times uglier optimized code can be good! As I always say, “It’s all about balance” and each specific project has its own unique balancing act!
@Others
Finally, I think it is important to take all of Antonio’s tips as tips and NOT rules.
@Antonio
Thanks for the tips Antonio!
I agree with Bradley regarding hardware, also i’d like to add: who’s gonna mantain the extra hardware you throw at your app?
I’m a one man show (on my Django projects at night!) and when i build an app i have to do everything A-Z, i’d rather not maintain an extra server or have to hire somebody to maintain a proper “server farm” if i can avoid it by optimizing code. Rails and similar technologies allow us to compete against bigger teams of .NET/Java programmers.
I have a fulltime job during the day and here where i work we use SQLServer basically as we would be using MySQL with MyISAM tables: no FK constraints (they slow things up!!), only stored procedures are allowed, nolock hints _everywhere_ which gives the risk of reading uncommitted transactions.
And i’m talking of one of the top 50 most visited Internet sites and we certainly don’t lack money or personnell or talented people.
As i said, sometimes more machines are more headaches.
You can throw out this entire post and replace it by 3 things:
1. Measure
2. Measure
3. Measure
Rich what you said is hardly true. What do you do after your measured? This post helps answering that question.
Antonio,
Wow! Lot’s of action on this thread. Thanks for posting your thoughts.
Others,
I don’t think Antonio is advocating WHEN to optimize, but rather showing some techniques needed IF you have to optimize.
RoR is a great framework. Use it to it’s full extent. The very fact that you are capable of using these techniques for optimization is a nod to the flexibility of RoR. If you really need to use a particular technique, use it. Don’t be afraid of the elegance vs. peformance monster.
Do what it takes for you to be happy with your app! Isn’t that what RoR is all about? Happiness?
Thanks again Antonio 🙂
thanks.
keep up the good work!
Well, what must also be mentioned is that when it comes to enterprise-size applications where optimizations of these sort are needed to handle the traffic, scalability is order of magnitudes more important than performance. So probably it is more worth the effort to make sure your application will benefit from an extra webserver, an extra database server, more memory or so on.
Often a overly optimized application is harder to scale linearly.
You may also want to look at the information in
http://www.infoq.com/articles/Rails-Performance
To help understand how to tweak rails performance 🙂 It seems very good on reporting some real world situations on it.
-Roger
I would add:
Try to avoid URL_FOR to avoid url look up.