Comments on: Benchmarking Tornado vs. Twisted Web vs. Tornado on Twisted

By: Nicholas Piël » Socket Benchmark of Asynchronous Servers in Python

Nicholas Piël » Socket Benchmark of Asynchronous Servers in Python — Tue, 22 Dec 2009 08:36:22 +0000

[…] async. approach called Tornado which is also using callbacks and recent benchmark show that it outperforms Twisted.A common heard argument against programming with callbacks is that it can get overly […]

By: samuel Sutch

samuel Sutch — Thu, 17 Sep 2009 09:59:05 +0000

is cogen competitive?

By: david

david — Tue, 15 Sep 2009 06:56:03 +0000

what? I wanted to see EM benchmarks… could you make another post comparing EM with Tornado please?

By: Greg

Greg — Mon, 14 Sep 2009 18:07:14 +0000

What command-line options did you use with httperf? I’d like to try this test against various Ruby async http servers.

By: Antonio Cangiano

Antonio Cangiano — Mon, 14 Sep 2009 06:46:23 +0000

@Eugeny: The updated post ensures that all three web servers use epoll().

@Taylor: Yes, that’s a good idea.

By: Taylor Luk

Taylor Luk — Mon, 14 Sep 2009 06:39:21 +0000

@antonio Thanks for this very interesting benchmark. I see that you have pulled down the EventMachine chart. I am aware that this post is focused on evented python servers. However, EventMachine and Twisted Python are both Evented framework for respective language. look forward more content on EventMachine since that EM server is pushing out some really nice numbers, perhaps a seperate post ?

By: Eugueny Kontsevoy

Eugueny Kontsevoy — Mon, 14 Sep 2009 05:41:07 +0000

Anotonio, thanks for the benchmark. But something is just not right here. There is no way one “framework” can beat another by so much. When it comes to massively parallel asynchronous I/O the bulk of the work is done by the kernel i.e. epoll() on Linux.

There are really two things going on:

a) I/O implementation (all of it is in C – for all your contenders)
b) Data marshaling b/w Ruby/Python and C

I just don’t see how one framework can be more performant than another. The differences you’re seeing can only be attributed to:

a) Richness of implementation, i.e. do all these “Hello world” apps do full HTTP headers parsing and forming “request objects”? How about initializing/maintaining other aspects of maintaining a request state? I doubt we’re looking at oranges-to-oranges here.

b) Error of configuration: did you make sure all of them rely on epoll()? This isn’t as trivial as it may appear since I/O libraries have a habit of falling back to slower select() or poll() silently.

By: Antonio Cangiano

Antonio Cangiano — Mon, 14 Sep 2009 03:11:07 +0000

@Bob: Indeed. I’m re-running all the benchmarks with httperf and will update this post very soon. Stay tuned.

By: Bob Aman

Bob Aman — Mon, 14 Sep 2009 02:40:43 +0000

httperf is a better tool if you really care about useful statistics.

By: Antonio Cangiano

Antonio Cangiano — Mon, 14 Sep 2009 01:13:51 +0000

Thanks for stopping by Glyph. No matter how trivial the benchmark, I’m interested in its accuracy. If ab cannot be trusted, I’ll run the tests with httperf or siege and update my results.