February 28th, 2006

  • evan

efficiency at scale

Something on my mind recently*: when I decide to write my apps in Ruby, I do so knowing that I'm throwing away some CPU performance and I'm ok with that. (According to microbenchmarks, the performance impact is over 10x relative to C++.) We like to say programmers are more expensive than computers, and so we're willing to waste computer time to gain programmer time. I agree with that in general, but there's an interesting exception.

Suppose your CPU-bound webapp is written in a language that is only half the speed of C++ (this hypothetical language is pretty zippy, compared to what's popular today). That means you'll need to buy two web servers where a C++ implementation would need one. That's fine; you really need your pet feature -- skipping compilation, duck typing, garbage collection, whatever. But as your app grows more popular** you grow to a larger cluster, a hundred machines, and suddenly you're throwing away fifty of them -- a whole rack dedicated to your programming language!

By this reasoning, no matter how fast your language is, as you scale up in machines it'll always eventually become more cost-effective to write it in a faster language. Consider search engines, with clusters of thousands of machines! For this reason I don't expect to be giving up C++ for a long time***.

What's especially weird to me about this is that it doesn't really apply to desktop applications, because desktops are so overpowered. Typically you can change some data structures around or sacrifice some memory to make a slow desktop app into a less-slow one. Instead, the distinguishing factor is sharing: these efficiency concerns come into play as soon as you can make efficient use of your computing resources -- time-sharing multiple users of your web app on one machine.

(A contrary point that I'll preemptively respond to is that webapps aren't typically CPU-bound, or that the bottleneck is elsewhere. Both are reasonable objections, but they don't apply in the sort of work I'm involved in.)

* And liberally borrowed from a friend's blog post, hope he doesn't mind!
** LiveJournal always comes to mind when I think of popular -- last I heard the machine count was somewhere around a hundred.
*** This doesn't make me enjoy Haskell any less.