I'm glad that Twitter is working to resolve its scaling issues. It's a service that I love and use on a daily basis and from which I have benefitted immensely. As far as I'm concerned, Twitter is a case-study in how Ruby on Rails does scale, even in their hands. Also, I know Alex Payne in person. Not very well, but enough that if I was in San Francisco and sat down to share a beer and discuss the Ruby/Scala debate, I imagine that it would be a fairly civil conversation.
My interest in the question of Ruby vs. Scala at Twitter had mostly consisted of curiosity and amusement, at least until last night. First I had an exchange of tweets with Alex which raised my warning levels, and second Alex wasted no time in creating for himself a virtual nintendo invincibility star for himself with this blog post: Mending The Bitter Absence of Reasoned Technical Discussion.
Who can disagree with the need for more reasoned technical discussion? Nobody, but frankly the timing of his essay made me throw up a little in my mouth. The nausea only grew this morning when I checked on how much it's being re-tweeted. Even though Alex's star is probably going to take a little while to wear off, here are some of the facts of this "debate" for your consideration and historical record:
First of all, my exchange with Alex last night, which I triggered by re-tweeting Jeremy McAnally:
RT @jm: If you have to use kind_of? all over your code to mimick a "type system," you're doing it wrong. (I had exactly the same thought!!!)
To which Alex responded:
@obie Indeed, using kind_of? in that way is "doing it wrong" in Ruby. But as our codebase grew, it became a necessity to combat bugs.
Hmm... A "necessity to combat bugs". What bugs, exactly? Bugs in Ruby or bugs in Twitter's codebase? Non-deterministic bugs? At this point I wasn't angry or anything, just curious and a little bit suspicious. So I asked:
@al3x Doesn't make sense. Do you mean non-deterministic bugs? An in-depth explanation of the bugs you're talking about would be enlightening
To which Alex quickly replied:
@obie Yes, I mean non-deterministic bugs in the giant, legacy, spaghetti parts of our system. Unexpected objects flying around.
Ah, suspicions confirmed. Alex is indeed scapegoating Ruby for Twitter's shitty legacy codebase and the mistakes that they made during what is unarguably a very steep growth curve that would have challenged even the most brilliant team of developers. But don't take my word for it, here are some direct quotes from Alex himself in the recent Artima interview:
Twitter continues today to be primarily a Rails application, with a bunch of Ruby daemons doing asynchronous processing on the backend.
Over time we found that although Rails works great for doing front-end web development, for doing heavy weight back-end processing, Rails had some performance limitations at runtime. And I think that—and this is more my personal opinion—the Ruby language lacks some things that contribute to reliable, high performance code, which is something we’re very interested in as we’re growing as a business. We want the code we write to be correct and maintainable. We want to keep our costs down—all the things most businesses want out of their stack. So that’s why we started looking at Scala.
Most damning:
I’d definitely want to hammer home what Steve said about typing. As our system has grown, a lot of the logic in our Ruby system sort of replicates a type system, either in our unit tests or as validations on models. I think it may just be a property of large systems in dynamic languages, that eventually you end up rewriting your own type system, and you sort of do it badly. You’re checking for null values all over the place. There’s lots of calls to Ruby’s kind_of? method, which asks, “Is this a kind of User object? Because that’s what we’re expecting. If we don’t get that, this is going to explode.” It is a shame to have to write all that when there is a solution that has existed in the world of programming languages for decades now.
Adding fuel to the fire, Alex seems to have created quite a negative impression about Ruby and Rails with his recent Web 2.0 talk in which he advocated usage of Scala. Tell me the impression you get from coverage of his talk in articles such as this one in MIT's Technology Review, which makes the following incendiary assertions:
The company is leaving behind a programming language that has caused it much pain in the past, and instead embracing a new and somewhat obscure language called Scala.
[...]
Right now Twitter's service is a hybrid of programming languages, Payne says. The user interface runs on Ruby on Rails, which is "fine for people clicking around Web pages," he says. But by the end of the year, Twitter hopes to have a set of services in the back end that are written entirely in Scala. And it's the company's plan to make sure that all the third-party services that connect to Twitter via the application programming interface (API) go through Scala code, bypassing Ruby on Rails completely. "When you're talking about a bunch of programs hitting the API rapidly," Payne says, "We found we can better optimize things...using Scala."
Did you actually say "fine for people clicking around Web pages", Alex? Maybe that Technology Review article is just sloppy journalism, a subject that Alex himself lamented not too long ago. But I don't think that's the case. I think that Alex is slyly pushing his Scala agenda forward, yet acting surprised when Ruby folks call him out for it.
What agenda? His agenda as an advocate of Scala and author O'Reilly's Programming Scala book. He said so himself in Recession Engineering:
I’m first to admit that I have something of an agenda with this prediction. I spend a fair bit of my time working with and writing about the Scala programming language, which provides the expressiveness and flexibility of dynamic and functional languages like Ruby, Python, and Lisp with the performance of C++ or Java. - Alex Payne
Have I been guilty of this stuff in my own past as a Rails advocate with a strong agenda? Yeah, probably so. Which is why I know it when I see it. :)
