About Rails and Ghettos

This is a post that has been a couple weeks in the making (in my head), but the more that time passes by the less interested I am in writing it. Nevertheless, there are a couple of points I want to get out there both for the record and for possible discussion.

First of all, it is obvious that Zed was using that ugly word in its current derogatory sense, and I'll keep that sense in my writing as well. According to the dictionary one of the definitions of ghetto is "an isolated group" or "a situation that resembles a ghetto especially in conferring inferior status or limiting opportunity".

In the greater realm of IT?

Does that definition ring true of Rails in the greater realm of Information Technology, or even just the programming world? I don't think so. The Rails community is not an isolated group, having members across a wide, diverse, range of enterprises and groups of individuals, from the highest-paid professional consultants, to the humblest program-for-fun enthusiasts.

What about inferior status, or opportunity limitations? That doesn't ring true either. If anything, the mainstream fascination with dynamic languages was ignited by Ruby on Rails a couple of years ago, which gives popular Rails practitioners a disproportionate amount of influence (ahem) in the programming world, and a prominent voice in online conversations about the future of our craft.

From my own (admittedly-biased) viewpoint, Rails is continuing to explode in popularity. Among other things, my book The Rails Way is selling extremely well, and we just had to do a rush reprint because demand exceeded even our optimistic estimates. It's currently a top-seller on Safari (a good indicator of traditional corporate developer interest levels), and has been hovering in the hundreds ranking on all of Amazon.com.

But where are the jobs?

To those that would use the number of Rails-related listings on major job boards as evidence of opportunity limitations, I point out that due to extraordinary demand for Ruby talent in the marketplace, it is almost pointless to advertise Ruby job positions via conventional channels. The way to find Rails talent is via targeted Ruby job boards, word of mouth, and blogging. Counts on mainstream job boards are meaningless at best, misleading at worst.

Most Rails developers worth their salt are contracting, simply because it is very lucrative right now. Rates for premium Rails talent is now in the neighborhood of $200/hour and continuing to climb. Average Rails talent costs at least $100/hour.  The work is finding them, not vice-versa. Why are clients willing to pay so much? Because they know we are a cost-effective bunch, doing the same amount of work (as individuals) that would take multiple traditional programmers in other mainstream languages, and with higher-quality as the  "icing on the cake".

How about just in the Ruby community?

Considering Zed strictly within the confines of the Ruby community, and not in the much bigger sphere of programming, I think he's still wrong. Like a newbie OO programmer violating the rule prefer composition over inheritance, he made an is-a/has-a mistake. What is true perhaps, metaphorically, is that Rails has ghettos, and Zed made the mistake of attributing qualities of a ghetto to the whole.

What I"m trying to say is that there are groups of people that have come over to work in Rails, the city of Rails if you will, who have settled into neighborhoods that can be very dangerous (monetarily) to inexperienced clients.

But why can't Zed get a job then?

What put me off about Zed's rant was not so much the content, but rather that it achieved such wide circulation. It wasn't meant to be read so widely, and I don't believe that he had destructive intentions when he wrote it. I lived and worked with Zed for five months during 2007, a time in which we became (and remain) good friends. I swear to you that Zed is a good, friendly guy, in person. (His created, over-the-top, narcissist online persona is another story!) The rant was filled with tons of inside jokes and commentary that really only make sense to other clued-in Ruby people, stuff that comes across as just mean, stupid or crazy to everyone else.

For instance, a significant percentage of readers obviously understood from the rant that Zed is unemployed, starving, and/or stark-raving mad. All three could not be further from the truth. Zed is happily employed by a major, household-name financial institution in New York City, where he comes into work everyday with a nice shirt and tie, and leads a Rails (yes, Rails) development team. Of course, none of this is news to those of us that really know him, but the rest of the world got a totally different story.

Comments

[Michal]

In my opinion, Zed started very important discussion about Rails position in the IT market.

Rails can't explode in popularity without reliable platform for a cheap shared hosting (like Apache with mod_php). I think this the main problem for now. In Europe it's harder to find hosting plans for one small project only. I guess that there is a lot of IT managers in a small companies, who afraid words like: "cluster", "dedicated server", "performance problems".

Your book is truly great. You should think about changing the title for "The Rails Bible" ;)

[Ryan Allen]

Damn it, Rails doesn't need to run in a shared environment. Companies who are afraid of the words cluster and dedicated server need a viable business plan so they can pay for these things. And they're not even expensive these days!

Don't listen to Dreamhost, they're blowing smoke. Rails (and other Ruby web dev options) are perfectly fine deploying as is. Shared hosting is stuff of the past, get over it.

[Karmen Blake]

Thanks for the insights about Zed and his comments. BTW, I read through your book and have picked up a lot of guiding principles for Rails. I've got many dog eared pages I refer back to.

[RJ]

Ryan Allen,

From the standpoint of an admin, I can guarantee you that Rails is not nearly as server-friendly as PHP. With mod_php, servers can host multiple applications with no more configuration than uploading the files. Rails not only requires extensive configuration, a separate instance needs to be created for every application. Hopefully JRuby will create a compromise where Rails apps can be deployed like Java apps.

In the mean time, most web sites are no where near a point where they need more than shared hosting. For those groups that do, it's just as easy to expand one's infrastructure with a PHP-like setup as it is with Rails.

From a more philosophical point, it seems contradictory to me that a framework that emphasizes reducing the work on the programmer would not apply that same philosophy to all areas. But that's not really my focus here.

[Justin]

I think we all know that ZS is gainfully employed and still using rails.

Likewise we can all see the rails community taking on the same characteristics of any successful product.

ZS may pick individuals, but you have to admit that one a product reaches a critical mass the general discussion tend to revolve around zealots rather than progressives.

ZS was ranting and he made that clear, I am not sure what's wrong with that

[Paolo Bonzini]

@Ryan Allen: if you find me a non-shared hosting that I can pay 30 bucks a year, I will agree that "shared hosting is a thing of the past".

[Robert]

"To those that would use the number of Rails-related listings on major job boards as evidence of opportunity limitations, I point out that due to extraordinary demand for Ruby talent in the marketplace, it is almost pointless to advertise Ruby job positions via conventional channels. The way to find Rails talent is via targeted Ruby job boards, word of mouth, and blogging. Counts on mainstream job boards are meaningless at best, misleading at worst."

So what you are saying is since there is no proof of Rails popularity just your saying so makes it so? Or maybe the Rails community is so small in the scheme of things and the market for Rails is so small they all get snatched up by the current Rails developers? That is equally plausible without needing any proof either. : )

[Daniel Reeders]

Zed's rant was a shotgun blast, and your is/has analysis of the word ghetto is an argument about a single pellet in it. I think a better word for what Zed describes is cabal - an in-group with associated reality distortion field. There's only a single fact in this entire post, and that's about Zed's current employment status, which in my view only adds to the power of his argument.

[Obie Fernandez]

@Daniel: His rant may have been a shotgun blast, but it was titled "Rails is a ghetto", and that's the assertion that I wanted to challenge. I fail to see how Zed's employment status adds to the power of his argument. I mean, which argument? That Rails is a ghetto? There were probably dozens of arguments in the overall rant, only some of which I disagreed with.

[Erik Peterson]

It is perfectly feasible to have gainful employment even in the worst ghettos. But that is stretching the metaphor a little too thin.

I think that Zed was mostly right in that there are some people in the Rails community that we should be wary of. We have a habit of taking the words of certain people for gospel, and not really challenging them. I think that will definitely change now, and the community will be better for it.

Zed's rant achieved such high circulation exactly because of that property. Zed was one of the people whose word we never really challenged. By making us challenge his word, a lot of people in the Rails community (myself included) looked upon everyone in a completely different light. The kind of essay/rant that makes you look at your environment differently is definitely rare, and fed its popularity significantly

[Zed A. Shaw]

Yep, 1/2 of the fun of my blog is fucking with people who take the internet way too seriously. I do work at a company doing Rails, but let's qualify some things.

First, it's a JRuby project doing Rails and a *bunch* of other technology. Rails is a tiny little piece once you include Java, C code, systems work, and deployment. After working on this project, I'd actually have not gone Rails at all because of the heavy Java requirement it had. JRuby is a champ though so similar folks should go that route if they want to sneak Rails in.

Second, I don't go to work happy or sad. It's just work. It's annoying and stupid and I don't work on anything interesting all day. The interesting ideas are more or less flushed down a toilet wrapped in bureaucracy. I track my time by the hour in a shitty time tracker and get angry emails if it's not accurate to the hour. Every decision is questioned. Every idea that doesn't involve a commercial product is considered stupid or dangerous. Network operators shutdown our network anytime we do things like run a samba server. If some clueless manager buys a piece of software we have to use it, even if it's crap. We have project managers who come by periodically through the day demanding status on things and not really telling us who's interested. It blows.

But, the job pays _very_ well, and compared to slaving away trying to find a few decent clients to pay for my expensive ass NYC apartment it's great. I don't worry about some shithead CEO blowing the money for my Net-30 invoice on coke and strippers. I don't have to put up with companies desperate to make money trying contract me into giving them all my life's work.

I just show up, teach people how to code, and go home.

Would I rather be working on my ideas and doing a business of some sort? Sure. But then nobody is interested in those ideas because, well, I *was* the fucking Ruby guy. Morons only wanted me to code their stupid ideas in Rails and STFU.

So, I wear a tie except on Fridays and don't consider going to work a happy or sad thing, just a necessary evil of trying to survive until Web 2.0 implodes and people are desperate to put their money in an few ideas don't suck.

[Mark Thomas]

There seem to be a lot of people clamoring for bad news about Rails. I think the whole Rails phenomenon scares people who would rather stay in their comfort bubble believing that $LANGUAGE_THEY_USE will always be the best for everything. They seem eager to discredit Rails. I'm the guy who knows Rails in an organization eerily similar to the one Zed describes above, and I've been emailed two posts more than any other: the "Why I switched from Rails back to PHP" post and now Zed's rant. The only real response that seems to work is to educate--"yeah, Rails isn't for everyone, but let me show you some cool stuff and how easy it is to do". A couple have now asked for my help getting their feet wet with Rails, another is learning Django. Equally good results, in my opinion.

[cap2501]

"If anything, the mainstream fascination with dynamic languages was ignited by Ruby on Rails a couple of years ago"

I would have to disagree with that. No matter what your current opinion of the language is PHP really kicked off the mainstream facination with dynamic languages. While there were plenty before it and I'm sure there will be plenty after it. It started the current mainstream dynamic trend.

[AT]

Obie,
I'm not sure I agree with you about the demand for Rails talent. I've been working with C++ and Java for 13 years and doing Rails on the side for about 1.5 years (I have a small consulting business). In the last 2 years I managed to land only 2 contracts in which Rails was in the picture. I've been looking and looking through my network and through job boards, its just not out there in a noticeable way (I'm in Toronto).

Also, even when I hear about rails demand, buck for buck, Java contracts pay much more. I love rails and think it has a lot of promise, but I need to pay the bills.

Maybe I'm hanging out with the wrong people but that in itself is proof that rails is still not mainstream enough to be profitable for the individual contractor/consultant in a sustainable way.

Nevertheless, I would be delighted to be proven wrong.

[smith]

Obie,

The reason I posted anon was simply that if you "dis" the rails community you get punished. It's like "dissing" muslims - you better be scared, case in point all the "religous" sounding comments around the blogosphere.

If you don't want to post the message thats fine but please at least address some of the points. Why is it that rails developers tend to be single-minded, and most couldn't write a program in prolog or lisp program to save themselves?

Its because its a fanboy culture rather than an engineer/scientific culture. Its part of the reason zed didn't fit in that I can see.

Opinionated software is all good but opinions without evidence are anecdote: and that anecdote has been a sham (400 restarts a day? wtf?)

[Obie Fernandez]

@smith: Define "punished" in this context. Do you mean getting strongly-worded replies? That's not being punished in my value system.

The Rails world has a super-wide variety of talent levels operating within it. Most of the Rails people that I talk to on a regular basis are on the stronger side of the spectrum. I don't know what being able to program in prolog or lisp has to do with anything, especially if we broaden the conversation to all Ruby programmers, who are probably *more* likely to know lisp than any other mainstream programmer.

I know this is orthogonal, but believe me, Zed has a legion of fanboys and he loves the attention.

Finally, the "400 restarts" meme is pure, unadulterated FUD. The claim was clarified even by Zed himself, but not until it established itself alongside other myths such as "Rails doesn't scale". As I'm typing this, I'm watching the production.log on one of my apps scroll by registering performance figures averaging 500-1000/requests per second. That doesn't include page-cached actions, which are served directly by Nginx an order of magnitude faster. Oh yeah, that's running on Amazon EC2 and doesn't need to restarted. Ever.

Ball's in your court.

[smith]

Obie,

Good reply - I didn't expect I would get one. Props.

-- Define "punished" in this context. Do you mean getting strongly-worded replies? That's not
being punished in my value system.

Well, in my book it is making a claim without evidence, but using your "cred", "popularity" or "coolness" to force your point. Its what we expect of fundamentalists. As C.S's we need evidence not anecdote... this is science after all.

All I say is lower the bar and you let stupid through the door... ROR makes it so simple that non programmers can build basic apps. Apps do not an engineer make.

-- Finally, the "400 restarts" meme is pure, unadulterated FUD.
Ok... still has some relevance though, care to quote what really happened? Considering a newbie NEVER has to restart a java app server except for system maintenance/upgrades, anything more than 1 restart every 30 days is unacceptable.

-- The claim was clarified even by Zed himself, but not until it established itself alongside other myths such as "Rails doesn't scale".
One question: Twitter vs facebook - which handles volume better?

-- As I'm typing this, I'm watching the production.log on one of my apps scroll by registering performance figures averaging 500-1000/requests per second.

Want to post some logs for proof? Would be a good way to dispel some myths. C1K is what a single tomcat instance can do trivially, with no restarts out of the box.

-- That doesn't include page-cached actions, which are served directly by Nginx an order of magnitude faster. Oh yeah, that's running on Amazon EC2 and doesn't need to restarted. Ever.

Umm... static content aint that hard! What about if your instance goes down (we use ec2 as well).

We had the complete opposite experience to you. granted, we weren't rails committers and haven't written a book on the platform, but I consider we were very fluent in ruby. We created 5 internal rails engines, an application framework overtop of rails, a forms framework similar to some of the newer ones etc etc... we were cranking out apps and had 5 in production... they were costing us to much to maintain (c.f. java apps) that we rewrote them all.

If you need to have written a book about ruby to make it work, or have tons of money like the twitter guys (and even they cant!!) then how is anyone out of the inner circle supposed to do it?

ROR was nice to program in (except for the stupid number of unit tests needed) but deployment sucked. Id much rather use any of the frameworks for php, python, scala, groovy, wicket/gwt or even lisp (personal fav).

Convince me to come back. Prove that deployment doesn't suck, and that you don't need to be a guru to make it work!

[Luke]

I'm not sure what people don't understand about the way that Rails is developed. The people pushing Rails forward are people who look at a problem like (to quote smith) "ROR was nice to program in ... but deployment sucked." and say: I should contribute some code to make deployment easier.

None of the people that are consistent contributors to Rails (AFAIK) are being paid to sit around and fix the "problems" with it that other people bitch about. They use it, every day, for their own businesses, and fix the parts of it that are broken for them.

If a part is broken for you, FIX IT.

I think the real problem with the Rails community is not so much a lack of responsibility on the part of the contributors as a misguided sense of entitlement that everyone else seems to have.

Use it or don't. If you use it, and it doesn't do what you want, find something that does, or write something that does and share it with everyone else. Quit whining about someone else not doing the hard work for you.

(P.S. I am not a core Rails contributor... but I have never once whined about some part of it sucking.)

[smith]

http://ejohn.org/blog/programming-book-profits/

This post basically entirely debunks the usage of book sales as language growth indicator. The vast majority of technical books sell less than 10K books. Thats a pittance, and I am really surprised. Hell there are probably 10,000 libraries! WTF?

Rather than quoting book percentage growth (percentages lie through the teeth). Its like saying your user count has grown 5000% (from 1 to 50 users). What are the numbers of your book sales?

[John]

Smith: Noone is keeping you from programming your brochureware engines in Prolog. From what I hear, deployment of Prolog web apps are especially simple and painless. If there werent so many damn books out on Rails, I would definitely be spending my time building the next Facebook on a Prolog framework. Oh well, maybe soon the publishers will realize that their continuing deluge of books about Rails and Ruby is entirely unprofitable and we can all get back to writing our web apps in *real* languages like C.

[Brian Adkins]

I must be missing something with respect to Rails deployment sucking. I have a handful of apps in production on a dedicated server using Apache, mod_proxy_balancer, Mongrel and Linux, and it just hasn't been a problem. Linux doesn't go down. Apache doesn't go down. Mongrel doesn't go down unless I push a new release.

Pushing out a new release is as simple as:

1) rsync the code to a specific directory e.g. my_app-v2
2) mongrel_rails cluster::stop -C /.../my_app.conf
3) rm my_app
4) ln -s my_app-v2 my_app
5) mongrel_rails cluster::start -C /.../my_app.conf

If there's a problem, just switch the links back and recycle.

(Yeah, I should probably be using capistrano, but it only takes me about a minute, so I haven't been motivated to read up on capistrano yet.)

Is this what everyone is complaining about being difficult? Sure, it may not be as easy as plopping a .php file in a directory, but it's hardly worth commenting on IMO.

[Aaron Evans]

Are you afraid to say where you work? Then why are you (or he) afraid to say where he works?