The Dark Side Beckons?
Lord Ries by Cody San
For years I've advocated a high-ceremony, test-first development methodology where use of BDD (or TDD) is sacrosanct and uncovered production code is a sin. Then I got to know Eric Ries and studied his Lean Startup stuff and got started with my own lean startup venture and over time...
Something's happening. I'm not the Agilist I should be. I want more. But I know I shouldn't.
I no longer believe in high-ceremony development practices for early-stage startups. Actually, I'm pretty sure that they're poisonous for most web startups, hence my lightning talk at Railsconf 2011. Until you are able to prove that you have a viable market, that customers will give you money for your product, you shouldn't be sinking a lot of time and money into implementation. Assuming you are a competent Rails developer, you should be able to hack out features and take on quite a bit of technical debt to build throwaway proof-of-concept versions of your product. I'm pretty much advocating construction of large "spike solutions" except that unlike the traditional XP definition, the problem you're trying to solve is the problem of finding your market fit. It has nothing to do with the technology.
Also, we are commonly told that spikes (aka prototype systems) are not meant for production and should be thrown away. I think that advice is very dependent on the skills of the people writing the prototype system. If they are experienced developers, they're going to inherently know a lot of the application patterns that should be used and the amount of technical debt introduced will be low enough to feel comfortable with the app in production.
- I'm specifically talking about typical web-based applications built with Rails where nobody is going to die if you occasionally get a 500 error on an edge case. Use continuous deployment so that you're comfortable that you can turn around code improvements and bug fixes almost immediately.
- Use TDD to "discover" the design of parts of the system that are unfamiliar to you. This primarily applies to unit testing
- By all means have some integration-level specs that test the "happy path" through your system so that you don't inadvertently break your entire app. AKA "smoke tests"
Early on in the startup process, it's much more important to be testing against business metrics than anything having to do with code. One of the most thought-provoking pieces of advice I've heard lately came from Hiten Shah of KISSmetrics who told the latest LSM NYC crowd to consider the effects of a feature of key business metrics to be the acceptance criteria for that feature.
In other words, before you implement a feature, you should have an assumption that tells you what the effect is going to be: higher click-through, engagement, satisfaction ratings, etc. This results in an acceptance test that is time-shifted; you won't know the results until you deploy. I believe this is fertile ground for technical innovation. While there are plenty of frameworks that simplify AB testing and real-time monitoring, I don't believe there exists anything along the lines of Cucumber that involves developers in cultivating a mindset of provable business value. David Chelimsky and I joked around that perhaps such a tool should be called Squash.
One of the natural questions that arises from this line of thinking is: When am I no longer an early-stage startup? The implication being that at some point, you know enough about your market fit and product requirements that you can transition into what I've been calling high-ceremony development practices.
I don't have the answer to that question. I suspect that the size of the development team has a lot to do with how you'll want to answer that question for your own effort, because multiple workstreams on a single codebase has a tendency to skyrocket technical debt. All of the above is premised on your ability to keep technical debt in control -- I have no doubt that irresponsible dependence on technical debt will kill your company just as effectively as relying too much on financial debt -- I've seen it happen.
The bottom line is that once again I find myself rejecting dogmatic adherence to a particular mindset. And contrary to my initial metaphor, that doesn't seem evil at all. Maybe I got it backwards and the high-ceremony Agilistas are the dark side, and the lean startup hackers are the Rebel Alliance... what do you think?