Obie Fernandez

Startup Life Is

Working on raising a half-million dollar seed round. Getting commitments for half of said seed round. Feeling really good about the future. Networking. Making phone calls. Doing demos. Banging out Keynote presos.

Booking last minute tickets to San Francisco for pitch meetings with a couple of the most legendary personalities in the valley. Feeling proud of yourself and a little scared at the same time.

Having to multitask between coding and emails and phone calls with investors and service providers and customers and potential customers and lawyers and advisors. Signing documents. Emailing documents. Faxing documents. Coding. Managing. Snacking. Somehow keeping everything from falling apart.

Remembering to call kids. Remembering to call mom.

Spending a few minutes adding a message in your app's notification emails telling users to hit reply to add a comment. Routing said replies to your own inbox, as a test to see if anyone will actually use that feature. Realizing that it would probably be really bad if they actually did start using that feature, so biting the bullet and writing incoming email processing. Taking a few hours to get it all tested and working, but mostly because you have trouble getting a reliable SSH tunnel between the outside world and your local development environment. Getting lots of help from #lazytwitter and singing the praises of modern development tools and web services.

Words with friends.

Realizing you're not sure where you're going to sleep while you're on your trip to San Francisco. Realizing you probably should get a rental car so you can drive yourself down to Mountain View. Kicking yourself for not having more meetings lined up, to make better use of the trip. Rationalizing that the trip was too last minute anyway.

Putting the finishing touches on a major piece of functionality. Beaming with pride. Realizing it's 10:30pm and you had commitments tonight. Rushing home. Dragging yourself out to a cousin's birthday party past midnight even though you're exhausted.

Drinking. Karoekeing. Subwaying. Sleeping.

Waking at 9 even though you went to sleep at 4:30am. Not being able to fall back asleep because you're thinking of great ideas to work on once you get up. Reaching over and grabbing your notebook to start hacking on some ideas while still in bed. Hunger finally gets you out of bed at 10.

Remembering you better pick up your laundry because otherwise you won't have any clean underwear for said trip. Further realizing that you don't have any shoes with you that aren't flip flops or sneakers, because you've been traveling and living out of a suitcase for the last four months and forgot your nice shoes somewhere along the way.

Deciding that since it's Saturday you might as well stop by the barbershop and get a haircut on your way in to the office since it's been a few weeks and you don't want to look shaggy for the upcoming SF trip. Finally getting into the office at noon, to meet up with the rest of your team.

Putting the finishing touches on another major piece of functionality. Beaming with pride again.

Breaking for lunch and deciding to celebrate Saturday by turning left around the corner instead of right. Amazing yourself with a whole new array of lunch options, which you have neglected until now in favor of the same falafel place where you've been buying meals about 75% of the time for weeks.

Going back to the office and banging out yet another major piece of functionality. Beaming with pride again, but celebration is somewhat tempered by your back, which is hurting since you've been sitting all day, everyday, for the past week. Deciding to alleviate said hurt by hanging upside down for several minutes, suspended by ankle shackles. Having your lead engineer hover nervously around you during your suspension, nervous that you might fall and break your neck prior to close of current funding round.

Getting less and less ambitious with coding as energy wanes until eventually you decide to call it a night and go home at 10:30pm. Buying a hummus wrap on the way home, because you realize that you forgot to eat dinner during your exciting all-day coding session. Eating said hummus wrap on the subway.

Walking home from subway stop in 100 degree heat. Smiling at the wide open fire-hydrant flooding the street next to the building where you've been living. Thinking that there were probably a bunch of kids playing in that hydrants cooling stream hours earlier. Feeling nostalgic for simpler times, but realizing you really couldn't ask for things to be any better.

Packing to trip. Turning down offers to go out with friends. Getting into bed at a reasonable hour. Not going to sleep.

Writing a blog post.

Saying goodnight.

 

The Next Big Leap

In my view, FITness, Cucumber and other acceptance test frameworks represented a big leap in software development effectiveness because they allow coders to get intimately involved with (and in some cases replace) Product Management/QA roles. Extreme Programming has developers working directly with product owners and capturing their requirements as "acceptance criteria" on story cards. Eventually, the desire to automate acceptance testing and best practices for how to express acceptance criteria led to GIVEN-WHEN-THEN support baked into actual testing frameworks such as Cucumber.

That's okay, but it's not nearly enough. Let's talk about how the next big leap in effectiveness is going to happen. Assume the following story is true:

• Given a specification for a feature that has not been demonstrably linked to business value,
• When a developer works to deliver an implementation of that feature,
• Then the worth of the implementation work is indeterminate until some future time when its business value is validated

Most feature requests that I've ever worked with in the early stages of a project, are ideations of the product visionary/designers/entrepreneur, based on intuition rather than customer feedback. And that's perfectly normal and acceptable to me. The more innovative a creation, the more work you usually have to do before you put it out in front of customers and get meaningful feedback. But the impact of those features on key business metrics are assumed, if it gets any explicit thought at all.

Aside: As for the implementation of those feature requests, I'm going to assume that being faithful to the specification is implicit, as it should be when we're talking about professional programmers. (Not working with professional programmers? GIGO)

Given those assumptions, I think what myself and others have realized as a problem with so-called "high-ceremony" adherence to methodologies such as BDD (Behavior-Driven Development) is that by obsessing over correctness of the implementation as faithful to the spec, it becomes easy to lose sight of the true value of our work: to deliver business value, as measured by key business metrics. We may obsess over crafting pristine, bulletproof, maintainable code, and yet its worth is indeterminate at best. And why, as developers, would we be satisfied with that!

"Measure as much as you can - no feedback == no direction"  Hugo Rodger-Brown

How can we establish that our work is not a waste of time? One technique that I hope to work on in near future is the specification validation of business value right in my acceptance criteria as part of its test cases. Said differently, I am looking for an easy way to specify non-functional requirements as part of acceptance criteria in such a way that it is automatically tested. It would be an extension or evolution of the current BDD toolset, not a replacement. You can even use de rigeur GWT syntax:

Given the implementation of email notifications,
When 100 customer events have been processed,
Then the mean time between events metrics should improve by 30%

or better

Then the net revenue per event should improve by 30%

or best

Then the net revenue per event should increase by $100

I care more about business value than faithful adherence to a particular functional requirement. My specifications should reflect that mindset. What makes this whole approach non-trivial is the challenge of establishing credible causal relationships in my acceptance criteria, given the multi-variate nature of dozens of interleaved features in a particular application and the way that key business metrics overlay many features at once.

Despite the challenges, I think the concept I'm describing is the next big leap. The first challenge is to get people talking about it and definition of goals. The second is to design a methodology and technical framework for accomplishing those goals.

What do we call the whole thing? Business-Driven Development? (no) What do we call the specs? Can you write specs against metrics in a tool such as KissMetrics? It's hard (or at least has been for me) to figure out what metrics I need until after I've written the features that yield measurement data used to calculate those metrics. How do I fix or at least improve that workflow? How do I describe the discipline needed to think of the metrics prior to specifying a feature?

Right now all I'm trying to do via this blog post and my public comments is to start a dialogue. I'm not trying to make any blanket statements about the correctness of BDD/TDD or testing in general, just pointing out that there's lots of room outside of that particular box for innovation.

As for my comments at RailsConf and Goruco, it seems to me that at a startup that has not achieved market fit, it's wrong to spend much time ensuring that a given piece of software matches its functional specification. (Specs that often won't exist, because the same person specifying the functionality is the person writing the code, in which case I'm saying don't bother writing the specs. Just "cowboy" it and do a complete rewrite later if it proves necessary.) Once you have your MVP, write specs for how particular code changes will impact key business metrics. Regressions will probably fail to improve key business metrics.

The question is not "should I test (or spec)", but rather it is "what should I test?" As an industry, to the extent that TDD/BDD has caught on at all, I don't think we're testing the right things, with our emphasis on correctness of implementation. When we figure out how to rigorously test for business value it's going to be the next big leap. That's a big green field waiting for all of us to play in and I want to be one of the first.