January 30, 2012

Rendering Collections of Heterogeneous Objects in Rails 3.2

This commit by Grant Hutchins & Peter Jaros back in July 2011 gives ActiveModels the ability to define their own partial paths by implementing a to_partial_path method. (You can find a great explanation of this killer new feature on José Valim's blog post about it. Just scroll down to #4 Custom Partial Paths.)

I'm blogging about it here, because it impacts a technique that I described in May 2011 as Rendering Heterogeneous Collections of Objects in Rails. If you happen to be using that technique or anything like it involving the use of ActiveModel's partial_path method, you'll get deprecation warnings like this when upgrading to Rails 3.2:

DEPRECATION WARNING: partial_path is deprecated and will be removed from Rails 3.2 (ActiveModel::Name#partial_path is deprecated. Call #to_partial_path on model instances directly instead.). (called from partial_path at .../config/initializers/partial_path.rb:12)

In this blog post I describe how to update my technique and get rid of the deprecation warning.

The heterogeneous aspect of my technique had to do with rendering collections of STI objects that all inherited from a single Event base class. For organizational reasons, I wanted to put all my disparate event partials in a single app/view/events directory. Without the hack, they'd need to be in their own view/foo_events/_foo_event.html.erb partials. Ugh.

In Rails 3.2, instead of having to monkey patch the framework to fix that crap, I can just implement to_partial_path on the Event base class like this:

class Event

  def to_partial_path
    "events/#{self.class.name.underscore}"
  end 

end

Ahh, that's much cleaner. Unfortunately, there's still a catch, but only if you use controller namespaces.

Rails 3.2 keeps the annoying merge of the namespace prefix as part of the generated partial path even if you override it using to_partial_path. The good news is that it is easier to get rid of it in Rails 3.2 than before. The monkeypatch in my earlier blog post now becomes:

ActionView::PartialRenderer.class_eval do

  private

  def merge_prefix_into_object_path(pfx, path)
    path
  end 

end

What this little bit of duck punching does is to ensure that when you try to render a collection of objects, it will always return the same partial path regardless of whether you are inside a controller namespace or not. I feel that behavior should be default or at least optional (which is why I describe the feature as annoying.) Let me know if you think it's worth submitting a patch.

Incidentally, in his blog post mentioned above Jose mentions needing to add an as: parameter to your call to render, like this:

render :partial => @activities, :as => :activity

In my testing and examination of the underlying Rails code, it is clear that the :as option is not necessary. The PartialRenderer object always checks for the existence of a to_partial_path method on the object being rendered and calls it if present. 

One more thing. Currently the documentation states the following (italics mine):

Instead of explicitly naming the location of a partial, you can also let PartialRenderer do the work and pick the proper path by checking `to_proper_path` method. If the object passed to render is a collection, all objects must return the same path.

I'm not quite sure why it says that all objects must return the same path, or why that would even be considered desirable behavior (it's really not!) The good news is that I can verify for you that it does work to return different partial paths within the collection, in other words, you can still do the rendering heterogeneous objects trick. In fact, depending on your naming convention, you may even get correctly name local variables in your partial for free.

The following method determines what to call your partial's automatic local variable corresponding to the current item of the collection being rendered:

def retrieve_variable(path)
  variable =
    @options[:as].try(:to_sym) || path[%r'_?(\w+)(\.\w+)*$', 1].to_sym

  variable_counter = :"#{variable}_counter" if @collection
  [variable, variable_counter]
end

That regex is a little tricky, but whether it's the intentional behavior or not, it's going to grab whatever is between the last underscore of your partial name and the dot to use as the variable name. In the case of my own Event hierarchy, that means each partial will get its own event local variable, even though the partials are named _user_created_event, _nomination_event, etc.

Perfect. Now to hope someone doesn't fuck it up in a future release thinking it's a bug :)


December 26, 2011

RSA (Retained Services Agreement)

Over the years I've blogged extensively about Master Service Agreements (known as MSA documents) and I even sell them via this blog. MSAs are a useful tool for consultants wanting to contract individual chunks of work as separate statements of work (via SOW addendums to the MSA). Those SOWs can specify either hourly or deliverable-based pricing.

MSAs are by no means the end-all, be-all of consulting contracts. In fact, if you have the reputation and marketing saavy to get away with using it, I think you should consider instead relying on a Retained Services Agreement or RSA.

The RSA is different from its cousin the MSA in several ways:

  • You only need to execute it once, there are no SOW addendums.
  • It specifies a monthly retainer that is paid upfront, every month
  • It can easily specify an irrevocable minimum contract length
  • It can easily mandate a minimum written notice of termination

Since you're always paid up-front with an RSA, it's significantly easier to pause work on late payment, which is one of the best, if not the absolute most effective way of making sure that you always get paid for your hard work.

Charging an upfront monthly retainer is an interesting adventure if you're accustomed to billing on an hourly basis. It's also a potential sticking point in client negotiations. In essence, an RSA establishes a minimum spend per month, which is always billed no matter how much work is performed. You can optionally establish that the monthly fee is calculated as a block of hours at an hourly rate, which would allow you to bill for overtime, but many people only specify a monthly rate that is not subject to pro-rating. That takes serious courage.

Use of an RSA is not for the faint of heart. You must be confident that you will be able to deliver results that the client wants on a consistent basis. You must be very communicative about the work that you're doing. Most importantly, you must know that you will be able to allot the minimum number of hours committed to that client to them on a monthly basis without stretching yourself too thin. Otherwise, you may be faced with a client that is demanding to see time logs and explanations of where their money went.

Personally, I use an RSA for my consulting contracts nowadays instead of a MSA. As an entrepreneur and independent consultant that's really short on time, I can't afford to constantly renegotiate terms or be chasing after late payments.

Have you signed an RSA before? As a service provider or a client? What is your opinion of RSAs versus MSAs, engagement letters or one-off contracts? Please let me know in the comments.

Buy my proven RSA template bundle here for only $99

December 20, 2011

My Survey

Obie's Survey

November 24, 2011

Hiring me as a consultant

While my web startup DueProps consumes most of my time, I don't make any money from it yet and probably won't for awhile. So I pick up short term consulting gigs from time to time to help replenish my savings.

What I offer is honest and experienced advice, but not deliverables. If you've followed me for awhile, then you know what my areas of expertise are:

  • Growing a consulting operation from scratch
  • Leadership of web development teams
  • Adoption of Agile and Lean Startup techniques
  • Ruby on Rails-related technology (I can pair-program with you or your team)

Hiring me as a consultant is really easy. You may purchase my time at an hourly rate or a daily rate. What you get:

  1. distilled knowledge and advice
  2. a no bullshit approach
  3. advice on demand

I will not try to sell you anything else if you hire me and I’m not interested in ongoing retainers. I might want to give you a free demo of DueProps, but that's just cause I'm so in love with the positive impact it can have on workforces.

To book my time, we just need to have a preliminary conversation about what you want to achieve by hiring me. I don't have problems with signing an NDA, if you require it. Once we officially book our billable time together and execute a simple consulting agreement, I’ll get whatever I need from you to prepare and be available for advice on demand via phone and email.

Getting down to specifics on pricing, if you want to book me hourly, I’m $300 an hour, billed in 15 minute increments.

If you want to book me in person on a daily basis, it's $10,000 a day plus travel and accommodation expenses. Booking a full day of my time is more expensive and involved because of the disruptive nature of travel and the need for more preparation. The precise schedule of a daily engagement doesn’t matter as long as we spend about 8 hours talking or working on your project.

To book a day with me looks like this:

  1. We sign a mutual NDA because while I'm on site I may share confidential stuff with you
  2. You share all documents and resources that we’ll be discussing at least a week prior to the trip
  3. We agree in writing on expectations for our time together, so that I can be 100% prepared

Since I live in Atlanta and you probably don't, booking me for a day will usually involve me traveling to your location. You will take care of booking the travel and accomodations and cover the cost. I only fly Delta Airlines (first class) and I only stay at Hilton Hotels.

That’s it. The terms are transparent and non-negotiable. My email is obiefernandez@gmail.com I can help you win.

--

Hat tip to one of my heroes, Mr. Tincup, for inspring me to describe this offer in this way. You should hire him for marketing advice.

August 22, 2011

Adding a Counter Cache to Existing Records

Adding a counter cache to a has_many/belongs_to ActiveRecord relationship after the underlying records have been around awhile is one of those tasks that tends to trip up Rails newbies.

Here's an example migration showing the necessary steps.

  1. Add the counter_cache column as an integer. (If following normal rails convention, name the column with the plural name of the has_many association followed by _count)
  2. Make sure that the column is set to disallow null values and defaults to 0.
  3. Reset the counters for existing records. The reset_counters method is not smart enough to take an array of ids, so you have to call it for each record you want updated.
class AddCommentsCountToNominations < ActiveRecord::Migration
  def self.up
    add_column :nominations, :comments_count, :integer, null:false, default: 0
# reset cached counts for nominations with comments only
    ids = Set.new
    Comment.all.each {|c| ids << c.nomination_id}
    ids.each do |nomination_id|
      Nomination.reset_counters(nomination_id, :comments)
    end
  end
  def self.down
    remove_column :nominations, :comments_count
  end
end

The :counter_cache option is covered in detail on pages 195-196 of my book The Rails 3 Way, including some interesting gotchas.

August 12, 2011

Deep Integration of Ruby with Semantic Web Ontologies

Download Deep Integration of Ruby with Semantic Web Ontologies (PDF, 225k)

My presentation file needed a new home since the primary link that comes up in Google when you search for it is a ThoughtWorks server that no longer exists.

I'm also hosting a related whitepaper by Dominic Mainz from 2009

Deep Integration of Semantic Web Using Ruby (PDF, 7.7mb)

Incidentally, it's been about 6 years since I first took interest in this subject and I still think there's room for significant innovation here. Maybe I've read too much Neal Stephenson, but I can imagine people moonlighting as domain experts and maintaining a global commons ontological library that can be imported to form the foundation of your application's domain model just as easily as we import ruby gems today. A similar intellectual energy drove Wikipedia -- why not make it work for software models?

where are un-ordered tools?

While digging up some stuff on my old blog I found this post: The new dynamics of strategy: Sense-making in a complex and complicated world that I published back in December of 2004. Almost 7 years later and I'm not so sure that we've made progress with real-world tools leveraging this research.

The authors coin the term un-order in much the same sense as undead, to refer to "emergent order". Not an absence of order, but instead the patterns that emerge from situations which undergo continual change, subject to chaotic influences. This is of particular interest to the agile community and particularly XPers, whose motto isembrace change, which explains why it seems to be getting a lot of attention from agile bloggers. I see a window of opportunity opening up for a slew of open-source projects that build on this information. While reading Wisdom of Crowds, I got the strong impression that a far-flung organization (such as the the consulting company that I work for), will benefit by moving agressively on these knowledge management advances. For instance, devise a polling tool with differing modes that match the various identities that we live on a day-to-day basis, namely: consultant, coach, manager, member of our client team, member of the parent company. Within each of these modes, features that allow referendums and continual feedback for all levels: personal, team and organization.

Maybe Asana is a step in the right direction? I've been using it with my new startup as our project management tool for everything that doesn't fit into Pivotal Tracker. Liking it so far.

July 24, 2011

Startup Life Is

Working on raising a half-million dollar seed round. Getting commitments for half of said seed round. Feeling really good about the future. Networking. Making phone calls. Doing demos. Banging out Keynote presos.

Booking last minute tickets to San Francisco for pitch meetings with a couple of the most legendary personalities in the valley. Feeling proud of yourself and a little scared at the same time.

Having to multitask between coding and emails and phone calls with investors and service providers and customers and potential customers and lawyers and advisors. Signing documents. Emailing documents. Faxing documents. Coding. Managing. Snacking. Somehow keeping everything from falling apart.

Remembering to call kids. Remembering to call mom.

Spending a few minutes adding a message in your app's notification emails telling users to hit reply to add a comment. Routing said replies to your own inbox, as a test to see if anyone will actually use that feature. Realizing that it would probably be really bad if they actually did start using that feature, so biting the bullet and writing incoming email processing. Taking a few hours to get it all tested and working, but mostly because you have trouble getting a reliable SSH tunnel between the outside world and your local development environment. Getting lots of help from #lazytwitter and singing the praises of modern development tools and web services.

Words with friends.

Realizing you're not sure where you're going to sleep while you're on your trip to San Francisco. Realizing you probably should get a rental car so you can drive yourself down to Mountain View. Kicking yourself for not having more meetings lined up, to make better use of the trip. Rationalizing that the trip was too last minute anyway.

Putting the finishing touches on a major piece of functionality. Beaming with pride. Realizing it's 10:30pm and you had commitments tonight. Rushing home. Dragging yourself out to a cousin's birthday party past midnight even though you're exhausted.

Drinking. Karoekeing. Subwaying. Sleeping.

Waking at 9 even though you went to sleep at 4:30am. Not being able to fall back asleep because you're thinking of great ideas to work on once you get up. Reaching over and grabbing your notebook to start hacking on some ideas while still in bed. Hunger finally gets you out of bed at 10.

Remembering you better pick up your laundry because otherwise you won't have any clean underwear for said trip. Further realizing that you don't have any shoes with you that aren't flip flops or sneakers, because you've been traveling and living out of a suitcase for the last four months and forgot your nice shoes somewhere along the way.

Deciding that since it's Saturday you might as well stop by the barbershop and get a haircut on your way in to the office since it's been a few weeks and you don't want to look shaggy for the upcoming SF trip. Finally getting into the office at noon, to meet up with the rest of your team.

Putting the finishing touches on another major piece of functionality. Beaming with pride again.

Breaking for lunch and deciding to celebrate Saturday by turning left around the corner instead of right. Amazing yourself with a whole new array of lunch options, which you have neglected until now in favor of the same falafel place where you've been buying meals about 75% of the time for weeks.

Going back to the office and banging out yet another major piece of functionality. Beaming with pride again, but celebration is somewhat tempered by your back, which is hurting since you've been sitting all day, everyday, for the past week. Deciding to alleviate said hurt by hanging upside down for several minutes, suspended by ankle shackles. Having your lead engineer hover nervously around you during your suspension, nervous that you might fall and break your neck prior to close of current funding round.

Getting less and less ambitious with coding as energy wanes until eventually you decide to call it a night and go home at 10:30pm. Buying a hummus wrap on the way home, because you realize that you forgot to eat dinner during your exciting all-day coding session. Eating said hummus wrap on the subway.

Walking home from subway stop in 100 degree heat. Smiling at the wide open fire-hydrant flooding the street next to the building where you've been living. Thinking that there were probably a bunch of kids playing in that hydrants cooling stream hours earlier. Feeling nostalgic for simpler times, but realizing you really couldn't ask for things to be any better.

Packing to trip. Turning down offers to go out with friends. Getting into bed at a reasonable hour. Not going to sleep.

Writing a blog post.

Saying goodnight.

 

July 08, 2011

The Next Big Leap

Zed Dive

In my view, FITness, Cucumber and other acceptance test frameworks represented a big leap in software development effectiveness because they allow coders to get intimately involved with (and in some cases replace) Product Management/QA roles. Extreme Programming has developers working directly with product owners and capturing their requirements as "acceptance criteria" on story cards. Eventually, the desire to automate acceptance testing and best practices for how to express acceptance criteria led to GIVEN-WHEN-THEN support baked into actual testing frameworks such as Cucumber.

That's okay, but it's not nearly enough. Let's talk about how the next big leap in effectiveness is going to happen. Assume the following story is true:

  • Given a specification for a feature that has not been demonstrably linked to business value,
  • When a developer works to deliver an implementation of that feature,
  • Then the worth of the implementation work is indeterminate until some future time when its business value is validated

Most feature requests that I've ever worked with in the early stages of a project, are ideations of the product visionary/designers/entrepreneur, based on intuition rather than customer feedback. And that's perfectly normal and acceptable to me. The more innovative a creation, the more work you usually have to do before you put it out in front of customers and get meaningful feedback. But the impact of those features on key business metrics are assumed, if it gets any explicit thought at all.

Aside: As for the implementation of those feature requests, I'm going to assume that being faithful to the specification is implicit, as it should be when we're talking about professional programmers. (Not working with professional programmers? GIGO)

Given those assumptions, I think what myself and others have realized as a problem with so-called "high-ceremony" adherence to methodologies such as BDD (Behavior-Driven Development) is that by obsessing over correctness of the implementation as faithful to the spec, it becomes easy to lose sight of the true value of our work: to deliver business value, as measured by key business metrics. We may obsess over crafting pristine, bulletproof, maintainable code, and yet its worth is indeterminate at best. And why, as developers, would we be satisfied with that!

"Measure as much as you can - no feedback == no direction"  Hugo Rodger-Brown

How can we establish that our work is not a waste of time? One technique that I hope to work on in near future is the specification validation of business value right in my acceptance criteria as part of its test cases. Said differently, I am looking for an easy way to specify non-functional requirements as part of acceptance criteria in such a way that it is automatically tested. It would be an extension or evolution of the current BDD toolset, not a replacement. You can even use de rigeur GWT syntax:

Given the implementation of email notifications,
When 100 customer events have been processed,
Then the mean time between events metrics should improve by 30%

or better

Then the net revenue per event should improve by 30%

or best

Then the net revenue per event should increase by $100

I care more about business value than faithful adherence to a particular functional requirement. My specifications should reflect that mindset. What makes this whole approach non-trivial is the challenge of establishing credible causal relationships in my acceptance criteria, given the multi-variate nature of dozens of interleaved features in a particular application and the way that key business metrics overlay many features at once.

Despite the challenges, I think the concept I'm describing is the next big leap. The first challenge is to get people talking about it and definition of goals. The second is to design a methodology and technical framework for accomplishing those goals.

What do we call the whole thing? Business-Driven Development? (no) What do we call the specs? Can you write specs against metrics in a tool such as KissMetrics? It's hard (or at least has been for me) to figure out what metrics I need until after I've written the features that yield measurement data used to calculate those metrics. How do I fix or at least improve that workflow? How do I describe the discipline needed to think of the metrics prior to specifying a feature?

Right now all I'm trying to do via this blog post and my public comments is to start a dialogue. I'm not trying to make any blanket statements about the correctness of BDD/TDD or testing in general, just pointing out that there's lots of room outside of that particular box for innovation.

As for my comments at RailsConf and Goruco, it seems to me that at a startup that has not achieved market fit, it's wrong to spend much time ensuring that a given piece of software matches its functional specification. (Specs that often won't exist, because the same person specifying the functionality is the person writing the code, in which case I'm saying don't bother writing the specs. Just "cowboy" it and do a complete rewrite later if it proves necessary.) Once you have your MVP, write specs for how particular code changes will impact key business metrics. Regressions will probably fail to improve key business metrics.

The question is not "should I test (or spec)", but rather it is "what should I test?" As an industry, to the extent that TDD/BDD has caught on at all, I don't think we're testing the right things, with our emphasis on correctness of implementation. When we figure out how to rigorously test for business value it's going to be the next big leap. That's a big green field waiting for all of us to play in and I want to be one of the first.

May 20, 2011

The Dark Side Beckons?

Ries3
Lord Ries by Cody San

For years I've advocated a high-ceremony, test-first development methodology where use of BDD (or TDD) is sacrosanct and uncovered production code is a sin. Then I got to know Eric Ries and studied his Lean Startup stuff and got started with my own lean startup venture and over time...

Something's happening. I'm not the Agilist I should be. I want more. But I know I shouldn't.

I no longer believe in high-ceremony development practices for early-stage startups. Actually, I'm pretty sure that they're poisonous for most web startups, hence my lightning talk at Railsconf 2011. Until you are able to prove that you have a viable market, that customers will give you money for your product, you shouldn't be sinking a lot of time and money into implementation. Assuming you are a competent Rails developer, you should be able to hack out features and take on quite a bit of technical debt to build throwaway proof-of-concept versions of your product. I'm pretty much advocating construction of large "spike solutions" except that unlike the traditional XP definition, the problem you're trying to solve is the problem of finding your market fit. It has nothing to do with the technology.

Also, we are commonly told that spikes (aka prototype systems) are not meant for production and should be thrown away. I think that advice is very dependent on the skills of the people writing the prototype system. If they are experienced developers, they're going to inherently know a lot of the application patterns that should be used and the amount of technical debt introduced will be low enough to feel comfortable with the app in production.

Some caveats:

  • I'm specifically talking about typical web-based applications built with Rails where nobody is going to die if you occasionally get a 500 error on an edge case. Use continuous deployment so that you're comfortable that you can turn around code improvements and bug fixes almost immediately.
  • Use TDD to "discover" the design of parts of the system that are unfamiliar to you. This primarily applies to unit testing
  • By all means have some integration-level specs that test the "happy path" through your system so that you don't inadvertently break your entire app. AKA "smoke tests"

Early on in the startup process, it's much more important to be testing against business metrics than anything having to do with code. One of the most thought-provoking pieces of advice I've heard lately came from Hiten Shah of KISSmetrics who told the latest LSM NYC crowd to consider the effects of a feature of key business metrics to be the acceptance criteria for that feature.

In other words, before you implement a feature, you should have an assumption that tells you what the effect is going to be: higher click-through, engagement, satisfaction ratings, etc. This results in an acceptance test that is time-shifted; you won't know the results until you deploy. I believe this is fertile ground for technical innovation. While there are plenty of frameworks that simplify AB testing and real-time monitoring, I don't believe there exists anything along the lines of Cucumber that involves developers in cultivating a mindset of provable business value. David Chelimsky and I joked around that perhaps such a tool should be called Squash.

One of the natural questions that arises from this line of thinking is: When am I no longer an early-stage startup? The implication being that at some point, you know enough about your market fit and product requirements that you can transition into what I've been calling high-ceremony development practices.

I don't have the answer to that question. I suspect that the size of the development team has a lot to do with how you'll want to answer that question for your own effort, because multiple workstreams on a single codebase has a tendency to skyrocket technical debt. All of the above is premised on your ability to keep technical debt in control -- I have no doubt that irresponsible dependence on technical debt will kill your company just as effectively as relying too much on financial debt -- I've seen it happen.

The bottom line is that once again I find myself rejecting dogmatic adherence to a particular mindset. And contrary to my initial metaphor, that doesn't seem evil at all. Maybe I got it backwards and the high-ceremony Agilistas are the dark side, and the lean startup hackers are the Rebel Alliance... what do you think?

My Companies

My Latest Book

My Book Series

My Conference

Bizconf is the only training event specifically crafted for owners and managers of small to mid-sized web design and development firms.

Dates for 2011 TBA