A Case of Mistaken Agility

There is a word that seems to be cropping up with ever greater frequency in my particular corner of the software world.  That word is: Agile.  This is odd, because my particular corner consists predominantly of lumbering mission-critical systems.  A place where agility rarely treads.  Yet, from upper-management, recruitment consultants and job specs alike, I'm hearing/seeing the word the 'Agile' a lot.  I can't explain the recent surge of popularity - maybe a hard-hitting marketing salvo from the Agile Marketing Board - but I can't help being concerned that a lot of the interest is built upon a misunderstanding.  Specifically, one that confuses the two quite different sentiments of 'being more agile' and 'using an Agile development methodology'.  They are not the same thing.  I fear the confusion arises because this is the first significant software movement with an overtly positive name.  You don't, for instance, hear many project managers saying, 'oh, I think we need to be more Iterative' - or Extreme, or Waterfall(!).  But, Agile, hey - who wouldn't want to be more agile?  Well, in fact, despite its positive (and, admittedly, apt) name, it is only suitable for a subset of projects, just like any other methodology.  Agile methods promote an environment that copes well with change, and that sounds great, but in doing so it promotes a culture that invites change.  And this is the crux, because in general, change is the death of many a project and it should be a eschewed in all but those cases where it is entirely unavoidable.

To understand where Agile fits we need to look at the bigger picture.  You can think of all development methodologies as on a continuum from predictive to adaptive.  The further you move from the predictive end of the continuum, the less formal planning and process is employed.  Agile methodologies fall at the adaptive end, so they can say less about what will happen in the future, but they are better equipped to deal with change.  Take a moment to read the Agile manifesto:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Glossing over the tragically ill-formed opening sentence (and trust me I'm having to hold myself back here), let's consider each of these statements in turn:

Individuals and interactions over processes and tools.  I can quote many occasions that process (and tools to enforce that process) are merely a substitute for good engineers, so in essence I agree with this sentiment.  But in practice it may be foolhardy to assume that every member of your team is such a high-calibre and well-rounded engineer that they will only ever do the right thing.  Maybe the Googles of this world can afford to assemble such a team, but generally it is unlikely.  Consider carefully how little process you can afford to do away with.  Are you really not going to employ a change-management or code review process/tool just because you trust your guys?

Working software over comprehensive documentation.  Obviously, the former is more valuable than the latter.  Stating it in a manifesto is at best bland, but at worst it engenders a culture that considers documenting software as not important.  It is important.  Ask someone who has been asked to upgrade a 15-year-old system consisting of one million lines of code.  But the real point here is that no compromise is actually necessary.  With model-driven development the software and the documentation can be one and the same thing.  Why should this not be a tenet in the manifesto instead?

Customer collaboration over contract negotiation.  Customer collaboration throughout the development life-cycle is essential, but if your project is being funded by an external customer then you must have a contract specifying at least the key user requirements.  Sure, the holy-grail is to sign a contract on an hourly-rate basis, where customer collaboration (and indeed procrastination) is just money in the bank, but that's why contracts are rarely drawn up this way and only between the friendliest of companies.  I've seen companies lose millions over woolly contracts.  In the real world it would be suicide to overlook the value of a solid contract with the promise of some jolly fun customer collaboration down the road.

Responding to change over following a plan.  Having the ability to respond to change is unquestionably beneficial, but suggesting that change is valued more highly than a plan is a little suspect.  There is no world in which regular change will result in a more efficient outcome than following a plan (and having no change).  Change is absolutely inevitable, I accept that.  But it should be minimised where possible rather than welcomed.  The key, as with everything, is balance - knowing what elements are reasonably stable with high confidence and so knowing at what level of detail to plan.

Despite my misgivings in the wording of the manifesto, I am not against Agile development methods in general.  But it is vitally important to be aware that such methods are only ever going suit a subset of projects.  Agile methods may work well with small co-located teams (<10) of high-skilled senior members, but the more the structure varies from this, the less chance an Agile method is going to have of achieving greater agility, with a lower-case 'a'.  In addition, I would argue that Agile methods are most suited to the development of products for a mass market rather than for projects with a small number of defined stakeholders.

As with all disciplines which offer extreme views, the right answer for any given scenario is almost always somewhere in the middle.  With this is mind I intend to propose what I have named the Balanced Methodology, and I shall detail this over coming posts.