A Case of Mistaken Agility

There is a word that seems to be cropping up with ever greater frequency in my particular corner of the software world.  That word is: Agile.  This is odd, because my particular corner consists predominantly of lumbering mission-critical systems.  A place where agility rarely treads.  Yet, from upper-management, recruitment consultants and job specs alike, I'm hearing/seeing the word the 'Agile' a lot.  I can't explain the recent surge of popularity - maybe a hard-hitting marketing salvo from the Agile Marketing Board - but I can't help being concerned that a lot of the interest is built upon a misunderstanding.  Specifically, one that confuses the two quite different sentiments of 'being more agile' and 'using an Agile development methodology'.  They are not the same thing.  I fear the confusion arises because this is the first significant software movement with an overtly positive name.  You don't, for instance, hear many project managers saying, 'oh, I think we need to be more Iterative' - or Extreme, or Waterfall(!).  But, Agile, hey - who wouldn't want to be more agile?  Well, in fact, despite its positive (and, admittedly, apt) name, it is only suitable for a subset of projects, just like any other methodology.  Agile methods promote an environment that copes well with change, and that sounds great, but in doing so it promotes a culture that invites change.  And this is the crux, because in general, change is the death of many a project and it should be a eschewed in all but those cases where it is entirely unavoidable.

To understand where Agile fits we need to look at the bigger picture.  You can think of all development methodologies as on a continuum from predictive to adaptive.  The further you move from the predictive end of the continuum, the less formal planning and process is employed.  Agile methodologies fall at the adaptive end, so they can say less about what will happen in the future, but they are better equipped to deal with change.  Take a moment to read the Agile manifesto:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:

Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan

That is, while there is value in the items on the right, we value the items on the left more.

Glossing over the tragically ill-formed opening sentence (and trust me I'm having to hold myself back here), let's consider each of these statements in turn:

Individuals and interactions over processes and tools.  I can quote many occasions that process (and tools to enforce that process) are merely a substitute for good engineers, so in essence I agree with this sentiment.  But in practice it may be foolhardy to assume that every member of your team is such a high-calibre and well-rounded engineer that they will only ever do the right thing.  Maybe the Googles of this world can afford to assemble such a team, but generally it is unlikely.  Consider carefully how little process you can afford to do away with.  Are you really not going to employ a change-management or code review process/tool just because you trust your guys?

Working software over comprehensive documentation.  Obviously, the former is more valuable than the latter.  Stating it in a manifesto is at best bland, but at worst it engenders a culture that considers documenting software as not important.  It is important.  Ask someone who has been asked to upgrade a 15-year-old system consisting of one million lines of code.  But the real point here is that no compromise is actually necessary.  With model-driven development the software and the documentation can be one and the same thing.  Why should this not be a tenet in the manifesto instead?

Customer collaboration over contract negotiation.  Customer collaboration throughout the development life-cycle is essential, but if your project is being funded by an external customer then you must have a contract specifying at least the key user requirements.  Sure, the holy-grail is to sign a contract on an hourly-rate basis, where customer collaboration (and indeed procrastination) is just money in the bank, but that's why contracts are rarely drawn up this way and only between the friendliest of companies.  I've seen companies lose millions over woolly contracts.  In the real world it would be suicide to overlook the value of a solid contract with the promise of some jolly fun customer collaboration down the road.

Responding to change over following a plan.  Having the ability to respond to change is unquestionably beneficial, but suggesting that change is valued more highly than a plan is a little suspect.  There is no world in which regular change will result in a more efficient outcome than following a plan (and having no change).  Change is absolutely inevitable, I accept that.  But it should be minimised where possible rather than welcomed.  The key, as with everything, is balance - knowing what elements are reasonably stable with high confidence and so knowing at what level of detail to plan.

Despite my misgivings in the wording of the manifesto, I am not against Agile development methods in general.  But it is vitally important to be aware that such methods are only ever going suit a subset of projects.  Agile methods may work well with small co-located teams (<10) of high-skilled senior members, but the more the structure varies from this, the less chance an Agile method is going to have of achieving greater agility, with a lower-case 'a'.  In addition, I would argue that Agile methods are most suited to the development of products for a mass market rather than for projects with a small number of defined stakeholders.

As with all disciplines which offer extreme views, the right answer for any given scenario is almost always somewhere in the middle.  With this is mind I intend to propose what I have named the Balanced Methodology, and I shall detail this over coming posts.

The Diary of a Software Manager

Some time ago I had the idea that I would write a book about software management, to persist into a more concrete form the gems of wisdom gleaned from my experiences over the last decade in the software world. As much for me as anyone else - lest I ever forget.  The idea was for the book to be structured like a diary of a theoretical 12-month software project.  It would state, month-by-month, each activity that must take place and each artifact that must be created in order for the programme to run to completion successfully: on time, on budget and with minimum headaches.  It would act as a step-by-step guide and checklist for any manager embarking on a software project.  And as the principles covered would be general, so the stages it refers to would be scaleable.  If a programme was to run for 24-months then you could simply double the length of the each of the phases detailed in the book. 

This is still the plan, but I figured that along the way I would also post the content here in a more haphazard format, should anyone be interested enough to read and comment on it.  Hopefully you will find something of use. 

But before all of that, why should you care what I have to say about the art of software development?  Well, by way of convincing you, let me first make you aware that I am an actual real fully-instantiated software manager.  I am not an abstract-thinking head-in-the-clouds pure-virtual academic.  After my computer science degree I went straight into industry and have been there ever since.  I started out as a developer, soon became a team lead, then architect and manager.  I have led the development of numerous large-scale systems, between 18 and 30 months in duration, with team sizes up to 30 people.  And so far, they have all been 'successful'.  And by successful I mean they have been completed in-line with the schedule and budget that I devised on my day one – not the schedule and budget imagined by project management during the bid phase, which in some cases can be quite a creative work of art.

Let me make clear that I attribute no success to any exceptional ability on my part - and those who know me can vouch for that - but simply to two factors.  Firstly, being lucky enough to be told how to do things properly early in my career; and secondly, being blessed with a basic dose of common sense.  I can do nothing to bestow upon you the latter, but I can at least help you out with the doing things properly bit.  I hope you enjoy.