Diary of a Software Manager

Stickman System

2011-11-22T07:21:00.000-08:00

If the terms 'Use Case diagram' and 'XKCD' both mean nothing to you, then neither will this (but I recommend finding out about one of them).

Social Media: It's Time to Open Up

2011-09-12T14:04:00.000-07:00

I've decided this geeky post is almost palatable by the normal people over at my other blog so I've posted it there: http://fenchurchepiphany.blogspot.com/2011/09/social-media-its-time-to-open-up.html

Android in Pictures

2011-06-27T12:58:00.000-07:00

On approaching a new framework or technology (or indeed system) it is the poor engineer who eschews the documentation; it is the good engineer who studies it diligently; and it is the manager who ... just looks at the pictures. For as the old adage goes: a diagrammatic abstraction using graphical notation is worth a thousand words. Or something like that. And cliche though it may be, true it nonetheless is.

Being the simpleton that I am (is there a design pattern for that? - see I'm getting confused again), I've always been a strong advocate of pretty pictures. And here I don't necessarily mean the use of formal notation for detailed design, but something at a much higher level. When I first get involved in a project, it may take me a while to get my little brain around all the big concepts involved, and nothing cements the process better than getting it all down in a good doodle. Done well, this doodle can become something else. It can become the first port-of-call for anyone joining the team - a way-in diagram. It serves to set the scene - capturing, as it does, some grand context or high level view of a system (or a fundamental component therefore).

There is no magic formula for arriving at such a masterpiece. But when you have one you'll know. It will chime with everyone on the team, and thereafter copies of it will be seen strewn about on people's desks for months to come, with notes and annotations scrawled upon it. It will often be the focal point of discussion or the starting point of meetings. It will become familiar to everyone. People will grow comfortable with it and be put at ease when it fades in as the opener to an otherwise daunting PowerPoint presentation.

Such a diagram need not conform to any formal notation, like UML. Dare I say, it should not conform. It must be a renegade. To get your message across at this level it needs to employ a lexicon that is accessible to everyone, from hardcore coder to programme manager. And the diagram needs to be as visually stimulating as possible without detracting from the content. As such, there is no notation that supports its genesis, any more than there is a notation for creating a still-life or a seascape. Every situation will warrant a different style and there is no substitute for being creative, dare I say artistic. Yes, even the paintbrush is a valuable tool in the software manager's armoury.

But enough of the pretext. I have recently be exploring the Android architecture to evaluate its applicability to larger industry. Being the 'good engineer' that I am, I actually read the documentation, which, on the whole is very good. But, as is often the case, I felt that the way-in diagram was missing. And is indeed not to be found anywhere on the web. There are an awful lot of references to the Android OS architecture diagram, as shown here. This is, no question, useful, but what would have been even moreso was a diagram depicting the architecture of an application. So, in its absence, I thought I'd cobble one together. It's not much and there is probably much room for improvement, but it's a start.

Android Application Architecture

Note, that most of the coupling has been omitted to save it becoming a tangled mess. I won't put many words to it. Read the Android Application Fundamentals documentation and it should all make sense.

Okay, if you really haven't got time, here's a very rushed precis. An Activity is effectively the logic running a single screen in an Android application. The featured application has two Activities and therefore two screens. Screens are defined in XML. The XML schema for defining screen layouts contain elements for all the widgets you'd expect to find in a UI toolkit. Layouts for Dialogs (which are shown in front of screens) can also be defined in XML. Activities can create and show Dialogs dynamically. Thematic elements (such as fonts and colours) can be separated from layout by defining them in separate XML files, as hierarchical styles analogous to CSS.

Capability that needs to run when no screen is being displayed is achieved using a Service. An Activity can register for updates from a Service via a Broadcast Receiver.

Transitory packets of data can be passed between Activies and Services as Intents (not shown). For data that needs to be shared throughout the lifetime of an application, one solution is via a singleton class which exposes public accessors to static data. Similarly, common logic can be encapsulated in singleton classes and accessed statically.

Persistence is provided by ContentProviders which wrap up access to either the file system, an SQLite database or the web.

So there we go, my trivial contribution to the Android movement.

Mobilising your Army

2011-06-21T09:07:00.000-07:00

A significant task during the mobilisation phase of a programme involves the mobilising of a team - which is not nearly as fun as it sounds. Often, within larger organisations, many of the names destined for a programme will already be in the frame, maybe those who worked on a bid/assessment phase of the programme, or resources from other parts of the business. (This may be a blessing or a curse - depending on who those names are). However, if you are charged with mobilising the team from scratch then you need to start considering this real quick. It won't matter how good your pivot tables are for your end-of-month progress report, if you haven't got any people then there ain't gonna be no progress to report. (Yes, yes, double negatives, I know). So, even metrics spreadsheets must take a back seat if you haven't got a team.

Building the right team takes time. If you are planning to grow, say, a twenty-strong development team then you'll need to start thinking about this a good three months before full-tilt development is due to begin. If it’s due to being tomorrow, then your first job is to announce to senior management a three month slip. Good luck with that.

But remember, in your hast a very very important lesson: it is better to get the right person, four weeks hence, than the wrong one tomorrow, even on a programme of only fifty-two weeks. Hell, even on a programme of only twelve weeks! I would comfortably state that productivity variance between a good and a bad engineer can be, say fourfold - easily. And this might be the case in an average organisation. But, in fact, the extremes are so much worse they can't even be expressed as a ratio. I can quote multiple cases of engineers who exhibited negative productivity. That is, they absorbed more time in management and/or undoing their work, than they actually produced. So, do not take recruitment lightly, and do not fall into the trap of some senior management in counting one body as one body. Not all bodies are equal - as well we know from reading Cosmo.

I will not talk about recruitment or interview techniques - that is a black art all of its own. And I certainly won't pass comment on what makes a good software engineer. I know what's good for me! But I will single-out one class of engineer for special mention, because, well, they are special, and so often are overlooked as such. Test engineers. When it comes to building a solid well-rounded team, the lesson here is obvious (yet seemingly easy to ignore): Test engineers must be test engineers.

Possibly one of the closest things to an accepted axiom in the software world is that testing is essential. Moreover I would say that the considered implementation of robust testing principles is fundamental to the success of a programme (and so will be discussed later). As such, the testing function is just as important as the development function - if not moreso. But, here's the rub, very few engineers actually like (or want) to do it. For this reason, often weaker or junior engineers get put in testing roles. This is a mistake. Testing is a specialism. So, particularly for your test team lead, you need someone who understands testing, who 'gets' it, and - vitally - who likes doing it. This is the only way you will get value out of your test function. Passionate testers are a rare find. If your firm doesn't have anyone that fits this bill, recruit - hire an expensive contractor if you have to. It will pay. I promise.

Which brings me to my next topic. To contract ... or not to contract? That is the old chestnut of a debate I shall re-ignite shortly. Just as soon as I've dug my bunker...

Tis Seasonal to be Jolly ... Unproductive

2011-06-15T08:55:00.000-07:00

As I stated in an earlier post, my original intention was to document the tasks of a hypothetical software project against a 12-month timeline Jan through to Dec. If I were to do this of course then before getting into the day-by-day basics, immediately there would a lesson to learn. And that is, never ever plan to complete a programme at Christmas. Planning for the most critical stage of your project to coincide with your resource pool being decimated by festive absenteeism is planning to fail from day one. At least, this is the case for UK and many other countries. Elsewhere, it may be summer that will result in your most emaciated team. It may be both that are troublesome. The message here is beware seasonal variations and resource profiles lumpier than the turkey gravy. If there are external reasons that a deadline is on say December 24^th then you have two options. Preferably convince your programme management and stakeholders to shift it out by a month or two. But, if the date is truly an immoveable object, then plan to finish a month sooner and resource accordingly, and aggressively try to stick to it. This last part is important. Don’t let anyone in your team, especially yourself, develop the mindset that this extra month is ‘slack’. There is no slack. Try to cast the December date from your mind, because your team will be fogged with festival glee, largely absent in mind, if not entirely in body.

What is a Software Manager?

2011-06-06T07:25:00.000-07:00

Most large software programmes will employ a software manager and a software architect. It is wrong to assume that these roles are anything but wholly interdependent. In fact it is foolhardy to believe that you can be a manager without being an architect or an architect without being a manager. How can a manager define work packages, estimate development costs or plan a development schedule without understanding how the pieces fit together? How can an architect design a solution without understanding the constraints of time, budget and human resources? In fact, in my experience, the success of software development programme relies on a strong relationship between manager and architect, to the extent that they are largely interchangeable. You must never rely on one person with all the answers. When your manager’s on leave who’s going to keep the team in check? When the architect is offsite who’s going to answer design queries? Maybe against the perceived wisdom of organisation hierarchies, I have found that running a programme with two people at the top of roughly equal seniority, and largely overlapping responsibilities, produces the most effective results. Personally, I need someone to bounce ideas off and to validate (or rubbish) my decisions. Some people (like all the contestants on The Apprentice), will see this as weak leadership. But seeing this as a weakness is a weakness in itself, because being responsive to other people's views is a fundamental strength of a good leader.

On the last three major programmes I have worked on, there has been a natural manager/architect pairing, and it worked extremely well in all cases. In this regard it's worth considering the parallels that can be drawn with the practice of Pair Programming and the flat management structure suggested by the agile development method, Extreme Programming, and the benefits thereof. Of course, you don't have to set up your structure like this to be successful, and indeed the size of your team may only warrant one person to perform both roles. But for anyone who argues that pair leadership can never work then just consider Page/Brin or Gates/Allen. They did okay.

Turning more specifically to the role of the Software Manager - this will change over the lifecycle of a programme as depicted in the graph below (very roughly).

There are a few things to note. Firstly, yes, you will be overloaded at the start of the programme. There is virtually nothing you can do about it - there are just so many things to get started that you will need a hand in. But if you get those first couple of months right then you will afford yourself time to put your feet up at the end. (Of course, this is when senior management will start calling on your time for other programmes, so keep your head down).

The second point to note is that if you are the kind of software manager who knows his/her stuff then you will spend much of your effort – say up to 80% – in the early phase involved in the architectural design. During this phase the team size will be small – architects and team leads. Coordination of them should be minimal but you must get involved in as many design discussions as possible. Remember, if you are going to be held accountable for the development of this system, you sure as hell should want to be involved in major architectural aspects of its design.

Towards the end of the programme you will mostly be just keeping developers in check, tracking work packages and allocating bugs to be fixed. But don’t believe for a minute that you won’t be involved in architectural decisions right up to the end. During system integration and testing things will crop up that no one ever thought about that you will have to make a decision on in quick time. So you must have the system in your head.

The intention for this blog is to be more about the What than the How. About what to do when and not how to do it. But as it is about software managers it will encroach more into the how for the managerial aspects than the architectural. That is, it may go into some detail regarding how to perform a work-package estimation but not how to do UML modelling. (Lots has been written on the latter already).

What to do in week one will follow in future posts, but the point to take away now is: just because you have the word 'manager' in your title do not think you can distance yourself from the architecture.

A Case of Mistaken Agility

2011-05-31T03:24:00.000-07:00

There is a word that seems to be cropping up with ever greater frequency in my particular corner of the software world. That word is: Agile. This is odd, because my particular corner consists predominantly of lumbering mission-critical systems. A place where agility rarely treads. Yet, from upper-management, recruitment consultants and job specs alike, I'm hearing/seeing the word the 'Agile' a lot. I can't explain the recent surge of popularity - maybe a hard-hitting marketing salvo from the Agile Marketing Board - but I can't help being concerned that a lot of the interest is built upon a misunderstanding. Specifically, one that confuses the two quite different sentiments of 'being more agile' and 'using an Agile development methodology'. They are not the same thing. I fear the confusion arises because this is the first significant software movement with an overtly positive name. You don't, for instance, hear many project managers saying, 'oh, I think we need to be more Iterative' - or Extreme, or Waterfall(!). But, Agile, hey - who wouldn't want to be more agile? Well, in fact, despite its positive (and, admittedly, apt) name, it is only suitable for a subset of projects, just like any other methodology. Agile methods promote an environment that copes well with change, and that sounds great, but in doing so it promotes a culture that invites change. And this is the crux, because in general, change is the death of many a project and it should be a eschewed in all but those cases where it is entirely unavoidable.

To understand where Agile fits we need to look at the bigger picture. You can think of all development methodologies as on a continuum from predictive to adaptive. The further you move from the predictive end of the continuum, the less formal planning and process is employed. Agile methodologies fall at the adaptive end, so they can say less about what will happen in the future, but they are better equipped to deal with change. Take a moment to read the Agile manifesto:

We are uncovering better ways of developing software by doing it and helping others do it. Through this work we have come to value:
Individuals and interactions over processes and tools
Working software over comprehensive documentation
Customer collaboration over contract negotiation
Responding to change over following a plan
That is, while there is value in the items on the right, we value the items on the left more.

Glossing over the tragically ill-formed opening sentence (and trust me I'm having to hold myself back here), let's consider each of these statements in turn:

Individuals and interactions over processes and tools. I can quote many occasions that process (and tools to enforce that process) are merely a substitute for good engineers, so in essence I agree with this sentiment. But in practice it may be foolhardy to assume that every member of your team is such a high-calibre and well-rounded engineer that they will only ever do the right thing. Maybe the Googles of this world can afford to assemble such a team, but generally it is unlikely. Consider carefully how little process you can afford to do away with. Are you really not going to employ a change-management or code review process/tool just because you trust your guys?

Working software over comprehensive documentation. Obviously, the former is more valuable than the latter. Stating it in a manifesto is at best bland, but at worst it engenders a culture that considers documenting software as not important. It is important. Ask someone who has been asked to upgrade a 15-year-old system consisting of one million lines of code. But the real point here is that no compromise is actually necessary. With model-driven development the software and the documentation can be one and the same thing. Why should this not be a tenet in the manifesto instead?

Customer collaboration over contract negotiation. Customer collaboration throughout the development life-cycle is essential, but if your project is being funded by an external customer then you must have a contract specifying at least the key user requirements. Sure, the holy-grail is to sign a contract on an hourly-rate basis, where customer collaboration (and indeed procrastination) is just money in the bank, but that's why contracts are rarely drawn up this way and only between the friendliest of companies. I've seen companies lose millions over woolly contracts. In the real world it would be suicide to overlook the value of a solid contract with the promise of some jolly fun customer collaboration down the road.

Responding to change over following a plan. Having the ability to respond to change is unquestionably beneficial, but suggesting that change is valued more highly than a plan is a little suspect. There is no world in which regular change will result in a more efficient outcome than following a plan (and having no change). Change is absolutely inevitable, I accept that. But it should be minimised where possible rather than welcomed. The key, as with everything, is balance - knowing what elements are reasonably stable with high confidence and so knowing at what level of detail to plan.

Despite my misgivings in the wording of the manifesto, I am not against Agile development methods in general. But it is vitally important to be aware that such methods are only ever going suit a subset of projects. Agile methods may work well with small co-located teams (<10) of high-skilled senior members, but the more the structure varies from this, the less chance an Agile method is going to have of achieving greater agility, with a lower-case 'a'. In addition, I would argue that Agile methods are most suited to the development of products for a mass market rather than for projects with a small number of defined stakeholders.

As with all disciplines which offer extreme views, the right answer for any given scenario is almost always somewhere in the middle. With this is mind I intend to propose what I have named the Balanced Methodology, and I shall detail this over coming posts.

The Diary of a Software Manager

2011-05-25T07:18:00.000-07:00

Some time ago I had the idea that I would write a book about software management, to persist into a more concrete form the gems of wisdom gleaned from my experiences over the last decade in the software world. As much for me as anyone else - lest I ever forget. The idea was for the book to be structured like a diary of a theoretical 12-month software project. It would state, month-by-month, each activity that must take place and each artifact that must be created in order for the programme to run to completion successfully: on time, on budget and with minimum headaches. It would act as a step-by-step guide and checklist for any manager embarking on a software project. And as the principles covered would be general, so the stages it refers to would be scaleable. If a programme was to run for 24-months then you could simply double the length of the each of the phases detailed in the book.

This is still the plan, but I figured that along the way I would also post the content here in a more haphazard format, should anyone be interested enough to read and comment on it. Hopefully you will find something of use.

But before all of that, why should you care what I have to say about the art of software development? Well, by way of convincing you, let me first make you aware that I am an actual real fully-instantiated software manager. I am not an abstract-thinking head-in-the-clouds pure-virtual academic. After my computer science degree I went straight into industry and have been there ever since. I started out as a developer, soon became a team lead, then architect and manager. I have led the development of numerous large-scale systems, between 18 and 30 months in duration, with team sizes up to 30 people. And so far, they have all been 'successful'. And by successful I mean they have been completed in-line with the schedule and budget that I devised on my day one – not the schedule and budget imagined by project management during the bid phase, which in some cases can be quite a creative work of art.

Let me make clear that I attribute no success to any exceptional ability on my part - and those who know me can vouch for that - but simply to two factors. Firstly, being lucky enough to be told how to do things properly early in my career; and secondly, being blessed with a basic dose of common sense. I can do nothing to bestow upon you the latter, but I can at least help you out with the doing things properly bit. I hope you enjoy.