Banking on failure

Over ten years ago I was working for a Major Retail Bank (MRB) and I was responsible for testing their first fully online Internet banking application. I had two teams of around twenty people, one in London for a sister bank and one at head office in what, at times, felt like the Arctic circle.

We had cabinet upon cabinet of manually completed test scripts as well as automated text tools checking the less popular browsers. In each location I had two or three test coordinators running the teams on an operational basis and I had an excellent boss, Cameron Mealyou, supporting me strategically. With all this in place, we delivered very high quality testing and the live launch went without a hitch.

A short time later, a minor amendment was required to the software, effectively a change to some help text. Perhaps on another, lower profiled project I might have agreed to some localised testing but this was an Internet banking system; we couldn't afford a single incident to undermine customer confidence. Thus, I proposed a full regression test, employing the full test team for a week, thereby causing a week's slippage to the delivery of the next release.

This suggestion proved to be unpopular with the programme manager and matters became heated. Having stated my case once I couldn't see the point in labouring the point, so I told the PM he was welcome to over-rule me but that I wouldn't sign off the change as tested. I think it was pretty clear that overruling me would have made my position untenable and he reluctantly agreed to the test.

The testing produced three significant bugs.

When the bugs had been corrected and the release was live, I was in the pub with the development manager, I nice chap called Neil. After a couple a beers, he said to me "You couldn't possibly have known those bugs were there." Which is an odd thing to say, isn't it? We don't test because we know there are bugs, we test in case there are (although that is usually the case).

A few months later I moved on to a new contract. I was told the test teams would be wound down "because we never have any bugs in the live environment." This is like saying "I think I'll stop exercising because I never have any problems with my fitness"!

I've been reminded of this a few times recently when I've encountered problems with my bank's banking software and also because of the high profile issues that have been reported in the press. These problems point - undeniably - to a lack of testing.

And it's not that testing is particularly complicated. It's mostly a combination of common sense, rigour and a conscientious approach (rather well suited to those who are a little OCD). It is, however, pretty expensive: there's a lot of preparation involved and the process itself is time-consuming. On many occasions I saw a project manager attempt to cut testing time in order to hit a deadline and this is a classic reason for bug-ridden live releases.

So, my message to the banks is: stop scrimping on the testing and support your test managers.