Banking on failure

Over ten years ago I was working for a Major Retail Bank (MRB) and I was responsible for testing their first fully online Internet banking application. I had two teams of around twenty people, one in London for a sister bank and one at head office in what, at times, felt like the Arctic circle.

We had cabinet upon cabinet of manually completed test scripts as well as automated text tools checking the less popular browsers. In each location I had two or three test coordinators running the teams on an operational basis and I had an excellent boss, Cameron Mealyou, supporting me strategically. With all this in place, we delivered very high quality testing and the live launch went without a hitch.

A short time later, a minor amendment was required to the software, effectively a change to some help text. Perhaps on another, lower profiled project I might have agreed to some localised testing but this was an Internet banking system; we couldn't afford a single incident to undermine customer confidence. Thus, I proposed a full regression test, employing the full test team for a week, thereby causing a week's slippage to the delivery of the next release.

This suggestion proved to be unpopular with the programme manager and matters became heated. Having stated my case once I couldn't see the point in labouring the point, so I told the PM he was welcome to over-rule me but that I wouldn't sign off the change as tested. I think it was pretty clear that overruling me would have made my position untenable and he reluctantly agreed to the test.

The testing produced three significant bugs.

When the bugs had been corrected and the release was live, I was in the pub with the development manager, I nice chap called Neil. After a couple a beers, he said to me "You couldn't possibly have known those bugs were there." Which is an odd thing to say, isn't it? We don't test because we know there are bugs, we test in case there are (although that is usually the case).

A few months later I moved on to a new contract. I was told the test teams would be wound down "because we never have any bugs in the live environment." This is like saying "I think I'll stop exercising because I never have any problems with my fitness"!

I've been reminded of this a few times recently when I've encountered problems with my bank's banking software and also because of the high profile issues that have been reported in the press. These problems point - undeniably - to a lack of testing.

And it's not that testing is particularly complicated. It's mostly a combination of common sense, rigour and a conscientious approach (rather well suited to those who are a little OCD). It is, however, pretty expensive: there's a lot of preparation involved and the process itself is time-consuming. On many occasions I saw a project manager attempt to cut testing time in order to hit a deadline and this is a classic reason for bug-ridden live releases.

So, my message to the banks is: stop scrimping on the testing and support your test managers.

Systems analysis: the neglected step in the journey.

When I was first working in IT, I was used to coding from a specification. I was so busy struggling with Cobol, JCL and DL/1 - yes, I'm that old! - that I didn't stop to think about where these specs came from.

A couple of years later I was working for VSEL amending some programs (as we called them in those days) and having been told what we needed to do, we'd look at the existing code and the database and work out what to do, often coding as we went.

It was only on my first contract that I began to see a better way of doing things. There were some people on the development team who didn't write code but whose job was to write these specification documents that I remembered from my first job. We all sat in one open plan office and each developer worked with one systems analyst.

I found this tremendously interesting, especially as I began to get a say in how the programmes might work,  and over the course of my next couple of contracts I made the transition from developer to systems analyst.

At this point, you might well be saying to yourself "this is all very well but what's a systems analyst?"

Good question.

The business analyst collects the business requirements. So, for example, a new system might track books in and out of a library and each card holder is allowed to have three books at any one time for up to a fortnight. That's a nice simple piece of business analysis.

The systems analyst, though, has to design the system, the database and the logic. It's the systems analyst who will typically find the gaps in the business analyst's documentation, sending them scurrying to the client with questions like "but what happens when the card holder has had a book for a fortnight?"

It is this final specification that the developer will code from and, whilst the developer has some leeway in exactly what they build, it is that specification written by the systems analyst that determines what will be tested. The programme should do what the specification says and no more.

Of course, this additional step appears to be very time-consuming and it is but, as is so often the case when a company does things properly, it pays dividends. The approach that simply sets the developer coding with a bunch of requirements often means that the developer will get halfway through before realising that he needs to go back and change something he's already written. It means that the client isn't suddenly presented with quite fundamental questions during the build process, which may be a little unnerving. And, in a constructive way, it presents a contract between the analyst and developer, who have agreed what is being built. If the requirements change and the spec needs amending, the developer can very reasonably ask for more time.

Furthermore, the specification forms an excellent basis for system testing, as it details precisely what the system is supposed to do.

As a business, Meantime prides itself on delivering on time, on budget and to specification. The systems analyst's role is key in helping us to keep this promise to our clients.

The important art of business analysis

"Hello. Is that the garage?"
"Yes, it is. Good morning. How can I help you?"
"I'd like to buy a car, please?"
"Great. We'd certainly be pleased to help you with that!"
"Oh good. Can you tell me how much it will cost?"
"Er..."

Well, OBVIOUSLY, you wouldn't go into buying a car like that (although I have in the past wished that you could!).

Of course, what you would actually do is tell the garage your requirements. Some of those might be quite specific: you might want a diesel engine or a sunroof. You might also provide a bit of background information, such as the fact that you have three children. This will give the salesperson an opportunity to add some value, as he (or she) might ask, for example, whether you want rear doors with childproof locks on them. And, of course, no sale would be complete without an attempt to sell you a few bits and pieces that aren't strictly necessary, like alloy wheels.

I'm mentioning all this because this current instalment in why IT projects fail concerns the Business Analysis stage. Now this isn't strictly sales - that's where my analogy breaks down - but a good business analyst will collect information not only about your specific requirements but also your business.

This enables the analyst to do three things: firstly to contextualise your requirements. Secondly to bring their experience of similar companies and projects to bear and thereby add value. And, thirdly, to ensure that the development fits in with your requirements strategically. After all, there's no point in buying a five-seater if your family is expecting a fourth child!

So, a business analyst needs to be a good listener. They need to ask you about your business, where it's come from, what it's doing now and where it's heading. The analyst needs to understand what systems the company uses now and whether any new software will be required to integrate with them.

Next the business analyst needs to listen to your requirements, using their experience to understand whether what you're asking for matches your stated intentions. Here their experience of other projects and companies will be of value, possibly preventing you from making mistakes and helping you to benefit from the experiences of other people.

The business analyst should also be able to make suggestions about what other options you might want to consider doing that aren't currently part of your plans. The cross-pollination of ideas across sectors is not always obvious but a good analyst will be able to spot them.

Once you have had this meeting - or possibly meetings - the business analyst should provide you with a business requirements document, which should detail the scope of the project and the requirements that you want satisfied, whilst also recording any discussions around future development. The document might include some design work but, really, it should be a summary of your business requirements.

Once you've approved this document, the company for whom the business analyst works should be able to give you a reasonably accurate estimated cost for the work. However, I wouldn't expect them to quote for the project until the more detailed analysis is complete and it wouldn't be unreasonable for their company to expect you to pay for this detailed ("systems") analysis, which will take days or even weeks. Next time I'll write about systems analysis and that stage of the project.

I think it should be fairly clear from what I've written here just what the issues are that might arise if the business analysis is not done properly; a client who ends up with a system that is not right for their business.

Scoping your IT project.

Wow! It's been a while since I was last here. Happy New Year! The last six months have flown by as we've acquired new projects, clients and colleagues. Plus, as readers of our newsletter will know, we've spent the first three months of this year rationalising our servers onto a single cloud-based platform. This has been an enormous undertaking and I'm absolutely delighted by its success, which was managed with absolute minimum disruption to our clients.

But in my last post I was starting a series of posts about why large IT projects fail. I had originally written a list of all the items I want to cover and this had gone missing. After I started sketching out this post I found it again and I was pleased to see I'd been consistent in what I see as the first thing to consider, and that is scope.

When I'm talking about IT or trying to explain it to non-technical folk, I often find that talking about building is a good analogy. So, let's imagine that you're going to build an extension for a new kitchen.

First of all, you need to consider the rest of your house. You might not be doing any work there but there's plumbing and electricity to consider, and the stresses on the existing infrastructure. This, then, is not the time to discuss whether the bathroom needs renovating although if that's being done at the same time, you need to make sure you aren't moving a pipe that you're planning to feed into the kitchen.

Similarly, then, for your new project, before you start talking about what it needs to do, what constraints are there? If it has to exchange information with other systems, how will that work? Have you checked whether those other systems have their own changes planned?

Next, focus on the essentials. What needs to to be done in this initial project? It might be sensible to defer considerations such as floor tiles, painting and decorating, and kitchen furniture. Sure, the kitchen might not be usable until these are in place but putting those to one side, you can focus on a manageable amount of work that is quite different in its nature. Your joiner is unlikely to want to sit in on a conversation about Italian tiles.

So, for your IT project, look at what needs to be achieved. Yes, you might want a letter printing facility but that's academic if your client data isn't right. Of course, you want to know that that will be a requirement at some point - you want to bear these things in mind - but for now you need to focus on the core deliverable. This enables you to keep your team smaller, which facilitates communication and eases change management. It also means you are less vulnerable to changes in requirements - you have a smaller target - and when those do arise, impact analysis is simpler.

Once you have reached this point, you can start getting into the detail of what you're building. Even on a project that has been scoped carefully, it's astounding how much there is to be done once you get into the detail. I think this is where a lot of big projects go wrong: a sensibly scoped project can look a bit unambitious, a bit limited in what it's delivering, so people think they might as well do a bit more while they're at it. Once you get into the detail., it's suddenly unmanageable. In 2004 I was on a project based in New York that was on a three month delivery cycle yet consistently failing to deliver. Really, all I did was cut the scope right back. There was a certain amount of scorn on the project, not least from the development team, who, I seem to remember, thought we could do it all in two weeks. We did it just in time for the next release date. We repeated the process twice more and, when I finished we had a successfully delivering project and a happy user community. It wasn't rocket science.

One final consideration. In our example, we're building a new kitchen but when that's done, we'll need to move all the kitchenware across from the old one. And we need to make sure there's room for that large steam-punk coffee machine. In system terms, I'm talking about data migration. You need to ensure the new system can handle your existing data and have a plan in place for moving it across. And rehearse that process, too. The worst project I ever worked on hadn't rehearsed its data migration. With a new system going live on the Monday, they kicked off the data migration on Friday night. By Sunday afternoon, only half the data had moved across. Rather than aborting, Monday saw this telecommunications company based in Britain with half its accounts on one system and half on the other. It was a disaster, operationally and from a PR perspective.

That's my guide to scoping then. Think about the big picture, don't be over ambitious. Simple, eh?

Why do large IT projects fail?

One way or another, pretty much everything I’ve done and written about for the last few years has been based on a single premise and that is that IT can be done well and that projects can be delivered to specification, on time and on budget.

Delivering websites and IT systems via a small business has only emphasised – quite painfully at times – the thoughts and experience I had gained over the previous fourteen years working freelance on blue chip IT projects.

Broadly speaking, there are two reasons IT projects fail. One of those reasons is the people working on the project and the other is the way in which the project is organised and run. Many of my subsequent posts will be about the latter but, just briefly, I’d like to touch on the former.

I’ve been in IT for twenty-three years now and every successful project I’ve worked on has had a high proportion of The Right People. The most successful project I’ve worked on was exclusively staffed by people of that calibre.

At Meantime, we have struggled with recruitment. This is partly down to where we are based – the one downside of having our office in the Lake District – and partly down to the fact that IT is not a proper profession. There are no widely accepted formal qualifications – particularly around systems development – and so recruitment rests on the shaky platform of CVs and interviews.

Over the years we have always recruited with optimism and looked to bring out the best in people and, at times, we have been badly let down. We’ve now settled on a recruitment formula, which is as follows: I interview the candidate and Steve, our lead developer, gives them a technical interview. (Steve is also a pretty shrewd judge of character.) If we both like them and, crucially, even if they are not technically quite up to scratch, we ask them to carry out an online psychometric questionnaire, which is followed up by an interview with a trained assessor. We then receive a report and, whatever, the outcome, carry out a third interview.

As you can tell, just from that brief description, it’s a laborious and expensive process. However, experience has shown us that it’s far more cost effective to go down this route than to put the wrong person onto a project, given the long term damage this can do.

I’ll illustrate this point with one brief example. Several years ago we did a project that involved classes and terms. The developer in question, let’s call him Paul, was given a data model to work from and was walked through the use of that data model. At the time, he queried the way terms and classes were related and the database structure and process were explained in detail.

In the end, however, and without consulting any of his colleagues, Paul decided to do the work the way he thought best. The project went into user acceptance test and, as we moved towards the first change in term, it became apparent that Paul’s solution wouldn’t work. I don’t know whether Paul had realised this earlier, certainly he handed in his notice around the time the issue became apparent and left his ex-colleagues to put things right.

You can imagine the stress this put on the company. The issue was not the client’s fault, so there was no extra funding, and we had other projects in progress. Being a small business, we didn’t have any spare staff, so Steve, myself and another colleague, Mary, worked extra hours to turn this around.

The issue here was not so much that Paul had coded the solution wrongly, it was that he decided he knew better than the person (me!) who had talked to the client and written the specification. Even then, the problem was not so much that Paul had his own opinion, it was that he didn’t discuss his intention with anyone.

These days we recruit people whom our interviews and the psychometric test identify as team players, people with empathy for our clients, who want to deliver the best solution for them. Following this method, we’ve employed people who didn’t have our skill set or the right experience, who have turned out to make a brilliant contribution to our team. Ultimately, you can teach people skills and give them experience, what you can’t do – at least, not very easily – is change their nature.

OK, so that was a little less brief than I intended and the calibre of the people contributing to projects will certainly crop up again in my following blogs, but the point I’m making is that all of the other things I’m going to write about, those things that contribute to a project’s success, will not work without the right people.

Despite what they say, recruitment agencies don’t filter candidates, except in the very broadest terms. If a candidate’s skillset matches the job requirements, the agency will forward the CV. Many claim to have interviewed clients and given the percentages that agencies demand, one would expect they have spent serious time vetting them. However, in my experience at least, that is simply not the case.

It’s not enough to like someone at interview or to hire them because they pass a technical interview. For a project to succeed you need people who are team players, people who care about a project’s success, people who want to make your clients happy.

Last weekend, we carried out a data migration. Without being asked, the developer involved emailed me to say he’d be available if needed over the weekend and another colleague, who wasn’t directly involved, rang me over the weekend to check everything had gone to plan. When people take this level of interest, when they care about the outcomes, when people like this are working for you, then, with the right process and organisation, you can deliver successful IT projects.

User interface: The 5% that's really 100%

We have a saying in our office, which is that the interface is only 5% of what we do but 100% of what the user sees. To be more accurate, it's usually me who says it, typically to a background chorus of grinding teeth. This is not to say that the rest of the team don't care about the user experience, but when, for example, you've just coded a complex administration function to enable to a client to set up their own algorithms for customer discounts, you're probably expecting a bit of praise and congratulation, and not someone scratching his head and asking whether the submit button shouldn't be a bit further up the screen. (There's a lesson for me, there.)

However, there are two very good reasons for thinking about the user and the interface that they will use.

Firstly, if you've built a system, then you want people to use it and there are four key components to giving them an experience they will be happy to repeat.

1. Visual appeal: People make very rapid, non-intellectual decisions about websites that they are presented with. If the screen is cluttered or badly rendered, the user is already less inclined to engage with it. Screen design is hugely important; it makes the system look like it will work.

2. Guidance: if your user has to study your screen to work out where to start, then you've already failed. It should be clear to the user what they need to do first or what their options are. Positioning the cursor for them, big, clearly labelled buttons, numbering steps: these all go a long way to giving the user the 'no brainer' experience that they want. And a little onscreen text is a simple, cheap way to help the user to understand what's required of them and what's going on.

3. Feedback. Our local tourism board has a purchasing system that it allows some attractions to use. Booking tickets for a show, I selected the number I wanted and clicked add to basket. Nothing happened so I tried again, twice more. Still nothing. So I went to the checkout only to find I had enough tickets in my basket to take most of the people living on my street to the theatre with me. I think the message here is clear; when the user completes an action, let them know it's done!

4. Deference. With software, you can, of course, force users into doing things by preventing them from proceeding if they don't do what they're told. However, this may not give you the results you want. It's all very well collecting, for example, marketing information from your users but if you force them to enter a date of birth, you may find they enter something completely random, thereby skewing your data. In a similar vein, an e-commerce site lost my business this week when they deemed my nine-character-with-a-numeric password to be not strong enough for their standards. Don't throw your weight around: give your users what they want and need, and if you want something from them, ask don't demand.

Secondly, there is a very selfish reason for helping your user to have a good experience with the software you build; fewer support calls. If people need to use the software - let's say it's for a timesheeting system - then if they can't do what they need to do, they're going to call. And, people being people, they probably won't read lengthy help text or instruction guides: they'll give up or pick up the 'phone. One way or another, that will come back to the software provider.

Usability, particularly the second point, can be tested easily during your User Acceptance Testing (UAT). The client knows what they want the system to do, the software provider has built that system. Leaving the client to test the system without initial guidance from the provider puts the client in the same shoes as the user. If the client can't get from A to B or product to checkout without guidance from the software provider, what chance has the user got, when they come to the website or software completely cold?

Usability is about empathy, putting yourself in the user's shoes: put the little bit of effort into giving them some guidance through your design and text, and you'll have happy users and a quiet help desk.

Security: it's not rocket science

This morning the news broke that Sony has announced that hackers have stolen the details of millions of online video gamers. The Telegraph's report can be seen here. The data includes user names, passwords and, possibly, credit card details.

I'm surprised by this for two reasons. Firstly, the fact that Sony were successfully hacked at all. God knows there are some very, very clever (if misguided) people out there involved in hacking. Anecdotally, a couple of times a year we see evidence of people attempting to hack our servers and we work hard to stay on top of our security. But we're not Sony. Surely a company as wealthy as Sony, responsible for the details of millions of people, should be employing the very best people - really: the very best - to safeguard their systems? The fact that they were breached suggests to me that they are not taking their security seriously enough.

However, the greater part of the surprise, for me, is that it seems store their data in an unencrypted state. I've blogged about this before but for all the times that disks go missing in the post, laptops are stolen or security is breached, no spokesman ever says "but it's OK because the data was all encrypted".

For our most secure data we use a combination of the encryption algorithms built into the database software but also some bespoke algorithms of our own. Thus, even someone with open access to our database couldn't interpret the information that is stored. I don't think we've done anything mind-boggling there; I'm sure most IT companies worth their salt would come up with a similar solution given the same requirement for data security, which just makes me wonder why Sony didn't.

Incidentally, if you have data that you want to encrypt, either on your hard drive or portable data device, I highly recommend TrueCrypt. It's very simple to use and very secure.