UAT: what is it for and who benefits from it?

When I was first working in IT in the late eighties, I remember one site where there was a cultural revolution taking place: they were going to start asking the users (or “the business”) what they wanted from IT systems.

Implied in this, of course, is a suggestion that the business took what they were given and that the IT department dictated who could have what. The truth, though, is a bit more subtle than that. Often the business wouldn’t have a clear idea what IT systems could do for them: they had no idea of what was possible and, of the applications that could be delivered, which ones constituted a ‘big ask’ and which ones were straightforward.

Over the last twenty years, the user base in blue chip companies has become increasingly familiar with business systems and it is more common now to see the business working hand in glove with the IT department. Consequently, a way of working has arisen, which includes gathering and documenting the users’ requirements before development starts, and, once development is complete, it is now accepted (and good) practice to give the users a chance to review the new work before it is put into their ‘live’ production environment, where it will be used with their real business data. This practice is called User Acceptance Testing or UAT.

For a completely new system the UAT environment will consist of an environment which is as close as possible to the proposed new live system, so that users can learn to use the new applications. For a change or upgrade to an existing system, the UAT environment should consist of a copy of the existing live system with the new work applied to it. This enables the users to see what has changed and what has stayed the same and also to see how the new processes cope with their existing data.

A word about data: this should, as far as is possible, be a copy of the live data. However, since a company won’t, for example, want its clients and customers receiving email from a test environment, it is necessary to ‘sanitise’ the data to some degree. Similarly, live credit card details should not be moved to UAT nor should the credentials for interacting with the live database or third party systems (such as payment gateways).

UAT provides three main benefits:

Firstly, it enables the users to reconcile what has been delivered against what was in their original requirements. It’s not unusual for there to be a form of ‘Chinese whispers’ as a requirement moves between the users, the business and systems analysts and the developers. The onus, of course, is firmly on the IT people to understand and keep sight of the original requirement and, if that is lost along the way, then the business have every right to refuse to sign off the release when they identify the omission in UAT.

Secondly, it provides an opportunity for ‘scenario based’ testing. A decent IT department will carry out thorough system testing against the functional specification (which is derived from the Business Requirements Document) but it is quite possible for this testing to be carried out properly and to sign the release off as fit for UAT yet for an error to be missed. This is best explained by example: a couple of years ago, we built an e-commerce system and one of the requirements specified that once items were sold, then the quantity should be subtracted from stock. We delivered this requirement, including automatic email notification when stock levels fell below a user defined trigger point. However, it was during UAT that the client pointed out that in order to minimise their costs, they held as little stock as possible, often ordering the required items in only after a customer had ordered them. Consequently, we had to amend the system to allow for negative stock values.

This brings me nicely to the last of these main benefits: less live fixes. If, in the example above, we had not had a user acceptance test, the client's business would have ground to a halt as many items would have not been available to order (as there were zero items in stock). The resolution to this problem was simple enough and took about half a day to apply. That half a day didn’t seem much when the client could be off testing other aspects of the system in UAT but it would have been a very different story if their business had lost half a day’s sales whilst we made the change.

So, who does benefit? As the points above illustrate, I believe the answer is everyone. The users have a huge amount of reassurance that their requirements have been met, without the stress of seeing the work for the first time in a live environment. Where issues do arise, they know they are low impact, affecting only test data and not the live business operation. From IT’s point of view, the department ends up with happier, better serviced users and, crucially, a minimum of high-pressure, risk laden live fixes with the business (understandably) demanding regular updates.

At Meantime, we are often working with users who don’t have blue chip experience but there is no reason why we shouldn’t use our experience and bring the good practice of UAT to our projects. The concept is easy to understand both in terms of execution and benefit, and our clients quickly grasp it. Ultimately, this simple step in the project life-cycle takes away a whole load of the stress and aggravation that is associated with IT delivery.

Case Study: Entrust Social Care

To make any case study worthwhile, it needs to highlight one or more salient points about the service that it is intended to illuminate. I'm starting with this particular case study because it demonstrates three important characteristics of well built bespoke software:

1. It has a clear cost benefit.
2. It improves the way in which the client's business is run.
3. It provides management information about the processes that it manages.

The client in this case is Entrust Social Care, whose website can be found at www.entrustsocialcare.co.uk. The Company provides temporary Social Workers (Locums) to the Public Sector across the UK, who in turn approach Entrust to satisfy their staffing requirements. One of the most important parts of Entrust's business is making sure that the locums are paid promptly after submitting their timesheets.

The locums are paid for the hours they work (sometimes working for different clients in the same week), for their expenses and they are also awarded bonuses. In addition to this, they might opt to take time off in lieu. Each week, the MD at Entrust, Ian Brindley, would process the timesheets submitted by the locums, using Excel to calculate the payments and track the time worked against bonus goals. It was a time consuming process and it was what Ian spent the Thursday and Friday of each week doing. It was boring work, yet vitally important, which is a poor combination. Furthermore, Ian didn't feel able to delegate the work out to his staff.

Ian and I had worked together at JPMorganChase in 2000, and he approached me to ask whether Meantime could do anything to help ease his situation. He explained that his business was developing well and that he was generally very happy: the only fly in the ointment was this weekly business with the timesheets. We spent some time talking to Ian about his business, the specific issue and about possible solutions.

This done we then designed a database that would store the details of both Ian's clients and the locums who were working through him. Over this we laid an administration system that would enable Ian to add and maintain this data. We then repeated this process, adding the relationships between the locums and the Social Work Teams in which they were placed.

The next step was to provide Ian with an easy to use interface where he could select a locum, choose a location where they were working and then enter the hours worked, expenses et cetera for a given week. Finally, we provided the function to produce a report of all the payments required for each locum.

As a consequence of this work, using the same inputs and providing the same outputs, we reduced the timesheet processing from two days to two hours (which is how long it took to type in the data). Additionally, we were able to provide valuable data from the system, simply because the relevant part of Ian's operational data was being processed by it. At the press of button Ian could access powerful business information regarding the number of contracts he had in place, the number of clients and locums he had on his books, plus vital financial information.

So, to summarise, let's look at those three points again:

1. The system has a clear cost benefit. Timesheet processing was taking up every Thursday and Friday, i.e. 40% of Ian's working time. Whilst I'm not privy to what Ian pays himself, I know that the cost of the software was less than two-fifths of his salary and, furthermore, it was a one-off cost.

2. The system improves the way in which the client's business is run. Once the system was in place, Ian suddenly had an extra two working days available in his week, something that would be hugely attractive to any Managing Director. This gave him more time to grow his business plus the confidence that he could manage the additional workload.

3. The system provides management information about the processes that it manages. Without even consulting Ian, common sense would have enabled us to provide useful reports to Ian. In addition to those mentioned above, we were in a position to answer questions like Who are my best clients? How much have I paid out to locums this quarter? Which care clients appear to be using more or less of my services over time?

All businesses are different and that is why they need bespoke software for their IT solutions. For those solutions to be effective, the businesses need to pick suppliers who are demonstrably strong when it comes to business analysis: there is a lot of work to be done before the first coding keystroke takes place. The Entrust project is a great example of how, by listening to the client and working with him, we were able to provide a solution that exactly matched his requirement.