For what ought to be a fairly precise industry, computing is fairly weak in a few semantic areas and one of those is the term 'database'. A database is, essentially, any store of data: it might be an Excel spreadsheet of customer names or even a Word document of contact addresses.
More properly, we use the term to describe a formally structured or organised datastore of information. These datastores are created, managed and accessed by specific software, the most common example being Microsoft's Access.
The data is stored in different 'tables' and these are linked together using the data items. So, for example, customer names and details might be on one table while the customer addresses are on another. Each address is on a separate row and has a unique reference. This reference is also held on the customer rows to which it applies.
The example above consists of two tables but a small business system will comfortably have ten times that number. Indeed, database design is an art in itself. When we take on a new project and the initial business analysis is complete, we well sketch out a data model - an initial design for the database - to check our understanding and identify areas requiring further analysis. Over the years we have found that once we have the data model right, many aspects of the system we are constructing will fall into place.
As well as being the foundation of a new system, the database remains the most important feature. Screens can be badly designed, load times may be poor, navigation may be unintuitive but these are all faults that can be fixed and, once fixed, there will be no sign they were any different.
Let me break out here and tell you a story from earlier in my career. I was working as a test manager on a migration project for a large financial institution. All of the existing data - clients, accounts, balances etc - was being moved from a system that was twenty years old onto a new one. My job was to make sure that all the data transferred correctly from the old system to the new one.
Everything went well in testing and the time came for us to do a trial migration. As part of the migration process, we generated a lot of reports, so we could tally the number between the original system and the new one: number of clients, number of accounts, total value and so on. We even threw in some logging with no immediate business value - average number of accounts per client, mean total per client - just so we could make sure all the numbers matched.
The testing had been thorough and I was very pleased with what we'd done to date so it's no exaggeration to say that I was horrified when our test run against live data returned results best described as nonsense. I came clean at the project board meeting and assured my colleagues that we'd get to the bottom of the matter.
Two weeks went by and the results against our test data continued to be spot on, as you would expect, but the more we delved into the live data, the more confused we became. Armed with a set of inexplicable examples I returned to the project board and a member of the user group came clean: at some point in the system's history, there was a bug in the code that meant that the relationship between a client and their accounts was not recorded properly. Without these relationships in place, of course, we had no way of doing a meaningful comparison between the old system and the new one. Furthermore, we had to - effectively - build bugs into the new system to accommodate this 'junk' data from the old one.
Quite apart from the importance of testing, this story emphasises the fact that if there is one aspect of an IT system where you cannot afford to make mistakes, it is with your data. Once data is lost or corrupted, there is no easy way of correcting the problem. Indeed, having 'holes' in your data can commit you to a future of weak software, where the processes and validations have to be weakened to compensate for the mistakes that you have made in the past.
When businesses change hands, websites and IT systems are assets of the sale and so, of course, is the data. If you aren't selling your business, then your data remains at the heart of what you do. Either way, it is vital that you take care if the information that is at the heart of what you do. Here are my recommendations for doing just that:
- Make sure that you, your IT department or your IT provider understand your data.
- When building a system, ensure that your data is captured in an appropriate structure.
- Back up all your data daily.
- When you make changes to the system, particularly - but by no means exclusively - to your data model, then insure that you carry out the appropriate amount of regression testing to ensure that your data is not adversely affected.
Prevention is better than cure, but what do you do if you have data that is corrupted? You need to cater for the existing data that is substandard but that is no reason to weaken the processing around your new data. The simplest measure to take is to flag your old data so that validations know when to apply strict rules and when to apply weaker ones. An example would be if you insist that anyone registering with you has a post code but this rule was, at some point, not enforced. By flagging those addresses with no post code as, say, 'weak data', you can ignore the post code validation when processing those addresses, whilst being as rigorous as you like with data that is not affected.
The bottom line here is to appreciate your data. Your understanding of your business almost certainly maps down to the data (whether you are aware of that or not!) and you must make sure that the people who build and manage your systems take the time to listen to you and thus share that understanding. If your data is corrupted or lost, that is equivalent to losing a filing cabinet in a fire, so you need to protect the data you have and treat it with respect. I think I have mentioned before that of the businesses located in the twin towers, a third went under after 9/11 because they had no backups of their business data.
Sunday, 29 November 2009
Wednesday, 21 October 2009
Mind the gap! The Ecommerce Expo 2009.
A few years ago I set my parents up with a broadband connection via Demon Internet (now Thus plc). As they live in London, they were able to get a fast service - at least for the time - and they never had a problem with it: they went to the PC, the broadband was available and worked without a hitch. About eighteen months ago, however, my father saw an advert for TalkTalk, offering broadband as well as 'phone services for only £6.99 a month, which was a third of what he was paying Demon.
Perhaps I was lazy or perhaps I didn't feel up to trying to argue against TalkTalk's excellent advertising campaign but I rolled over and said it sounded like a good deal, so why not go ahead? As you might have already guessed, there followed weeks of 'phone calls to support numbers, promises to call back that were never fulfilled and a very unhappy father. Even now, over a year later, the service is patchy and more akin to what one might expect in a cottage in Cumbria, five miles from the nearest exchange.
So, having set the example of how the marketing of a product can be very different from the actual delivered goods or services, let me turn my attention to this year's Ecommerce Expo. Yesterday I went down to Earls Court* for the day to catch up with where the sector is regarding Ecommerce and its related industries.
There were certainly some interesting ideas around payment methods, although nothing that struck me as an alternative worth considering to SagePay, which is our preferred solution for ecommerce payments. One apparently global company I spoke to even seemed surprised at the notion of validating cards but not taking payment until shipping. Have they never used Amazon?
My journey down was certainly made worthwhile by the invaluable twenty minutes I had with a technical advisor from Rackspace, who verified some of my concerns about cloud computing and made me think that we'll revisit that next year. The alternative solutions that he offered,though, were sound and intelligent. As a company, they continue to impress me.
Of course, I spent a lot of time visiting the stands of our competitors, most of whom were selling package solutions. (Dydacomp were there touting their MOM product, although they seemed to have less angry customers around their stall than last year, possibly because they have belatedly managed to release a PCI Compliant version.) What was notable about all the stalls - bar one, which I'll come to in a moment - was the fact that they all seemed to be offering the same things - a better customer experience, better sales, email marketing, SEO (of course), more effective order processing and so on - but there was very little in the way of customer testimonial. I must say I was very envious of the brand management company who had a glowing reference from Sainsburys and the PayPal stand predictably had some impressive names and logos on it but they were the exception.
However,I know two or three of these testimonial-free companies by reputation (from their ex-clients), and I know that the service they offer is not always as slick as they would like. And, to be clear, I think that's true for any small business. The problem is, then, this gap between the marketing and the deliverable. Spend enough money with a marketing firm, rent sufficient floor space at the Expo and your company can look like a million dollars. But, if everyone is saying the same thing, what's the point?
What was the one stall that was different? UKFast had by far the largest stand and they had set up a Formula One car with a plasma screen in front of it, so vistors could play a racing game. They backed this up with a bar and half a dozen attractive young women dressed in shorts and high heels. My knee jerk reaction was "how passé" but ultimately if there was a game to be played at the Expo of having the most attractive stand, then rather than big posters with a load of claims they might or might not be able to deliver on, UKFast went straight for the jugular and they "won".
I wouldn't claim to be the first to highlight the gap between the well-functioning marketing department and the product or service being delivered but it did strike me yesterday that this year's Expo was all about saying what your company could do without really backing it up. In a way, this reflects the issue we had with our own website: when there are so many sites out there telling you that their company can build you the best website, ecommerce solution, SEO etc etc, then how do you compete and stand out? Surely the only differentiator is the testimonials page that tells you not what a company says it can do but what it has actually done. Here's ours: www.meantime.co.uk/testimonials.php.
*I really will send a box of chocolates to anyone who can explain the apostrophe anomaly between Earls Court and Earl's Court Road.
Perhaps I was lazy or perhaps I didn't feel up to trying to argue against TalkTalk's excellent advertising campaign but I rolled over and said it sounded like a good deal, so why not go ahead? As you might have already guessed, there followed weeks of 'phone calls to support numbers, promises to call back that were never fulfilled and a very unhappy father. Even now, over a year later, the service is patchy and more akin to what one might expect in a cottage in Cumbria, five miles from the nearest exchange.
So, having set the example of how the marketing of a product can be very different from the actual delivered goods or services, let me turn my attention to this year's Ecommerce Expo. Yesterday I went down to Earls Court* for the day to catch up with where the sector is regarding Ecommerce and its related industries.
There were certainly some interesting ideas around payment methods, although nothing that struck me as an alternative worth considering to SagePay, which is our preferred solution for ecommerce payments. One apparently global company I spoke to even seemed surprised at the notion of validating cards but not taking payment until shipping. Have they never used Amazon?
My journey down was certainly made worthwhile by the invaluable twenty minutes I had with a technical advisor from Rackspace, who verified some of my concerns about cloud computing and made me think that we'll revisit that next year. The alternative solutions that he offered,though, were sound and intelligent. As a company, they continue to impress me.
Of course, I spent a lot of time visiting the stands of our competitors, most of whom were selling package solutions. (Dydacomp were there touting their MOM product, although they seemed to have less angry customers around their stall than last year, possibly because they have belatedly managed to release a PCI Compliant version.) What was notable about all the stalls - bar one, which I'll come to in a moment - was the fact that they all seemed to be offering the same things - a better customer experience, better sales, email marketing, SEO (of course), more effective order processing and so on - but there was very little in the way of customer testimonial. I must say I was very envious of the brand management company who had a glowing reference from Sainsburys and the PayPal stand predictably had some impressive names and logos on it but they were the exception.
However,I know two or three of these testimonial-free companies by reputation (from their ex-clients), and I know that the service they offer is not always as slick as they would like. And, to be clear, I think that's true for any small business. The problem is, then, this gap between the marketing and the deliverable. Spend enough money with a marketing firm, rent sufficient floor space at the Expo and your company can look like a million dollars. But, if everyone is saying the same thing, what's the point?
What was the one stall that was different? UKFast had by far the largest stand and they had set up a Formula One car with a plasma screen in front of it, so vistors could play a racing game. They backed this up with a bar and half a dozen attractive young women dressed in shorts and high heels. My knee jerk reaction was "how passé" but ultimately if there was a game to be played at the Expo of having the most attractive stand, then rather than big posters with a load of claims they might or might not be able to deliver on, UKFast went straight for the jugular and they "won".
I wouldn't claim to be the first to highlight the gap between the well-functioning marketing department and the product or service being delivered but it did strike me yesterday that this year's Expo was all about saying what your company could do without really backing it up. In a way, this reflects the issue we had with our own website: when there are so many sites out there telling you that their company can build you the best website, ecommerce solution, SEO etc etc, then how do you compete and stand out? Surely the only differentiator is the testimonials page that tells you not what a company says it can do but what it has actually done. Here's ours: www.meantime.co.uk/testimonials.php.
*I really will send a box of chocolates to anyone who can explain the apostrophe anomaly between Earls Court and Earl's Court Road.
Wednesday, 30 September 2009
31 flavours of testing
So, as promised, here is the post about the different types of testing. Maybe there aren't thirty-one but there are perhaps more than you might expect, although I think all the ones detailed here will make absolute sense to you.
The first three derive directly from the documentation that is used for large projects but which also has its analogues in some smaller applications. The relevant pieces of documentation are: the business requirements document; the system specification; and the technical specification. I will talk briefly about each of these in turn:
Business requirements document: As its name implies, this document describes the user's requirements. It is a scoping document for the project and describes, at a high level, all of the processing and functionality that is required.
System specification: This describes in more detail how the system will be built, perhaps including screen mock-ups, business rules and a database schema.
Technical specification: This is the document that is used by the developer. It will include specific rules around input validations, how the database should be updated and so forth.
The three main branches of testing - certainly the three that are most commonly used - correspond to each of these three documents. (Sometimes they are shown as a "V model" with the documents on the left arm and the corresponding testing on the right.) These tests are:
Developer testing (sometimes known as 'unit' testing): as its name suggests, this is testing that is carried out by the developer. This is the most basic form of testing yet, in many respects, the most important. Indeed, there is an old IT adage that says that bugs are ten times more expensive to fix for each step down the testing path they remain undetected (not least because of the regression testing involved (see below)).
At this stage the developer should test all the screen validations and business rules relating to the screens on which s/he is working. They should also check that the reads and writes from and to the underlying database are working correctly.
It's worth re-emphasing the point that any issues missed at this stage will slow down later stages when they are discovered, returned to the developer for fixing and then the testing is repeated.
System testing: Once the developer testing is complete, then the entire system can be pulled together for an end to end test. This form of testing is more scenario based and runs along the same journeys that will be used in production. So, for example, testing an e-commerce appliaction, a system tester would add products and prices, then pose as a customer and make purchases and then ensure that the orders are recorded and stock levels are amended correctly.
User acceptance test: I have blogged about this in some detail before, so I will just say that this is the testing where the user can ensure that what has been delivered matches the brief.
So, if those are the most common forms of testing, what other types might you come across? I have described half a dozen others, below:
Implementation testing: moving code and database changes through test environments needs to be a closely managed process but the move into a live environment can be slightly different and therefore needs separate testing. So, for example, in an e-commerce application, the live transactions to the bank will only run once the software is live. This means this part of the process can only be tested in the production environment.
Regression testing: A test cycle - i.e. a series of related tests - is invalidated as soon as any of the components that were tested is changed. Of course, sometimes it is necessary to change components - if a bug has been found or if the user requests a change to the process - and then the affected test cycles need to be re-run and this is called regression testing.
Volume (or 'bulk') testing: As more and more data is added to a system, so performance begins to change: database response times may slow, screen load times can be affected and lists may become unmanageable. Sometimes these issues can be managed as a system grows but if a change is being released to a large, existing customer base, then it is essential to test against high volumes of data.
Load testing: this is related to volume testing (indeed, the two are sometimes combined for operational acceptance testing or OAT). Load testing involves having many, many users accessing the system simultaneously. This can be difficult to simulate and there are specific tools - such as Astra Load Tester - that can be used (at some expense!).
Automated testing: Sometimes the same test cycles need to be repeated over and over again. An example would be testing a web application against many different operating systems and browsers. There is a high overhead to automated testing; test scripts must be changed to mirror any system changes and, of course, the scripts need testing themselves. However, it does have its place.
Using reports for testing: Sometimes a system can, in part, be used to test itself. If a system has a decent reporting function, then that can be used to check that the system is correctly recording its own activity. So, if the testing started off with twenty widgets in stock and seven have been 'sold' in the tests, then the stock report should show thirteen left. If it doesn't, then either the system or the report needs debugging.
Part of the skill of testing is understanding the appropriate tests for an application: a simple testing might not need a separate system test for example. However, two types of testing should always take place before a change is put live: the testing done by the development team and the UAT carried out by the client.
The first three derive directly from the documentation that is used for large projects but which also has its analogues in some smaller applications. The relevant pieces of documentation are: the business requirements document; the system specification; and the technical specification. I will talk briefly about each of these in turn:
Business requirements document: As its name implies, this document describes the user's requirements. It is a scoping document for the project and describes, at a high level, all of the processing and functionality that is required.
System specification: This describes in more detail how the system will be built, perhaps including screen mock-ups, business rules and a database schema.
Technical specification: This is the document that is used by the developer. It will include specific rules around input validations, how the database should be updated and so forth.
The three main branches of testing - certainly the three that are most commonly used - correspond to each of these three documents. (Sometimes they are shown as a "V model" with the documents on the left arm and the corresponding testing on the right.) These tests are:
Developer testing (sometimes known as 'unit' testing): as its name suggests, this is testing that is carried out by the developer. This is the most basic form of testing yet, in many respects, the most important. Indeed, there is an old IT adage that says that bugs are ten times more expensive to fix for each step down the testing path they remain undetected (not least because of the regression testing involved (see below)).
At this stage the developer should test all the screen validations and business rules relating to the screens on which s/he is working. They should also check that the reads and writes from and to the underlying database are working correctly.
It's worth re-emphasing the point that any issues missed at this stage will slow down later stages when they are discovered, returned to the developer for fixing and then the testing is repeated.
System testing: Once the developer testing is complete, then the entire system can be pulled together for an end to end test. This form of testing is more scenario based and runs along the same journeys that will be used in production. So, for example, testing an e-commerce appliaction, a system tester would add products and prices, then pose as a customer and make purchases and then ensure that the orders are recorded and stock levels are amended correctly.
User acceptance test: I have blogged about this in some detail before, so I will just say that this is the testing where the user can ensure that what has been delivered matches the brief.
So, if those are the most common forms of testing, what other types might you come across? I have described half a dozen others, below:
Implementation testing: moving code and database changes through test environments needs to be a closely managed process but the move into a live environment can be slightly different and therefore needs separate testing. So, for example, in an e-commerce application, the live transactions to the bank will only run once the software is live. This means this part of the process can only be tested in the production environment.
Regression testing: A test cycle - i.e. a series of related tests - is invalidated as soon as any of the components that were tested is changed. Of course, sometimes it is necessary to change components - if a bug has been found or if the user requests a change to the process - and then the affected test cycles need to be re-run and this is called regression testing.
Volume (or 'bulk') testing: As more and more data is added to a system, so performance begins to change: database response times may slow, screen load times can be affected and lists may become unmanageable. Sometimes these issues can be managed as a system grows but if a change is being released to a large, existing customer base, then it is essential to test against high volumes of data.
Load testing: this is related to volume testing (indeed, the two are sometimes combined for operational acceptance testing or OAT). Load testing involves having many, many users accessing the system simultaneously. This can be difficult to simulate and there are specific tools - such as Astra Load Tester - that can be used (at some expense!).
Automated testing: Sometimes the same test cycles need to be repeated over and over again. An example would be testing a web application against many different operating systems and browsers. There is a high overhead to automated testing; test scripts must be changed to mirror any system changes and, of course, the scripts need testing themselves. However, it does have its place.
Using reports for testing: Sometimes a system can, in part, be used to test itself. If a system has a decent reporting function, then that can be used to check that the system is correctly recording its own activity. So, if the testing started off with twenty widgets in stock and seven have been 'sold' in the tests, then the stock report should show thirteen left. If it doesn't, then either the system or the report needs debugging.
Part of the skill of testing is understanding the appropriate tests for an application: a simple testing might not need a separate system test for example. However, two types of testing should always take place before a change is put live: the testing done by the development team and the UAT carried out by the client.
Tuesday, 15 September 2009
What successful testing looks like.
If you were feeling reckless (or just had a lot of time to fill) you might ask me where I.T. goes wrong. Amongst the many theories and anecdotes that would ensue, I believe there is one theme that would crop up repeatedly to such an extent that ultimately if someone were, in turn, to ask you for your thoughts on the source of I.T.'s failings, you would say "testing". (Or, perhaps, "would you mind if we talked about something else".)
On Saturday I met up with a man for whom I used to work at the Royal Bank of Scotland Group. We had both been successful test managers - indeed, Cameron still is - but we agreed that ultimately you only need three things to achieve that success: common sense; tenacity; and no desire to be popular.
Testing is a simple science. In my next blog I will talk about the different types of testing but, at a high level, testing is just about making sure that the software works the way it is supposed to. What could be simpler? you might reasonably ask and the answer is not much. Assuming you have good communication with your client or, better yet, a decent system specification document, you will know what you are building and what the system is supposed to do and, therefore, what you need to test.
For a simple website, that will mean making sure that there are no mistakes in the text, that the images all load and that the links work, that your SEO is in place and that your site conforms to standards around accessibility and W3C. For an e-commerce site you will, amongst other things, check the product search, that items can be added to the basket, that VAT is calculated correctly, that your secure certificate is in place and that a customer can actually complete a transaction. And so on.
So if testing is that simple, and it is, how on earth has that ended up being a prominent and consistent a factor in I.T. failures?
Firstly, I think, testing has never been a strong feature in I.T. I was on my fourth I.T. role when I encountered my first colleague whose job was, specifically, testing. He introduced himself as a system tester to universal bemusement. A few years later, in 1995, in fact, I worked with a chap called Graham Bradford who speculated that testing could become big business. I don't think Graham had anticipated the Millennium Bug but he was absolutely right.
By coincidence, I made an uncertain move from systems analysis into testing the following year when IBM interviewed me for the wrong job but gave it to me anyway. In the early days I was delighted to find that I was apparently being paid for old rope - for my common sense, in fact - but I quickly learnt where a test manager proves his worth. Time and again I have seen the time set aside for testing on a project plan effectively viewed as contingency. As development delivery dates slip, the go live date does not and the element that suffers is testing.
And this is where the tenacity comes in: if, as a tester, you are told that you are going to receive the new system for testing two weeks later than planned, then you need to make sure that the live date is pushed back by two weeks, so that you don't lose the time you need. Project managers do not like this. And that's why you need to be prepared to be unpopular.
Furthermore, you need to insist that any change that is applied to a system needs to be tested. A few years ago I was working for a company who wanted to make some "simple changes" to the website. We had a lot of clients using the site for financial transactions and I insisted that any changes needed to be tested. I was told that the changes were not functional and that there was no need for testing. I dug my heels in and told the project manager that whilst I couldn't stop the changes going live, I certainly wouldn't sign them off. Eventually, with an attendant dip in my popularity ratings, the testing was authorised.
Lo and behold, we found some issues. There was no ticker tape parade and no bounce in my popularity, just a few gripes about how I couldn't have known there would be any bugs. Well, of course I couldn't. But at least the next time a "simple change" came along, I had a good argument to fall back on.
So, what does successful testing look like? Well, in detail, it looks like a test strategy and test plans and test scripts and good test data and a strong test team. But most importantly it looks like a decent amount of time allocated to testing and for that time to be guarded jealously and not squandered to compensate for other problems on a project.
In conclusion, then, poorly tested projects result in dissatisfied users and lost customers. They require lots of support once they are live and consequently have a continuous and unpopular ongoing cost. Successful projects stick to their plans, including testing, are candid about their slippage when it occurs and ensure that when the system does 'go live' it works as it was intended to. A poorly tested product offers a constant reminder of the problems with I.T. A successful project invites further development and, as software developers, that is the goal we should be pursuing.
On Saturday I met up with a man for whom I used to work at the Royal Bank of Scotland Group. We had both been successful test managers - indeed, Cameron still is - but we agreed that ultimately you only need three things to achieve that success: common sense; tenacity; and no desire to be popular.
Testing is a simple science. In my next blog I will talk about the different types of testing but, at a high level, testing is just about making sure that the software works the way it is supposed to. What could be simpler? you might reasonably ask and the answer is not much. Assuming you have good communication with your client or, better yet, a decent system specification document, you will know what you are building and what the system is supposed to do and, therefore, what you need to test.
For a simple website, that will mean making sure that there are no mistakes in the text, that the images all load and that the links work, that your SEO is in place and that your site conforms to standards around accessibility and W3C. For an e-commerce site you will, amongst other things, check the product search, that items can be added to the basket, that VAT is calculated correctly, that your secure certificate is in place and that a customer can actually complete a transaction. And so on.
So if testing is that simple, and it is, how on earth has that ended up being a prominent and consistent a factor in I.T. failures?
Firstly, I think, testing has never been a strong feature in I.T. I was on my fourth I.T. role when I encountered my first colleague whose job was, specifically, testing. He introduced himself as a system tester to universal bemusement. A few years later, in 1995, in fact, I worked with a chap called Graham Bradford who speculated that testing could become big business. I don't think Graham had anticipated the Millennium Bug but he was absolutely right.
By coincidence, I made an uncertain move from systems analysis into testing the following year when IBM interviewed me for the wrong job but gave it to me anyway. In the early days I was delighted to find that I was apparently being paid for old rope - for my common sense, in fact - but I quickly learnt where a test manager proves his worth. Time and again I have seen the time set aside for testing on a project plan effectively viewed as contingency. As development delivery dates slip, the go live date does not and the element that suffers is testing.
And this is where the tenacity comes in: if, as a tester, you are told that you are going to receive the new system for testing two weeks later than planned, then you need to make sure that the live date is pushed back by two weeks, so that you don't lose the time you need. Project managers do not like this. And that's why you need to be prepared to be unpopular.
Furthermore, you need to insist that any change that is applied to a system needs to be tested. A few years ago I was working for a company who wanted to make some "simple changes" to the website. We had a lot of clients using the site for financial transactions and I insisted that any changes needed to be tested. I was told that the changes were not functional and that there was no need for testing. I dug my heels in and told the project manager that whilst I couldn't stop the changes going live, I certainly wouldn't sign them off. Eventually, with an attendant dip in my popularity ratings, the testing was authorised.
Lo and behold, we found some issues. There was no ticker tape parade and no bounce in my popularity, just a few gripes about how I couldn't have known there would be any bugs. Well, of course I couldn't. But at least the next time a "simple change" came along, I had a good argument to fall back on.
So, what does successful testing look like? Well, in detail, it looks like a test strategy and test plans and test scripts and good test data and a strong test team. But most importantly it looks like a decent amount of time allocated to testing and for that time to be guarded jealously and not squandered to compensate for other problems on a project.
In conclusion, then, poorly tested projects result in dissatisfied users and lost customers. They require lots of support once they are live and consequently have a continuous and unpopular ongoing cost. Successful projects stick to their plans, including testing, are candid about their slippage when it occurs and ensure that when the system does 'go live' it works as it was intended to. A poorly tested product offers a constant reminder of the problems with I.T. A successful project invites further development and, as software developers, that is the goal we should be pursuing.
Wednesday, 5 August 2009
What I Talk About When I Talk About I.T.
I recently read Haruki Murakami’s book ‘What I Talk About When I Talk About Running’ on the basis that it was recommended to me by a couple of people and also because I am keen, if amateur, runner. However, much as I enjoyed the book, what really grabbed me was the title: I was really set thinking about how the subjects we pick when we talk about a topic can help us to better understand our approach to it, and also help us to think about what we do well. Of course, we can also gain by considering the subjects that we don’t discuss, although it is, of course, always harder to spot things that are missing.
That said, I know that when I talk about I.T., one thing that I don’t talk about – unless asked – is code. Unless they are complete frauds, I would normally work on the assumption that anyone employed as a developer is actually able to read and write code. Furthermore, by and large a good developer can code using different languages on a given platform, as the concepts are the same: it’s just the syntax that is different. What I am saying here is that within I.T. development, the coding should be a given.
What is important is the framework around that code. So, for starters, let’s talk about the people who do the development. I’ve already said that I’m assuming these people can write code, so what makes for good developers? For a start, they need to be effective in a team environment. I cannot emphasise this enough. For example, developers need to be able to participate in a discussion about solutions and accept that, sometimes, their solution won’t be the one that is adopted, yet go away with good grace and cheer and work on the accepted solution. I have any number of anecdotes of where a developer has gone off and done it their way, regardless of what was agreed, and for this to come back and bite the entire project.
Good developers will naturally put comments in their code - describing what they are doing - because it is implicit in their mindset that, later on, someone else will come to look at it. I can clearly remember sitting once at a code handover when a developer was leaving and, in response to a question about how a function worked, could not even find the lines in his own code.
As a team player, a good developer will accept that he is part of a process, a project life-cycle, and that as he may rely on an analyst for a clear, concise specification or a project manager to effect strong change control, so there are other developers and testers who are dependent on the punctual delivery of working code. It follows from this that the developer will focus on what needs to be delivered.
At Meantime, 50% of our resource consists of developers. Around that development function, we have project management that ensures that the user requirements have been established and agreed, and that any changes to those requirements are introduced in a manner that is controlled and avoids confusion. The project manager for a piece of work will identify the resources required and plan their work accordingly. It is this discipline that enables us to deliver our clients’ requirements on time and on budget.
Business analysts spend time with the client, discussing their business and their requirements, making sure that the work has clear – and preferably demonstrable – cost benefit and also that the proposed solution is what the client wants and needs. The results of this work will result in some documentation which comes under the term of a business requirements document. For a small piece of work this may be just an email but, at the other end of the spectrum, I have just completed one that came to over seventy pages.
Once the requirements are signed off, then the business analyst then works closely with a systems analyst who will turn the business requirements into a specification for the developer. Again, the detail in the specification will vary from project to project but it should clearly set out the functionality that must be completed for the work to be considered finished.
I will write in more detail in a future blog about responsibilities for testing and with whom they lie but once the developer has completed his own testing then – for any significant piece of work – the code should go to a system tester. Their job is firstly to ensure that everything in the spec has been delivered and is working correctly and also to test various scenarios.
Once the testing is complete, the application can go to the client for them to test in a dedicated User Acceptance Test (UAT) environment. The user should be checking that everything in the business requirements document has been delivered and also that the system is intuitive and usable from their perspective.
All of the above are things that I discuss with people, mostly client, when I talk about I.T.: the importance of understanding what the user wants, sometimes when they aren’t clear themselves; the amount of work that needs to be done before a single line of code is written; the enormous value in communication with clients (all our developers talk to our clients); the importance of developers who are highly effective team players; the discreet testing function; and the handover to clients and UAT experience before code is made live.
Generally, I define the above as I.T. culture: people, processes and practice. Where I have participated in (or observed) successful projects, the culture has been positive and powerful, celebrating its own successes and understanding where it gets things right. It is a cliché but those projects have a ‘can do’ attitude. There is a prevailing team attitude that doesn’t depend on artificial glue like paint balling or getting drunk together but rather on the brilliant momentum that is gained by working together and doing a job well.
(With thanks to Haruki Murakami and Raymond Carver.)
That said, I know that when I talk about I.T., one thing that I don’t talk about – unless asked – is code. Unless they are complete frauds, I would normally work on the assumption that anyone employed as a developer is actually able to read and write code. Furthermore, by and large a good developer can code using different languages on a given platform, as the concepts are the same: it’s just the syntax that is different. What I am saying here is that within I.T. development, the coding should be a given.
What is important is the framework around that code. So, for starters, let’s talk about the people who do the development. I’ve already said that I’m assuming these people can write code, so what makes for good developers? For a start, they need to be effective in a team environment. I cannot emphasise this enough. For example, developers need to be able to participate in a discussion about solutions and accept that, sometimes, their solution won’t be the one that is adopted, yet go away with good grace and cheer and work on the accepted solution. I have any number of anecdotes of where a developer has gone off and done it their way, regardless of what was agreed, and for this to come back and bite the entire project.
Good developers will naturally put comments in their code - describing what they are doing - because it is implicit in their mindset that, later on, someone else will come to look at it. I can clearly remember sitting once at a code handover when a developer was leaving and, in response to a question about how a function worked, could not even find the lines in his own code.
As a team player, a good developer will accept that he is part of a process, a project life-cycle, and that as he may rely on an analyst for a clear, concise specification or a project manager to effect strong change control, so there are other developers and testers who are dependent on the punctual delivery of working code. It follows from this that the developer will focus on what needs to be delivered.
At Meantime, 50% of our resource consists of developers. Around that development function, we have project management that ensures that the user requirements have been established and agreed, and that any changes to those requirements are introduced in a manner that is controlled and avoids confusion. The project manager for a piece of work will identify the resources required and plan their work accordingly. It is this discipline that enables us to deliver our clients’ requirements on time and on budget.
Business analysts spend time with the client, discussing their business and their requirements, making sure that the work has clear – and preferably demonstrable – cost benefit and also that the proposed solution is what the client wants and needs. The results of this work will result in some documentation which comes under the term of a business requirements document. For a small piece of work this may be just an email but, at the other end of the spectrum, I have just completed one that came to over seventy pages.
Once the requirements are signed off, then the business analyst then works closely with a systems analyst who will turn the business requirements into a specification for the developer. Again, the detail in the specification will vary from project to project but it should clearly set out the functionality that must be completed for the work to be considered finished.
I will write in more detail in a future blog about responsibilities for testing and with whom they lie but once the developer has completed his own testing then – for any significant piece of work – the code should go to a system tester. Their job is firstly to ensure that everything in the spec has been delivered and is working correctly and also to test various scenarios.
Once the testing is complete, the application can go to the client for them to test in a dedicated User Acceptance Test (UAT) environment. The user should be checking that everything in the business requirements document has been delivered and also that the system is intuitive and usable from their perspective.
All of the above are things that I discuss with people, mostly client, when I talk about I.T.: the importance of understanding what the user wants, sometimes when they aren’t clear themselves; the amount of work that needs to be done before a single line of code is written; the enormous value in communication with clients (all our developers talk to our clients); the importance of developers who are highly effective team players; the discreet testing function; and the handover to clients and UAT experience before code is made live.
Generally, I define the above as I.T. culture: people, processes and practice. Where I have participated in (or observed) successful projects, the culture has been positive and powerful, celebrating its own successes and understanding where it gets things right. It is a cliché but those projects have a ‘can do’ attitude. There is a prevailing team attitude that doesn’t depend on artificial glue like paint balling or getting drunk together but rather on the brilliant momentum that is gained by working together and doing a job well.
(With thanks to Haruki Murakami and Raymond Carver.)
Monday, 3 August 2009
Why IT projects fail: an anecdote
A few years ago, in one of my last freelance roles, I was in a test management role for an international investment bank. My project manager reported to a programme manager one of whose other projects was failing. Three times it had missed its implementation date and the test function had been identified as the part of the process that was letting everything else down.
Since the testing for which I was responsible was proceeding well, I was asked whether I would cast an eye over the test plans for this other project and perhaps “sort things out”. Consequently, I met with the two people who were managing the testing and spent a couple of days with them, learning about their project and the testing that they had planned and attempted to carry out.
They were both clearly competent people and they seemed satisfied with the capabilities of the people working for them. Furthermore, they had a good grasp of their project and I couldn’t find much fault with their testing (and people will approach the same task in slightly different ways, anyway). So, at the end of all their explanation, I asked the simple question: so what’s going wrong?
The explanation was straightforward and will be familiar to anyone who has been involved in testing: some dates on the project plan had slipped but the deadline had not been changed and it was the testing that had seen its time budget diminish to make up for the slippage. The testing had not been completed and the implementation had been aborted. A second run, and then a third had been attempted but each time with only a small amount of time for testing. Defects were raised but there was no time to fix them and thus the project had ended up in its current state.
I had this meeting at the end of May and there were nine weeks of testing to be done, according to the test plan. So, I went back to the programme manager and gave him the good news. There was nothing wrong with his test team, they had a good plan and the testing – including time for bug fixing – would be complete by the end of August. I said a September implementation seemed reasonable but that an October live date would probably be prudent, especially when reporting up to the board, since it was realistic and contained the possibility of early delivery.
A couple of weeks later I bumped into one of the test managers and so I naturally asked how the project was progressing. It transpired that the programme manager had announced an *August* live date and that the test team had, once again, been set up to fail.
The moral of the story is, I think, self-evident, yet you will hear similar stories from IT people everywhere. If people are not given the time to do their jobs properly, then your projects will fail and your staff will become de-motivated. People should certainly be held responsible for the timescales and deadlines they give but there is no excuse for ignoring what they say and then holding them to account for dates that are imposed upon them.
Since the testing for which I was responsible was proceeding well, I was asked whether I would cast an eye over the test plans for this other project and perhaps “sort things out”. Consequently, I met with the two people who were managing the testing and spent a couple of days with them, learning about their project and the testing that they had planned and attempted to carry out.
They were both clearly competent people and they seemed satisfied with the capabilities of the people working for them. Furthermore, they had a good grasp of their project and I couldn’t find much fault with their testing (and people will approach the same task in slightly different ways, anyway). So, at the end of all their explanation, I asked the simple question: so what’s going wrong?
The explanation was straightforward and will be familiar to anyone who has been involved in testing: some dates on the project plan had slipped but the deadline had not been changed and it was the testing that had seen its time budget diminish to make up for the slippage. The testing had not been completed and the implementation had been aborted. A second run, and then a third had been attempted but each time with only a small amount of time for testing. Defects were raised but there was no time to fix them and thus the project had ended up in its current state.
I had this meeting at the end of May and there were nine weeks of testing to be done, according to the test plan. So, I went back to the programme manager and gave him the good news. There was nothing wrong with his test team, they had a good plan and the testing – including time for bug fixing – would be complete by the end of August. I said a September implementation seemed reasonable but that an October live date would probably be prudent, especially when reporting up to the board, since it was realistic and contained the possibility of early delivery.
A couple of weeks later I bumped into one of the test managers and so I naturally asked how the project was progressing. It transpired that the programme manager had announced an *August* live date and that the test team had, once again, been set up to fail.
The moral of the story is, I think, self-evident, yet you will hear similar stories from IT people everywhere. If people are not given the time to do their jobs properly, then your projects will fail and your staff will become de-motivated. People should certainly be held responsible for the timescales and deadlines they give but there is no excuse for ignoring what they say and then holding them to account for dates that are imposed upon them.
Sunday, 19 July 2009
Data security
Last week I was at a meeting in London with a project team from a publicly funded organisation and we were discussing how Meantime (the company I work for) would receive some data required for the initial stages of the project. One of the people 'round the table joked that they would give us the data on a flash drive as long as we promised not to leave it on any public transport.
Over the last few years, of course, there have been a number of incidents where this has happened - laptops and external drives left on buses and trains - sometimes with very sensitive data being lost as a result. Worryingly, I think that most people assume that it will happen again, which is dangerous; the acceptance that such incidents can be classed as just 'one of those things' makes people more careless.
I believe that the fundamental issue is that companies are very good at looking after data when it's where it is supposed to be. The database may be behind a firewall and access to that data may be only via an application that requires a valid user name and password from the user. The problems start when the data is extracted or reported out and stored somewhere else.
Earlier this year I had a meeting with a client who was so cautious about his data - which was, admittedly, of enormous commercial value - that he disabled his (password protected) laptop's wireless functionality before he'd open the spreadsheet containing the core data. I asked him where he backed up his data to and he produced an external drive from his laptop case. Quite apart from the fact that someone stealing his laptop bag would also have his backup, the drive was not protected and the spreadsheet, which was not itself password protected, could easily be accessed.
Similarly, data is downloaded to disks and printed out to paper reports, i.e. taken away from its secure environment, which are then being handled by people who under normal circumstances would not have access to that data.
However, there is a solution and it's one that well established, easy to use and free.
True Crypt is available for download from the web and you can read all about it here: www.truecrypt.com. And if you don't want to read all that - it is rather techie - then let me just say that I've been using True Crypt for a couple of years now and I can't sing its praises highly enough. It is simple to install and to use and means that no one can access the data without a valid password.
Another alternative is to use WinZip (www.winzip.com). While many applications will open a ZIP file, allowing the files to be seen (if not their contents) it is still possible to password protect the files.
For me, the biggest advantage is that True Crypt is able to turn a whole device into an encrypted drive meaning that if, for example, you have a flash drive that contains your business data, it cannot be accessed at all without the True Crypt software and, of course, the password required to access it.
It goes without saying that we live in an increasingly data driven society and the boundaries around our personal data are increasingly blurred. Businesses cannot afford to be anything but strict and diligent about their data protection: slip ups will certainly lead to a massive drop in credibility - either within your organisation or with your clients and customers - and may lead to legal action and a loss of business advantage depending on the data that is leaked.
Policies and procedures are essential but the use of tools like WinZip and True Crypt offer a concrete method of ensuring those practices are enforceable.
Over the last few years, of course, there have been a number of incidents where this has happened - laptops and external drives left on buses and trains - sometimes with very sensitive data being lost as a result. Worryingly, I think that most people assume that it will happen again, which is dangerous; the acceptance that such incidents can be classed as just 'one of those things' makes people more careless.
I believe that the fundamental issue is that companies are very good at looking after data when it's where it is supposed to be. The database may be behind a firewall and access to that data may be only via an application that requires a valid user name and password from the user. The problems start when the data is extracted or reported out and stored somewhere else.
Earlier this year I had a meeting with a client who was so cautious about his data - which was, admittedly, of enormous commercial value - that he disabled his (password protected) laptop's wireless functionality before he'd open the spreadsheet containing the core data. I asked him where he backed up his data to and he produced an external drive from his laptop case. Quite apart from the fact that someone stealing his laptop bag would also have his backup, the drive was not protected and the spreadsheet, which was not itself password protected, could easily be accessed.
Similarly, data is downloaded to disks and printed out to paper reports, i.e. taken away from its secure environment, which are then being handled by people who under normal circumstances would not have access to that data.
However, there is a solution and it's one that well established, easy to use and free.
True Crypt is available for download from the web and you can read all about it here: www.truecrypt.com. And if you don't want to read all that - it is rather techie - then let me just say that I've been using True Crypt for a couple of years now and I can't sing its praises highly enough. It is simple to install and to use and means that no one can access the data without a valid password.
Another alternative is to use WinZip (www.winzip.com). While many applications will open a ZIP file, allowing the files to be seen (if not their contents) it is still possible to password protect the files.
For me, the biggest advantage is that True Crypt is able to turn a whole device into an encrypted drive meaning that if, for example, you have a flash drive that contains your business data, it cannot be accessed at all without the True Crypt software and, of course, the password required to access it.
It goes without saying that we live in an increasingly data driven society and the boundaries around our personal data are increasingly blurred. Businesses cannot afford to be anything but strict and diligent about their data protection: slip ups will certainly lead to a massive drop in credibility - either within your organisation or with your clients and customers - and may lead to legal action and a loss of business advantage depending on the data that is leaked.
Policies and procedures are essential but the use of tools like WinZip and True Crypt offer a concrete method of ensuring those practices are enforceable.
Subscribe to:
Posts (Atom)