First in an occasional series looking at how we have sought to design, develop and of course, test this site.

We have all been there. The project is initiated with a huge fanfare. Meetings, press releases and promises of management support. For the Test Team, the project starts with a massive test planning exercise. During the course planning, a small rain forest disappears producing documentation. The centrepiece of which is the Test Plan.

A year down the line, it is a different story. The product is defect ridden, late and the Test Team are under pressure to complete testing in an impossible deadline

At the height of the panic, a new tester arrives. Looking for inspiration, he pulls the once mighty "Test Plan" off the dusty shelf and says, "do we follow this?" To which everyone laughs.

Does it have to be this way?

When deciding that we were going to build the web application you are viewing, we wrote a test plan. (To view the test plan.). Further information can on documentation can be found at Planning.

Why did we write a test plan?. Obviously it was for the highest possible motives. We wanted to lay down how the website was to be tested. The methods we were going to use and to scope the effort. Additionally we wanted to be see following good practice. (Practice what we preach asically.) Everyone in the organisation agreed that this was a good idea. Proper planning with the correct amount of ceremony, would aid us in keeping the application defect free.

In many cases this is the case. Lots of documentation done by highly idealistic people. Typically written by test management. The problem is that to execute the test plan requires resources. Unfortunately these are provided for by business management, who have to focus "on the bottom line". This is a euphemism for much less money spent on testing. Correlation can be very variable between the real world an ideal test plan world.

We set standards that we wanted to meet. Everyone that visited our site would leave with the same impression. Namely that it is professional, corporate. Additonally it was to bear no resemblance to a personal homepagDocumentation, it was felt would lay a solid foundation to acheive these aims.

Actually writing the plan took one night. "Thinking" about how we were going to test took much longer. We made a conscious decision to follow a template. The one we chose is based on the < href="a href="cont18.htm" target="_top">IEEE829:1998 standard for Software Testing. Some may consider this overkill. However it focused us on making some hard decisions, such as do we purchase test tools? (The answer was only if they are cheap. See 11.4 on the plan.)

Our methodology in planning was to follow industry good practice. However we mad a conscious decision not to follow religiously one framework. The other consideration was to be realistic. We were not builing life critical systsems such as heart monitoring software. Also at any moment work or family would take precedence.

Taking each section in turn we will look at what we planned, and what we do.

The first section is merely an identifier. However this looks out of date as it says "New Testing Site."

Section 2 has the introduction and overview. Many fine words are spoken about the testing and techniques and how it will be partitione. Clear and concise. To a large extent the spirit of these paragraphs has been implemented. We do implement a test first, design, build, test. This is also done iteratively.

The most important sentence in the whole section, if not document is:-

The test schedule is extremely constrained by the lack of time resource. Aniticipated completion times are extremely vague. Deliverables and milestones are anticipated to be incremental in nature. This had to be where realism set in. Family and work remain the prioritY, and if the site did not get tested, then so bad.

Sections 3-5 lay down the scope of the testing. If it is new page it gets tested. An old one does not.

Section 6 lays down the techniques to use. The test design types were based on the type of application we were building. A static HTML with a large amount of text and few images. These are exactly the techniques we use. Unfortunately in practice the edges are very blurred.

7-8 are the administrative side of things. It is these two section which will require some amendment. The amount of bureaucracy involved in implementing them is too high. If someone was going to die if they were not followed, would be more of an incentive.

The deliverables listed in 9.1 have all appeared except the incident log. (This has fallen into disuse as everything is fixed as soon as we notice it.)

10-11 have been fulfilled. I.e. there are no outstanding tests at the end of each enhancement. The temptation has been to show some testing as "out of scope". However this has been resisted. Instead of skimping on testing, it has been decided to remove the functionality that would need testing.

12-14 essentially state that testing is going to be a very small team affar. In practice, this has definitely been the case.

Section 15- Risks and contingencies. Most of the risks listed have been mitigated successfully. Procedures have been put in place to monitor and hopefully reduce risk.

The next article will focus on our actual procedures for developing and testing a new piece of functionality.