What
Testing where the system is subjected to large volumes of data.
BS 7925-1.British Computer Society Specialist Interest Group in Software Testing (BCS SIGIST)

At first glance it might appear that volume testing should be fairly high up on the V model, especially if we take the above definition. Certainly it comes under non-functional requirements testing and Performance testing.

In my view Volume testing can (and should) be used in component testing. This would be closely correlated with the code, so might at this level be connected to Dynamic Analysis.

Volume testing will also be undertaken (normally) as part of the User Acceptance test. Stress testing is closely related, as it seeks to find out how the software will behave beyond its specified limits.

Why?

   1) To find faults
   2) Give credible information about the state of the component, on which business decisions can be taken.

As regards faults that should be found through volume testing are those, where the behaviour of the software deviates from that expected for a specified volume of data. Thus a bank system will be tested for faults at much larger volumes of data, than that of small retailer software. A fault which is only manifested on a table with a million records will be of no concern to the retail software, but will be picked up by the bank testers.

Credible information about how the software will behave is essential. During the dot com boom many websites went live without knowing, what the effect would be if the back end database grew exponentially. Of course many suffered crashes as a result.

Why test at the component level? This is because we can then see how the code behaves, and confirm that the component will not be a bottle neck and slow down the whole system. (Alternatively, use too many system resources.)

For example, a window often used window object is populated with data, by calling a database object which runs a complex SQL query. Supposing the component is tested against tables with only 4-5 records. Of course it will return within seconds. Everything seems fine. It is then integrated with the window, and system tested. Again everything seems ok. It is only when the application is in User Acceptance (or even gone live) and it is tested against 100,000 records, is it discovered that, the SQL was not properly optimized and the tables not indexed. Thus it should have been tested at the component level.

Some methodologies such as the RUP suggest early testing. Therefore in the early phases, (inception and elaboration), volume testing might take place to confirm that the central core architecture is the one to proceed with.

Who?
As volume testing can be introduced almost anywhere, from component testing to user acceptance testing, the range of people who can undertake it is similarly large. Developers through to customers and end-users can do it. The testing may be outsourced to a testing lab, which specialises in performance testing.

Where?
Depends on which phase, but would usually be done at the developers site.

When?
All through the development lifecycle.

How?
Volume testing needs two things. Firstly clear expected outcomes of how the software is to behave for a given level of data. Secondly, data, and lots of it.

The expected behaviour at various levels, should be in the specification documentation. Ideally this will say something like "the customers details will be returned returned on the screen within 3 seconds, from a database with 1 million customer records." This gives the tester a benchmark to base a test case on.

The second requirement for data, needs either real life data, or simulated data. Usually, real life data will come in the form of a customer database, that has had private information, such as names and account numbers scrambled. Alternatively records can be created from scratch using automated tools or by adding rules directly on to the database, with SQL.

As with all testing, proper records must be kept showing the inputs, outputs other information, to aid potential debugging and audit purposes.