Limitations of Testing

Now, having painted a rather glorified picture of what usability testing is intended to accomplish, let's splash a bit of cold water on the situation. Testing is neither the end-all nor be-all for usability and product success, and it is important to understand its limitations. Testing does not guarantee success or even prove that a product will be usable. Even the most rigorously conducted formal test cannot, with 100 percent certainty, ensure that a product will be usable when released. Here are some reasons why:

- Testing is always an artificial situation. Testing in the lab, or even testing in the field, still represents a depiction of the actual situation of usage and not the situation itself. The very act of conducting a study can itself affect the results.

- Test results do not prove that a product works. Even if one conducts the type o test that acquires statistically significant results, this still does not prove that a product works. Statistical significance is simply a measure of the probability that one's results were not due to chance It is not a guarantee, and it is very dependet upon the way in which the test was conducted