Basics of the Methodology - II

The preceding approach is the basis for conducting classical experiments, and when conducting basic research, it is the method of choice. However, it is not the method expounded in litis book for the following reason. 

■ It is often impossible or inappropriate to use such a methodology to conduct usability tests in the fast-paced, highly pressurized development environment in which most readers will find themselves. It is impossible because of the many organizational constraints, political and otherwise. 

It is inappropriate because the purpose of usability- testing is not necessarily to formulate and test specific hypotheses, that is, conduct research, but rather to make informed decisions about design to improve products.

■ The amount of prerequisite knowledge of experimental method and statistics required in order to perform these kinds of studies properly is considerable and better left to an experienced usability or human fac-

tors specialist. Should one attempt to conduct this type of tight research without the appropriate background and training, the results can often be very misleading, and lead to a worse situation than if no research had

been conducted.

■ In the environment in which testing most often takes place, it is often very difficult to apply the principle of randomly assigning participants because one often has little control over this factor. This is especially true

as it concerns the use of existing customers as participants.

■ Still another reason for a less formal approach concerns sample size. To achieve generalizable results for a given target population, one's sample size is dependent on knowledge of certain information about that population, which is often lacking (and sometimes the precise reason for the test). Lacking such information, one mav need to test 10 to 12 participants per condition to be on the safe side, a factor that might require one to test 40 or more participants to ensure statistically significant results.