Consider a Matrix Test Design

Because you want to see a range of behavior and performance within your small group of participants consider a matrix design for the test that balance different variables or classifiers in such a way that no particular user group or cell of the matrix is left out. 6 p

Take the banking example. These days you can assume that having some computer experience is a requirement for getting a job either at the teller vvicket or in the back office, so that's probably not a useful way of classifying these participants. What could determine levels of expertise in the two roles that would be important to your test? Possibly a combination of time on the job and types of responsibility may describe an "expert" teller from a "novice" one or an "expert" back office clerk from a "novice" one. You could look at the factors you use to determine expertise as domain knowledge (for example, what the participant knows about being a teller) or tools knowledge (for example, the efficiency and effectiveness with which the participant uses the software or other technology required in her job). You'll want some of each and you'll

have to create some type of operationalized definition for determining who falls into which cell.
Good test design mea ns that you balance the mixtures of user groups in such a way that the major categories or cells are all represented. For the banking example, you might test 16 participants whose job and experience level look like the ones in the table below (assuming that you have clearly defined the criteria for "novice" and "expert"; see "Define Expertise," above).