Differentiate between Purchaser and End User

When determining the user profile, make sure to differentiate between the purchaser or buyer of the product and the actual end user. Sometimes the user profile is slanted toward the person who makes the buying decision but who knows very lUtle about the problems that the end ujrs Le. Cprobablv would not want to bring in purchasers" to test the product; that wiU give vou a very slanted view of how the product is performing 

Another potential pitfall is the fact that the end user of the product and the end user of the documentation may differ. This is especially true for laree systems being developed for internal end users, where onlv one or two people such as the system administrator, will receive documentation. The rest of the end users will be left to their own devices or will have to ask specificallv for the documentation

Visualize the Test Participant

The specific characteristics that make up the user profile will of course depend on your product and the tasks you want participants to perform in the test. Tie the main objective of your test to the user profile. Start by visualizing and describing the person you want to be in the testing room with you. This is a simple narrative sentence or short paragraph, such as this:

We want to find out where in the process of booking a room at one of our hotels travelers meet obstacles to completing reservations /the test objective/. The user of our web site reservation system travels at least a feiv times a year and stays in a hotel each time. He or she typically books his or her ou>n travel online, especially

leisure travel. He or she may belong to hotel rewards programs.

While this is rather general, you can probably conjure up the image of someone who might fit this description. That's exactly the idea: See a person rather than a list of qualifications. Now, on to expanding your understanding of the target audience and documenting it.

Characterize Users

Now your first thought might be that the information describing the user profile should already exist and be written down somewhere within your organization, because the design and makeup of any product is supposed to be predicated on knowing the end user. While this is so in theory, experience shows that this information is rarely collected systematically or written down. Or, if it is written down, it does not quite make it to the desks of the design team, the people who really need it. Or, if it makes it to the desks of the

design team, they either pay it little mind or have a distorted version of the information.

Find and Select Participants

The selection and acquisition of participants whose background and abilities are representative of your product's intended user is a crucial element of the testing process. After all, your test results will only be valid if the people who participate are typical users of the product, or as close to that criterion as possible. If you test with the "wrong" people, it does not matter how much effort you put into the rest of the test preparation. Your results will be questionable and of limited value.

Selecting participants involves identifying and describing the relevant behavior, skills, and knowledge of the person(s) who will use your prod- uct. This description is known as the user profile of the target audience and should have been developed in the early stages of the product development.

Then, once that has been determined, you must ascertain the most effective way to acquire people from this target audience to serve as participants within your constraints of time, money, resources, and so on.

Test Observers

A test observer is not particularly a testing role per se. Rather, it refers to anyone else who attends a test, whether members of the development team, members of other projects, managers, or even members of other companies who are developing products that interact with yours While not strictly a testing role, observation of the test by a variety of people helps the product and the organization immensely. The product benefits because the people responsible for product development get direct feedback from representative

users. In addition, testing often brings a positive change in perspective toward user-centered design, which can help the development of future products. All development team members should be encouraged to attend as many test sessions as possible.

Additional Testing Roles

Additional testing roles will depend on the nature of the test. One or more people might be needed to simulate different roles during the test, as part of the test design. You might need someone to staff a hotline in order to simulate "help" calls during the test. Or, you might need someone to send a text message to a cell phone or to mimic a repair person sent out by the company.

Product/Technical Expert(s)

This is one or more individuals who know the technical aspects of the product being tested. Their role is to ensure that the product does not malfunction during the test For example, if a system crashes and jeopardizes the test, someone needs to be available to restore it or provide a working backup. This is crucial if you are testing at an early stage of development when the product is temperamental and "buggy/' If the prototype or product you are testing is unstable, you should have at least one product expert available at every test

session to troubleshoot. If there is a group of people on the development'team who can take on this role, it makes sense to coordinate their time so at least one is always present.

Data Gatherer/Note Taker

A data gatherer takes notes or classifies the expected critical activities and events of a test into coded categories. For example, if the participant is expected to access four screens during a test, each screen could be categorized with a different letter: A, B, C, or D. During the test, when the participant accesses a particular screen, page or module, the data gatherer need only record a letter for that action. At the same time, the data gatherer could also note the time in order to cross-reference with the video or audio recording,

unless he or she is using data logging software that is synchronized with the video recording.

Before the test the person who designs the test and the data gatherer

develop appropriate data gathering tools or devices. You can use simple Word files for entering long-hand notes that include pick lists that anticipate issues or cover specific data you want to collect. Or you can set up more elaborate data gathering tools in Excel, a database program, or some other logging application.

If you are recording sessions using Morae or other similar tools, you can configure the software to collect some types of data for you automatically.

Identify Co-Researchers, Assistants, and Observers

You may be the lone person working on usability testing, at least at first, but that does not mean you should do everything by yourself. Ask for help, get others involved by giv ing them specific jobs during the test sessions.

Gathering Biometric Data

At the writing of this edition, equipment traditionally used in other types of experiments to gather data from participants about how their bodies respond tostimuli is being incorporated in some types of usability testing. Eye-tracking is very popular with companies concerned with where to place advertisements on their web pages. Heart rates, blood oxygen levels, and other measures are interesting to your business partners in advertising or sales.
Tin's equipment is still fairly expensive, and the output is difficult to interpret.
As market and user research continue to evolve, we expect that the prices for equipment will come down and researchers will better understand when and why to collect this biometric data and what is meaningful and useful about it.

Basic Equipment Tools, and Props

In addition, there may be items that are particular to a test that you must ensure are properly prepared. The following table gives a list of basic equipment, forms, and note-taking tools as well as artifacts and props.

Gather and Check Equipment Artifacts, and Tools

No matter the location of the test, there are basic articles of equipment, tools and materials that you will need to gather and check before you can do your dry run or pilot session.

Gather and Check Equipment Artifacts, and Tools

No matter the location of the test, there are basic articles of equipment, tools and materials that you will need to gather and check before you can do your dry run or pilot session.

Gather and Check Equipment Artifacts, and Tools

No matter the location of the test, there are basic articles of equipment, tools and materials that you will need to gather and check before you can do your dry run or pilot session.

Recommended Testing Environment: Minimalist Portable Lab

The setup in the section "Minimalist Portable Lab Setup" is the setup we recommend for organizations just beginning to test and for those with limited resources. It gives for the best tradeoff of test moderator access to the participants, accommodation of observers, and cost (because there is nothing in the way of physical plant required). While the setup doesn't specifically accommodate viewing of a test by observers, it is extremely flexible in that you can use it in any combination of room (or remote) configurations. Management

will look favorably on such a situation, if it makes more efficient use of existing facilities.

Eventually, if your organization adopts a full-scale testing program, you will want to switch to a dedicated lab similar to the classic testing laboratory setup shown in Figure 6-8.

Disadvantages

■ Depending on the skill of the testing staff, the control room setup can create a very impersonal  environment. This is sometimes referred to as the "guinea pig" syndrome, with the participant feeling overly

self-conscious during the test. (This effect can be exacerbated by the type of intercom system used, some of which make the test moderator's instructions sound like the "voice of God/') This self-consciousness is worse for less sophisticated (in terms of exposure to high-tech equipment) participants who are insecure about their abilities. The test moderator may need to mitigate the effects of this setup by staying in the room

with a less sophisticated participant.

■ Unless you have ideal camera placements, you may not be able to see exactly what the participant is viewing or experiencing. For example, sometimes you may nol be able to tell where in a document or on a

screen a participant is looking unless there is a dedicated and well-placed camera {or you are employing eye-tracking equipment). To compensate, you may need to use a "thinking aloud" protocol, which may not fit in with your test design

■ For exploratory tests, the control room setup may offer little advantage to moderating the test, because you will need to be next to the participant in any case. However, for observers, it is still superior to the other

types of setups.

Advantages

■ It allows unparalleled unobtrusive data collection. Because the moderator is not in the room, this eliminates almost entirely any  possible biasing effects due to inadvertent nonverbal communication or mannerisms.
Being in a (nearly) soundproof room, the testing team and observers can easily communicate among themselves about the proceedings and discuss possible product improvements.
This environment can accommodate many observers (typically 4 to 12) viewing the test at one time.

Classic Testing Laboratory Setup - II

Classic Testing Laboratory Setup - I

This setup (see Figure 6-8) consists of one room designated as the testing room and a second room designated as an observation and control room. The only individual inside the testing room is the participant. All other test personnel, including the test moderator, observers, camera operator, and so forth are stationed inside the control room viewing the proceedings through a one-way mirror.
All communication between the test moderator and participant occurs through an intercom and speaker arrangement, and, in the more elaborate laboratories, the test is extensively monitored with multiple video cameras, audio tape recorders, data loggers, and other modern electronic equipment.

However, as an alternative, depending on the type of test, the test moderator could also be in the test room with the participant.

This setup requires a large capital outlay and commitment to testing by management. However, if you are interested in trying out such a setup for a test that you are conducting, you may do so for a modest rental fee. Simply locate a !ab facility (there are many throughout the country; www.Quirks.com has a directory of such facilities). Some facilities do none of their own research but only make the space and equipment available for a per diem rate. Research firms, market researchers, universities, and other corporations may have lab spaces that you can rent. Most come with a receptionist and other niceties. Many also offer participant recruiting.

Advantages

- All the advantages of the simple single-room setup also apply to this one.
■■ It accommodates a large number of observers.
■ Because so many observers can attend, debriefings between or after sessions can be more useful than when a smaller number attend.
- You may not have to record sessions because everyone who wants to observe can.
- With present, engaged observers, you can gather notes and insights from them more easily.

Large Single-Room Setup

Similar to the setup found in the section "Simple Single-Room Setup/' the moderator and participant are situated near each other (see Figure 6-6). In this case, though, by using a much larger room — such as a training room — you can invite many more observers to each session. (Some usability practitioners

routinely have up to 30 observers per session.) Here, the participant and moderator are in front of the room. Both wear microphones so that the observers can hear without strain. The image of the participant's screen is
projected onto the wall above the participant or above the observers, who could even be facing met11/ from the participant.

Disadvantages

Loss of proximity to the participant limits what the test moderator car see directly of the proceedings. It is possible to miss subtle behaviors It is crucial to have a good angle via the video camera on what the participant is seeing and doing, because you may be blocked from seeing directly.

Because the test moderator is not within the participant's peripheral vision, if the test moderator is directly behind the participant and within 10 feet or so, the test moderator could make the participant feel

very uneasy and overly self-conscious. The guideline here is to remember when you are behind the  participant but within ten feet, you must stay within the participant's peripheral vision. Farther back than approximately ten feet should not cause any discomfort to the participant.

As with the simple single-room setup, there is limiled space for observers.

If the participant can hear you typing/ he or she may feel intimidated or may assume that he or she is doing things wrong that you are recording.

Advantages

The test moderator is more free to move about, take notes use data In. ging software while the test is goinR on, and yet is still wiZ^f proximity of the participant. Being somewhat removed from he par ,c .pant, you need not be a, concerned about controlling body anX mannerisms, and so forth. In addition, you may use a computer w thout distracting the participant. K Ul
The participant does not have a complete sense of isolation, because the participant is still in the room with the test moderator.
This setup is also more likely to encourage the participant to think aloud, than if left alone in the test room.

Modified Single-Room Setup

This setup (Figure 6-5) enables you to use a room large enough to position yourself at a workstation behind or to the side of the participant without