Sample Performance Measures - I

■ Number and percentage of tasks completed correctly with and without prompts or assistance

■ Number and type of prompts given

■ Number and percentage of tasks completed incorrectly

■ Count of all incorrect selections (errors)

■ Count of errors of omission

■■ Count of incorrect menu choices

■ Count of incorrect icons selected

List the Data You Will Collect

This sect-ion of the test plan provides an overview of the types of measures you will collect during the test, both performance and preference data. Performance data, representing measures of participant behavior, includes error rates, number of accesses of the help by task, time to perform a task, and so on.

Preference data, representing measures of participant opinion or thought process, includes participant rankings, answers to questions, and so forth. The data collected should be based on your research questions. Sometimes these measures will have already been alluded to in a previous section of the test plan, such as the methodology section. You can use both performance and preference measures either quantitatively or qualitatively, depending on the test objectives. See Figure 5-8 for example measures for a test of a hotel

reservations web site.

Listing the evaluation measures you will use enables any interested parties to scan the test plan to make sure that they will be getting the type of data they expect from the test.

The following is a sample of the types of measures you might collect during a typical test.

Explain What the Moderator Will Do

This section helps to clarify what you as a test moderator will be doing, and it is especially important when there will be observers of the test who are unfamiliar with the testing process. {See the example in Figure 5-7.) Specify when the test moderator will do something out of the ordinary that may lead to confusion. For example, sometimes it is unclear why and under what circumstances the test moderator is probing and intervening. This is especially

Describe the Test Environment, Equipment, and Logistics

This section of the test plan describes the environment you will attempt to simulate during the test and the equipment that the participants will require. For example, you might want to simulate a sales office for a product that insurance agents use. Or, perhaps chemists use your product in an environmental laboratory. Or, suppose thatyou simply want to test the product in a very noisy, somewhat crowded office where phones are constantly ringing. 

Whatever the typical operating environment, try your best to simulate actual conditions. Not only does this help the participants to take on the role of actual end users, but it also means the test results will be a better predictor of the product's performance in the place where it is normally used.

The equipment described here only includes the equipment that participants  will use. Examples of equipment are phones, computers, printers, and so forth. It is not necessary to describe data collection equipment or cameras you will be using to monitor the test. Figure 5-6 shows one example.

Ways to Prioritize Tasks - II

Prioritize by criticality. Critical tasks are those that, if performed incorrectly or missed, have serious consequences either to the end user, to the product, or to the reputation of the company; for example, when the tasks result in a support line call, cause loss of data, or cause damage to the product or bodily harm to the user. In short, you want to make sure that you catch those tasks that result in the most pain and potentially bad publicity

Prioritize by vulnerability. Vulnerability in this case means those tasks that you suspect, even before testing, will be hard to perform or that have known design flaws. Often, the development team will have

a good handle on this and, when asked, will voice concern for a new feature, process, interface style, section of a document, and so on. 1/ so, include tasks in the test that address these major areas.

Sometimes, developers pretend, in the name of "being unbiased," that all functions work equally well (or poorly), and that none are particularly problematic Whether for a well-intended or a less noble

reason, they do not want known problems exposed during the test. Consequently, tasks that are obviously hard to perform and that represent whole components, web pages, or sections of a document are left out of the test and prove to be albatrosses much later when there is no time to fix them. To avoid that, use your critical judgment about which tasks/ features are not quite worked out, are new or never-before-tested features, or have been difficult for in-house personnel to perform. If you are unsure, a human factors specialist can help determine the vulnerable aspects of the product by performing an evaluation. (An expert evaluation can also help you to tighten your test objectives in general.)

Ways to Prioritize Tasks - I

Now that you have reviewed an example of developing a task, the next issue is ascertaining what tasks you need to include. Due to time constraints verv rarely do you actually test the full range of tasks that comprise an entire interface, documentation, or both together. (It is impractical to conduct test sessions that last for days at a time, unless you are willing to commit an inordinate amount of resources.) Instead, you typically face a situation of testing a representative sample of the product's functions.

When choosing this sample of tasks, it is important that you exercise as many of the most important aspects of the product as possible and address all test objectives. Filter or reduce your task list to something manageable, while ensuring that you capture as many of the usability deficiencies as possible. 

The following list outlines some common methods you can use that prioritize or pare down the task list without needless sacrifice.

- Prioritize by frequency. Select those tasks that represent the most frequently performed tasks of your end  user population. The most frequent tasks are the ones that the typical end user performs daily, possibly up to 75 to 80 percent of the time, when using the product. For example, if you were testing a word processing package, you would want to make sure that the end user could easily perform the following tasks before you concern yourself with the more esoteric tasks such as "how to hide a comment that does not print out."

1. Open a file.

2. Save a file.

3. Edit a file

4. Print a file.

TASK COMPONENT DESCRIPTION

Task
State

Successful completion
criteria

Benchmark

DESCRIPTION

Arrange images into collections.

Web site with six navigation tabs leading to
different sections of the site.

Participant finds the Organize link and then
groups preloaded pictures into sets.

Participant puts more than two pictures into
one set with no "wrong turns."

Tasks

Participants start from one of three starling points All participants will use www. h . com to book a hotel room (up to the point of entering a credit card number or just before completing the rewards reservation) In a major U S city that has multiple H properties. 

Within that task, participants will select a hotel and room based on a combination of price and amenities. Each group will start at a different point:

Group 1    Start at H.com

Group 2    Start at non-branded search from Google (example: premium San

Francisco hotel 4 star hotel).

Group 3    Start from a branded search from Google (example H Hotel Atlanta)

Let the participants start where they would normally start Because you'll select participants for different combinations of characteristics, expect that different types of participants are motivated to do different things. Briefly interview the participant at the beginning of the session to get some impression of how the particular participant approaches booking travel arrangements-especially accommodations-and let them perform the task within their own context. This way, in addition to getting a feeling for the overall usability  of www. h . com. you can also identify usage patterns that could be further investigated in follow-on research. Finally, you will also get a better understanding of the traveler's thought processes and how H.com fits into that traveler's life. 

Figure 5-5 Task description for an exploratory usability test

Example Task: Navigation Tab on a Web Site

Suppose that one of your test objectives is to test how easy it is to understand a label for a tab that appears on an image-sharing web site that amateur and professional photographers use. The test objective is written as, "Establish whether users can understand the meaning of the XYZ label." There are six tabs with text labels on the web site, but the XYZ label is the problematic one. It's called Organize.

On the current version of the web site, users expect to use the feature to change the order in which their images appear on the viewing pages, but this feature is for organizing images into categories.

If you simply take the objective at face value ("Establish whether users can understand the meaning of the XYZ label."), you might decide to have a task that has the test moderator:

Show the participants the XYZ label and have them explain its meaning to you. In other words, the test moderator will get feedback about the label. This seems simple and direct, because the label is the offending aspect of the product. However, this is oversimplifying the situation. By performing a simple analysis, you ascertain that there are actually three discrete processes associated with correctly using the simple label.

1. Noticing the label

2. Reading the label

3. Processing the information and responding correctly

In addition, these three processes occur within the very specific context of using the web site to post images on the web:

- If you simply show the participants the label, you only address the second and third processes. You will not know if the participants even notice the label, which precedes the other behaviors. You will also negate
the entire context. In the course of using the web site, the participants will perform a particular task(s) at the time when they are supposed to be reading the label, not having someone point out the label and ask
them what they think. This "context" is critical because it dramatically affects their ability to process information.

-   You also need to address how the location of the label on the web page affects things. If it resides among five other labels and other actions, you should see how the participants perform with those potential distractions in place.

Tips for Developing the Task List

While this may seem straightforward, it is a very subtle process. The trick is to indirectly expose usability flaws by having the participants perform tasks that use the parts of the product in question. What you are really testing is the relationship of your product to the end user. From the end user's viewpoint, your product and its associated documentation are a means to an end, either used to solve a problem or provide a service.

The tasks that you develop for the test need to reflect this relationship and, as much as possible, allow the test to expose the points at which the product becomes a hindrance rather than a help for performing a task. Let's look at a simple example of a task to satisfy a test objective and, in so doing, review some possible pitfalls.

ABOUT BENCHMARK TIMINGS THAT ESTABLISH THE MAXIMUM TIME LIMITS FOR PERFORMING (continued)

Jeff established benchmarks for one test for an organization with no previous usability testing experience. The text was for a hardware product that would be tested with documentation. Jeff had three engineers provide estimates of the maximum time that they felt a user would need to correctly perform each task on the test He also had three technical writers on the project give the estimates because their perspective on the end user was different He then averaged all estimates, and, to give everyone the benefit of the doubt, he multiplied the average for each task by a constant of 2.5 to come up with the maximum time for a participant to complete the task. This constant was rather arbitrary and quite generous. Jeff simply wanted everyone to feel that the participants were given ample time before the task was classified as "incomplete." The generosity was due to Jeff's confidence given his familiarity with the product design and its potential flaws as well as participants exposing the problematic areas, even with the generous time allotments.

As it turned out, some of the tasks took up to three times longer than even these generous benchmarks, which really drove home the point about difficulties. Experience has taught the authors that poor product design will make itself known eventually.

Measuring time on tasks is not always the best, most accurate measure of  task success. If you are asking participants to think out loud, doing so takes time and unnaturally lengthens the duration. Instead, you may want to count only errors against the success criteria or completion criteria along with numbers and types of prompting.

ABOUT BENCHMARK TIMINGS THAT ESTABLISH THE MAXIMUM TIME LIMITS FOR PERFORMING

Timing or Other Benchmarks

You may want to use time as a criterion for success or as a benchmark. If you do set benchmarks that are based on timing, it's recommended you do this very thoughtfully and under just the right circumstances. For example time-on-task is a good measure for validation/summative tests, but it is rarely appropriate for early exploratory or formative tests. It is inadvisable to measure time-on-task if you're also asking participants to think aloud, because doing so typically slows task performance. For more about benchmark timings, see

the Benchmark Timings sidebar. If you don't want to use time as a benchmark you could use error rates; for example, completing a task with no errors of any kiding 

A Description of Successful Completion of the Task

How will you measure success? It is amazing how much disagreement there will be over this question and how often developers have differing opinions on what represents successful completion of a task. When you include successful completion criteria (SCC) with the task description, you add precision to what you are measuring and how you view the task. SCC define the boundaries of your task and help to clarify test scoring. When you have difficulty ascertaining the SCC, it reflects the development team's confusion about the product design. Establishing and documenting the SCC is a good exercise just for that reason alone.

Criteria for successful completion can include reaching a certain point in the task or screen flow, a maximum number of errors or wrong turns (for information-finding tasks), and whether you will consider the task "complete" if the participant reaches the appropriate end point but makes mistakes alone the way. 

Parts of a Task for the Test Plan

For the test plan, you need only touch on four main components of each task:
A brief Description of the Task
Include only enough detail at this time to communicate the task to the project team. A one-line description is usually enough.
The Materials and Machine States Required to Perform the Task

Context is everything in usability testing. As the test moderator, you may actually be providing these materials or simulating the machine states if the product is in an early stage. For example, if you were testing a web site

before the screens are coded or prototyped, you might provide printed wireframe drawings of the pages. Or if the page were available in a file on the computer but not hooked up as part of a working prototype yet, you (or the participant) might open that file on the screen for viewing at the appropriate time.

Or. perhaps parts of the test will be performed with documentation, while other sections will not. For example, if you are testing how well instructions work for installing a wireless network, and the later tasks will be done without documentation, such as specifying drive designations on the new network, you need to specify this. If it is appropriate and helpful, your task list might also include components of the product that are being exercised for that particular task. If, for example, a task asks a participant to enter a customer name into an online form, you might specify the screens or web pages that the participant will navigate during task completion. This helps to give you a sense of whether the full system is being exercised or not.

List the Tasks

The task list comprises those tasks that the participants will perform during the test. The list should consist of tasks that will ordinarily be performed during the course of using the product, documentation, and so on.

There are two stages to developing these tasks. In the early stages of developing the test, the task list description is intended only for members of the project team and not for eventual participants. You need to supply only enough detail so that reviewers of the test plan can judge whether the tasks are the correct ones and are being exercised properly.

Later, you will expand the tasks into full-blown task scenarios, which are presented to the participants. The scenarios will provide the realistic details and context that enable the participants to perform tasks with little intervention from the test moderator. Expanding the initial tasks into task scenarios is covered in Chapter 8. For now, your task list need only include the following.

Testing Multiple User Croups - II

Note If you are new to the game and are not confident that you can conduct a test with experimental rigor, then by all means keep the test simple. The more straightforward the test, the easier it is to keep everything consistent from session to session, it is better to attain meaningful results from a smaller, simpler study than to acquire a wealth of meaningless data from a larger study. Do some usability testing as early and often as possible. It need not be elaborate to be

useful or cost-effective.

Testing Multiple User Croups - I

Now let's look at a slightly more complex, yet realistic scenario. Suppose your user profile consists of two different user groups, managers and clerks, who will be using your product- One of your test objectives is to see if there are differences in ability to use the product between or among user groups. In addition, you also want to see if there are differences in novice and experienced users within each group. You will therefore need to vary experience and job type, each of which will have two levels. Once again, you will use a matrix
design, as shown below.

Each one of the four conditions or cells shown in the table above will be populated with a different set of participants. If you want to acquire at least four participants per cell, as shown, you will need a total of 16 participants.
If this is too many participants for your budget and time, (four participants is about the bare minimum per group required to evaluate group differences), then you cannot simply apply a within-subjects design. Instead, you will either have to limit each cell to fewer participants or simplify the study. Remember, limiting a cell to less than four participants severely limits the conclusions you can draw about each group. You will probably need to simplify the research to exclude a study of group differences (see Figure 5-4).

Independent Croups Design or Between Subjects Design - II

To account for these potential differences, you will again counterbalance the order of presentation of the versions. As shown in the table above for eight participants, some participants will do Version A first, and others will  do Version B first. Note that each version is performed as many times in the first position as it is in the last position, which negates the potential biasing

Independent Croups Design or Between Subjects Design - I

This is called an independent groups design because each part of the web site is tested by a unique set of users. For our example, shown in the table below, this design requires 15 participants and mitigates the potential transfer of learning effects caused by doing one set of tasks prior to performing other similar tasks. In other words, performing Task A may help one to perform Task B, and mask any usability problems associated with Task B. You can also use this design if the tasks are extremely lengthy and there is a possibility that the participants may become fatigued.

Describe the Method

This section of the test plan is a detailed description of how you are going to carry out the research with the participants, and how the test session will unfold. Essentially, it is a synopsis of your test design. It should provide an overview of each facet of the test from the time the participants arrive until the time they leave, in enough detail so that someone observing the test will know roughly what to expect. If you are questioning why this amount of detail is necessary in the test plan, the following reasons should satisfy your curiosity.

■ It enables others to understand and visualize what will happen so that they can comment and make suggestions accordingly.
■ It enables you as the test developer to focus on what has to be done and the types of materials that have to be developed before participants arrive.
■ It reveals the need to communicate your plans to additional resources whom you might have forgotten, such as a receptionist who will greet the participants in a corporate lobby when they first arrive,

■ It allows multiple test moderators (if that is required by the test design) to conduct the test in as similar a mariner to each other as possible. Test design is one of the more highly specialized skills required of a usability professional, often requiring knowledge of experimental design and method and basic statistical analysis. Designing a test requires one to clearly iden- tify and understand the test objectives, and then to select the test design that will effectively ferret out the answers to the questions posed. If the test design

is flawed or if the test is carried out with little attention to experimental rigor, then the results will be suspect. Not only can this result in faulty recommendations, but it also sabotages the progress of usability engineering
per se within the organization. Therefore, the first few times that you conduct a usability test, get advice and feedback on your test design from someone more experienced than you.
The test design is mainly predicated upon your test objectives — what you need to learn about the product and its audience. The design will be greatly affected by your resources, your constraints, and your creativity. Constraints are time, money, management backing, development team support, ability to acquire participants, and other real-world concerns. The following sections  give examples of test designs for some of the most common situations you will face. Following that, we present some guidelines for ensuring experimental rigor.
The simplest test design, shown in the table in the next section, consists of testing several different users, all from one type of user group (e.g., older adults), and having them perform a series of representative tasks on different parts of the web site.

Summarize Participant Characteristics - II

had we only tested four participants. Until you become experienced at testing, employing more participants decreases the probability you will miss an important problem, while providing additional opportunities to practice your moderating skills.
If you find you have very limited time and budget, you may want to institute a practice of "discount" usability testing, in which you would run several small, iterated usability tests over time. That is, conduct a test with 4 or 5 participants from one cell and one or two conditions, incorporate the findings into the interface, and then conduct another test with a similar set of participants and conditions. Over three or four tests, you end up with a large sample of participants, but the development team is able to accommodate changes in between tests.

Summarize Participant Characteristics - I

This section of the test plan describes the characteristics of the end user(s) of the product/document that you will be testing. It is Important to work closely with others in your organization to determine the characteristics of the target users. For detailed procedures on how to establish the user profile and acquire participants, see Chapter 7. A basic example of participant characteristics for a usability test of a hotel reserv ations web site appears in Figure 5-3.

One thing to remember when describing the participant characteristics is to use the right number of participants. When it comes to selecting the number of participants to employ for a test, the overriding guideline is "You cannot have too many participants." When thinking about achieving statistically valid results, small sample sizes lack the statistical power to identify significant differences between groups. For a true experimental design, you must use a minimum of 10 to 12 participants per condition. However, for the purpose of conducting a less formal usability test, research has shown that four to five participants who represent one audience cell will expose about 80 percent of the usability deficiencies of a product for that audience, and that this 80

percent will represent most of the major problems. Of course, if you have the time and resources to study more than four or five participants, by all means do so. It is possible that the additional 20 percent of deficiencies you might find could be important for your product,

We have conducted many tests that held true to the preceding principle. In one, Jeff tested eight participants and discovered about 80 percent of the problems within the first four participants. However, participant 8, the last one, performed a particularly grievous error on one task that would have required a servicecall for the product. This would never have been uncovered

Communicate Research Questions - III

Communicate Research Questions - II

If you find that you are having unusual difficulty designing the test and/or appropriate measures, or deciding on the appropriate end users, or even designing the data collection form, you might return to the research questions to see if they are clear or need further clarification.

Communicate Research Questions - I

This section is the single most important one in the test plan, because it describes the issues and questions that need to be resolved and focuses the research, as well as the rest of the activities associated with planning, designing, and conducting the test. It is essential that the research questions be as precise, accurate, clear, and measurable (or observable) as possible.

Even when conducting exploratory testing in the early stages of developing a product, which is typically less structured, you still need to accurately describe what you hope to learn.

Without a clear succinct research question(s), you might End yourself in the unenviable position of conducting a wonderful test that neglects to answer the key concern of developers on the project team. Or, you might find yourself with a test whose dev elopment bogs down in controversy because no one can agree on what to test. Speaking from experience, we have seen test preparations move in circles and the test itself result in controversy because the test objectives were never committed to paper.

The following are two examples of unfocused and vague research questions.
■ EXAMPLE 1. Is the current product usable?
— EXAMPLE 2. Is the product ready for release or does it need more work?

Good Reasons to Test

The following list gives some more rational reason f™ J which should result in successful outcomes LTSS^EffiS
- You want to know whether or not the documentation is able to compen- sate for some acknowledged problems with the interface.
- You have received numerous complaints associated with using the product. You are interested in determining the exact nature of the problem and how you will fix it within your development budget for
this year.

When Not to Test

Following are some rather vague, inappropriate reasons for usability testing a product. These are rarely placed on paper but are usually communicated via word of mouth. They are not sound reasons for testing, and invariably they often come back to sabotage the project.
- You can improve the user experience (you may be able to test one part of the customer experience but not all the touch poir.ts your company has with customers).
» Everyone else has a usability testing program (everyone else has many things). 1
- The meeting rooms used for testing are available the third week of the month (so is the cafeteria every evening).
- Lou just went to the latest ACM SIGCHI (Associadon for Computing Machinery Special Interest Group on Computer-Human Interaction) conference and learned about this really neat testing technique (let Lou
promote the technique's benefits to the organization first).
■ You want to see if there is a need for this type of product in the marketplace (backwards logic; a focus group or survey is a more appropriate technique early on).You might say to yourself, especially if you are eiger to begin usability testing, "As long as we test, I don't care what the reasons are. We'll worry about the consequences later." And for the short tern, there is no problem with any of the reasons stated previously. However, in the long term, if You want testing yo become an integral part of he wat tour organization develops products tou must yie testing to the neeed of the product and to the organizations business nedds.

Review the Purpose and Goals of the Test

For this part of the document, you need to describe at a high level the reasons for performing this test at this time. You need not piovukttte very specific objectives or problems to be explored here - rather, the ma,or or impetus is the key point, often from the viewpoint of vour organization
For example.
« Js the test attempting to resolve problems that have been reported by the company's call center or support desk?
■ Have server logs or web usage statistics shown that visitors to your company's web site leave the site at a particular point in a process that leaves a transaction incomplete?
- Has a new policy recently been instituted stating that all products must be tested before release?
- Does management feel it is critical for the development team to see real users at this time?
It is okay if the test purpose remains at a high level, because the research questions and problem statements will reduce the goal(s) to measurable statements. The important point is that the testing be tied to business goals within the organization and that testing is the most appropriate technique for addressing the problem or opportunity.

The Parts of a Test Plan

Test plan formats will vary according to the type of test and the degree of formality required in your organization. However, the following are the typical sections to include, along with a description of each one. At the end of this chapter is a sample test plan.

Purpose, goals, and objectives of the test
Research questions
Participant characteristics
Method (test design)
Task list

Test environment, equipment, and logistics
Test moderator role

Data to be collected and evaluation measures
Report contents and presentation

These parts are discussed in detail in the following sections, with the exception of the role of the test moderator, which gets its own chapter. Chapter 4, because it merits a special discussion.

It Provides a Focal Point for the Test and a Milestone

Without the test plan, details get fuzzy and ambiguous, especially under time pressure. The test plan forces you to approach the job of testing systematically, and it reminds the development team of the impending dates. Having said all that, it is perfectly acceptable, and highly probable, that the test plan will be developed in stages as you gradually understand more of the test objectives and talk to the people who will be involved. Projects are dynamic, and the best laid plans will change as you begin to approach testing.

It Defines or Implies Required Resources

The test plan describes or implies required resources, both internal and external. Once you delineate exactly what will happen and when, it is a much easier task to foretell what you will need to accomplish with your test. Either directly or by implication, the test plan should communicate the resources that are required to complete the test successfully.

It Serves as the Main Communication Vehicle

The test plan serves as the main communication vehicle among the main designer and developer, the test moderator, and the rest of the team. The test plan is the document that all involved members of the development team, as well as management (if it is interested and involved), should review in order

to understand how the test will proceed and to see whether their particular needs are being met You use it to get buy-in and feedback from other members to ensure that everyone agrees on what will transpire. Because projects are dynamic and change from day to day and from week to week, you do not want someone to say at the end of the test that his or her particular agenda was not addressed. 

Especially when your organization is first starting to test, everyone who is directly affected by the test results should review the test plan. This makes good business sense and political sense as well.

It Serves as a Blueprint for the Test

Much as the blueprint for a house describes exactly what you will build, the test plan describes exactly how you will go about testing your product. Just as you don't want your building contractor to "wing it" when building your house, so the exact same logic applies here. The test plan sets the stage for all that will follow. You do not want to have any loose ends just as you are about to test your first participant.

Why Create a Test Plan?

A sound approach is to start writing the test plan as soon as you know you wUl be testing. Then, as the project proceeds, continue to refine it, get feedback buy-in, and so forth. Of course, there is a limit to flexibility, so prior to the test you need to set a reasonable deadline after which the test plan may not

change. Let that date also serve as the point at which the product can no longer change until after the test. You may find that the test plan is the only concrete milestone at that point in time in the development cycle and, as such, serves an important function.

Once you reach the cutoff date, do all that you can to freeze the design of the product you will be testing. Additional revisions may invalidate the test design you have chosen, the questions you ask, even the way you collect data.
If you are pressured to revise the test after the cutoff date, make sure that everyone understands the risks involved. The test may be invalidated, and the product may not work properly with changes made so close to the test date.

Develop the Test Plan

The test plan is the foundation for the entire test. It addresses the how when where, who why, and what of your usability test. Under the sometime unrelenting time pressure of project deadlines, there could be a tenden ™ ° forgo writing a detailed test plan. Perhaps, feeling that you have a good idea of what you would like to test in your head, you decide not to bothfr writing t down This informal approach is a mistake, and it invariably will come back

Practice "Bare Attention"

;'Bare attention" practice is an adjunct to meditation practice exceot .hat » heighten your abUity to concentrate during test sessfons. TopcC attention set aside a period of time (15-30 minutes is more than enough to begin) when you .nationally and very deliberately heighten vour awarene^ of whatever you happen to be doing and of your surroundings: For exam!?, you are working at a computer, experience very deliberately the sense of vour fingers hitting the keys, of your eyes looking from the paper to the screen of

your thought process. Notice when (and how often) your mind wanders from what you are doing, and when it does, gently bring it back to the present task at hand. The intent is to stay in the present moment 100 percent of the time. Trv it sometime just to see how difficult it is. This practice, as with the previously
described meditation practice, helps to foster mindfulness and awareness

Practice "Bare Attention"

;'Bare attention" practice is an adjunct to meditation practice exceot .hat » heighten your abUity to concentrate during test sessfons. TopcC attention set aside a period of time (15-30 minutes is more than enough to begin) when you .nationally and very deliberately heighten vour awarene^ of whatever you happen to be doing and of your surroundings: For exam!?, you are working at a computer, experience very deliberately the sense of vour fingers hitting the keys, of your eyes looking from the paper to the screen of

your thought process. Notice when (and how often) your mind wanders from what you are doing, and when it does, gently bring it back to the present task at hand. The intent is to stay in the present moment 100 percent of the time. Trv it sometime just to see how difficult it is. This practice, as with the previously
described meditation practice, helps to foster mindfulness and awareness

Learn to Meditate

Meditation practice, specifically the type of meditation that fosters mindfulness and awareness, can be a valuable aid in learning to see clearly and in observing subtle nuances in behavior. This type of discipline is based on the belief that to understand another's mind, you first have to master your own.

Meditation practice or mindfulness training involves setting aside a period of time to sit down on a cushion and practice a simple breathing technique, while at the same time acknowledging thoughts that arise and letting them be. 

Over time, the result of this practice is a very personal and heartfelt recognition of how everything we perceive is filtered and biased by our version of things. Through continual practice, one's thoughts become more transparent, which in turn frees one to perceive more clearly and directly. During a test session,

this is exactly what an excellent test moderator attempts to do; observe the participant's behavior free from the tyranny of his or her own expectations and biases.

Practice Moderating Sessions

Start with the right attitude. Do not be a perfectionist. You are going to make mistakes, bias participants, reveal information you should not, and invent new ways to invalidate a test session's results. This is just par for the course.
Usability testing has a twofold saving grace — testing multiple participants and iterative design Testing multiple participants means that if you invalidate  something in one session, there is always anothe opportunity to do it right.
Iterative design also makes up for any mistakes you might make, because you have several chances throughout the product development lifecycle to catch problems with the product. The important thing is not to get discouraged. Continue to practice, continue to learn, continue to improve. Even the most
experienced test moderators make mistakes.

Describe How the Results Will Be Reported

This section provides a summary of the main sections of your test report and tlie way in which you intend to communicate the results to the development team. For the report contents section, simply list the sections that will appear in your test report, as shown in the example in Figure 5-9.

For the presentation section, describe how you wilt communicate results to the development team both prior to and following the report. For example, you might hold an informal meeting with those on the critical path of the project just after the test is completed and prior to analyzing all the data. Then,

Work with a Mentor a Mentor

Work closely with an experienced test „ it is a test with work on a test and have that moderator do the same .many participants, perhaps you can conduct some of the sessions. Have your mentor watch you and critique your performance. If you hire a consultant to help conduct a test, arrange for the consultant to work closely with you in a mentor/coaching relationship, so that you can learn faster than by just observing.