Usability testing: watch five people struggle, fix it, then watch five more

Usability testing works best in small rounds: five realistic users, tasks that describe situations, fixes, then five more. Here is how, including for B2B experts.

Instant Expert EditorialPublished Updated 6 min read

Usability testing works best small: watch five realistic users try real tasks, fix what tripped them up, then test five more. A usability test is a session where someone uses your product or prototype to complete tasks while you watch. Nielsen Norman Group names three core parts: a facilitator who gives tasks and asks follow-ups, the tasks themselves, and a participant who is a realistic user (Usability Testing 101). Participants usually think out loud as they work.

It tells you whether people can use what you built, not whether they want it. That question belongs to customer interviews and the rest of product discovery.

The example we will use

Here is a made-up example. You build accounts-payable software for restaurant groups, and you have redesigned the screen where a finance manager approves supplier invoices before payment. Can finance managers spot an invoice that doesn't match its delivery, hold it, and approve the rest? (Somewhere, a case of lemons has been billed as a case of limes.)

1. Write one research question and test five people

Put the question in one line: "Can finance managers at multi-location restaurant groups approve a weekly invoice batch and hold the exceptions without help?" NN/g defines a realistic user as a current user, or someone with a similar background and the same needs.

For a typical qualitative study of one user group, NN/g recommends five participants. Jakob Nielsen argues that five users find most of the common problems, so a budget for 15 users is better spent on three rounds of five, fixing problems in between (Why You Only Need to Test with 5 Users). For two distinct user groups, Nielsen suggests three to four people from each; for three or more groups, three from each.

Five is for finding problems. For benchmark metrics such as task success rates, NN/g's follow-up on quantitative studies recommends 20 people.

In the example, controllers who approve payments and general managers who send invoices in are different groups. Test three or four of each.

2. Describe the situation, never the button

GOV.UK says good tasks set a clear goal, feel relevant and believable to the participant, are hard enough to reveal problems, and don't hint at the answer (Using moderated usability testing). NN/g warns that small wording errors can confuse participants or nudge them.

Weak taskBetter task
Click "Hold" on the invoice with a price mismatchThis week's produce invoice from one supplier looks higher than usual. Decide what to do with it, then pay everything else that is ready.
Use the filter to find invoices over $1,000Your owner wants to see the biggest bills before they go out. Find them.

The weak versions name the button.

Use real data when you can handle it securely. GOV.UK says participants working with their own documents are more engaged and reveal more than those using dummy data. Otherwise, use supplier names and amounts a restaurant group would recognize.

3. Script a session of 30 to 60 minutes

GOV.UK suggests a discussion guide that holds your introduction, the tasks and a planning checklist. A simple outline:

  1. Introduction (about 3 minutes). Explain what will happen, confirm consent and recording, and say you are testing the product, not them.
  2. Warm-up (about 5 minutes). Ask about their job and how they approve invoices today.
  3. Tasks (30 to 40 minutes). Read each task aloud, ask them to think out loud, then stay quiet. Step in only if they are completely stuck, with neutral questions such as "What makes you say that?"
  4. Wrap-up (5 to 10 minutes). Ask about anything you saw but didn't understand, then any final thoughts.

How to ask for recording consent covers the consent step, and how to take interview notes covers capturing what happened.

4. Moderate the session when the workflow depends on context

In a moderated test, a facilitator is present, in person or on a video call, and can ask follow-ups. In an unmoderated test, the participant works alone in a testing tool that records the session for you to watch later (Remote Usability Tests: Moderated and Unmoderated).

ModeratedUnmoderated
Follow-up questionsYes, specific to what the person didOnly the same pre-written questions for everyone
If the participant gets stuckYou can help them recoverNo real-time help; you find out afterward
Thinking aloudYou can prompt quiet participantsNo one is there to remind them
Best forAn overall review of a whole workflowA few specific elements or a small change; tight timelines

For remote studies, NN/g advises recruiting a few extra people: more may not show up, and some unmoderated sessions turn out unusable.

Choose moderated for a B2B workflow like invoice approval: you will want to ask why someone hesitated.

5. Give expert users harder tasks and watch more than you listen

In Testing Expert Users, Nielsen says the basics hold (representative users, realistic tasks, thinking aloud), but some things change for people who use a tool every day:

  • Give harder, deeper tasks. Experienced users can work through larger, more realistic problems.
  • Expect less narration. Skilled behavior is often automatic, so experts may not know why they did something. Watch what they do more than what they say.
  • Steer critics back to the task. Experts may turn into design reviewers. Accept the comments, then bring them back; behavior is the data you came for.
  • Recruit typical users, not stars. If account managers help recruit, ask for average users rather than the top performers managers like to show off.
  • Build experience for a new product. If nobody has used it yet, Nielsen suggests quick training or practice time before the test.

B2B participants are busy, so keep sessions short and pay for their time. Research interview budgets covers what to pay, and how to interview B2B users, buyers and champions covers who does what inside a customer.

6. Fix the shared problems, then test again with new people

After each session, mark each task as completed, completed with difficulty, or not completed, and note where it went wrong. Fix the worst problems that more than one participant hit, then test again with new people. Nielsen's case for small rounds depends on that second round: it checks whether your fixes worked and often finds deeper problems the first round's surface issues hid.

Your next step

Write one research question, three tasks that describe situations instead of buttons, and a one-page guide. Then book five sessions with people who do the job.

If your users are professionals outside your own customer list, Instant Expert can find people who match a description you write, such as "finance managers at multi-location restaurant groups." You review who it finds, it sends your invitations, and you pay only for calls that get booked. The finance professionals in restaurants directory page is one place to start.