How to conduct a moderated usability test in two weeks
Usability testing is a method of evaluating a design, product, or system by observing real users as they complete tasks. It helps identify usability issues, measure efficiency, and improve the overall user experience. This guide will tell you how to usability test your app or software product.
One common misconception we hear from our clients is that usability testing is time-consuming and expensive. This could not be further from the truth. Our usability tests typically take less than two weeks to complete. The insights we gain from usability testing help proactively catch usability issues before hours, days, and (yikes) even weeks are wasted developing untested experiences. Usability testing is a great way to mitigate risk by validating designs before development is started.
What is moderated usability testing?
Usability tests can be moderated or unmoderated. In a moderated test, a facilitator instructs the participants and guides them through the test. In an unmoderated test, an application is used to display tasks or questions and record participants as they complete each task. The most common approach we take with our clients is remote moderated testing. We typically use software, like Microsoft Teams, to conduct usability testing via webcam and screensharing. This allows us to ask contextual follow-up questions as participants go through the test and also helps reduce cost, since there are no facility fees.
When to choose moderated usability testing over unmoderated
It’s not that moderated testing is better than unmoderated testing. Both have their applications. However, if your product is extremely nuanced–for example, if you’re building enterprise software for a niche group of people, like your employees, it may be necessary to moderate the test, so that you can ask follow-up questions. The discussions can be more dynamic, and you can often learn more that way.
Resources Needed:
- Facilitator
The facilitator proctors the test by instructing participants on their tasks and asking clarifying questions. - Observer
The observer watches participants complete their tasks, notes whether the tasks were completed successfully, how much time it takes to complete each tasks and may also note any issues encountered. In some cases, the facilitator is the observer. Since we record the tests, researchers can rewatch sessions and take notes after the test is concluded. - AI Notetaker
Is it absolutely required? No. But it saves time. We use our AI Notetaker to group and organize the videos/transcripts with time stamps, so that we can easily go back and rewatch things that we have flagged ourselves, or things that it caught.
- Participants
Typically, we need around 5-8 participants for usability testing. Participants can be recruited from real users (especially if you have an established product), clientele, or third-party recruiters. - Prototype
The prototype can be low or high-fidelity. A low-fidelity prototype could be a simple as a wireframe or a mockup. Higher fidelity prototypes include Figma click-thrus, HTML/CSS, Angular, or React prototypes. Most often, we use Figma click-thru prototypes. - Tee-up email
The facilitator should send a tee-up email to tell the participants what to expect and instruct them on any software they will need to install or consent forms they need to sign in order to participate and anything else they might need to know.
- Test Script
The test script should include a brief introduction about the test and overall instructions as well as the tasks that the participants will be completing. - Rubric
A rubric is a list of tasks with the expected answer or behavior. We will use this to determine if the task was completed successfully.
Steps to Conduct a Moderated Usability Test in Two Weeks
- Define your goals (Days 1 to 2)
Start with the decisions your team needs to make. What research questions are you trying to answer? Clear goals keep the study focused and make the findings easier to act on. - Recruit participants (Days 1 to 5)
Start recruiting right away, since this step takes the longest. Write a short screener to make sure participants match your real users by role, experience level, and how often they’d use the product. Aim for 5 to 8 participants per user group, and recruit one or two extras in case of no-shows. Schedule sessions for Days 6 to 8. Niche B2B audiences can take longer to find, so build in extra time if you’re testing with specialists. - Prepare the prototype (Days 2 to 5)
Make sure the prototype supports every task in your study. For a Figma click-thru, walk each path from start to finish and check that every needed screen, link, and state is in place. Dead ends during a session waste time and distract participants from the real experience. - Write the test script (Day 3)
Your script should include:- A short introduction explaining the purpose of the session
- A reminder that you’re testing the design, so there are no wrong answers
- A request for participants to think aloud as they work
- A few warm-up questions about their role and experience
- Each task, written as a realistic scenario
- Wrap-up questions about their overall impressions
Word tasks carefully. “Order a replacement part for a door closer” works better than “Click the Products menu and find door closers,” because it describes a goal without pointing to the answer.
Build your rubric. (Day 4)
For each task, list the expected path or answer and define what counts as success, partial success, and failure. The rubric keeps scoring consistent across sessions and observers.- Send the invite and tee-up email. (Day 5)
Invite your users to their sessions. Send it the day of or day before each session. Include the date, time, and meeting link, how long the session will take, any software to install, consent forms to sign, and a note that the session will be recorded. Send a short reminder the day before. - Test the test (Day 5)
Run one practice session with a colleague or extra participant. A pilot reveals confusing task wording, prototype glitches, and timing problems while you still have time to fix them. Adjust the script and prototype based on what you learn. - Conduct the sessions (Days 6 to 8)
Three to four sessions per day is a sustainable pace for a facilitator. Leave at least 30 minutes between sessions to reset, catch up on notes, and handle late starts.
During each session, the facilitator reads tasks, keeps the participant talking, and asks neutral follow-up questions like “What are you looking for?” or “What did you expect to happen?” Resist the urge to help. When a participant gets stuck, that’s a finding. If a participant asks, “Where would I do X,” respond with something like “How would you expect to be able to do that?”
The observer records the response/behavior for each task in the rubric, and notes issues and memorable quotes. Encourage stakeholders to watch the clips from the recordings. Seeing a real user struggle builds buy-in faster than any report, but having them sit in live can sometimes intimidate the participants. - Analyze and synthesize (Days 9 to 10)
Review your notes, the rubric observations, and the AI notetaker’s flagged moments. Group issues into patterns and rate each one by severity and frequency. A problem that blocks task completion for most participants comes first. Pull short video clips that show the most important issues in action. - Share prioritized recommendations (Day 10)
Deliver a short readout with a prioritized list of fixes, supporting clips, and task success rates. Note what worked well, too, so the team knows which patterns to keep. End with clear next steps your team can act on right away.
