“Usability Testing” Continues To Be Confused With “User Feedback” and Other “Testing”
In 2013, I wrote a blog post called “Focus Groups Vs Usability Testing“. Eight years later, the confusion only seems to have gotten worse. Not only do people confuse Focus Groups with Usability Testing but many also confuse User Feedback and even QA Testing with Usability Testing.
In the early stages of a client engagement, we often ask if any usability tests have been performed on the software product we’re being asked to redesign. Sometimes we get responses such as “Sure, our tech support team gets feedback from our customers all the time.” or “Yes, our marketing agency tested the product with focus groups.” or “Our team of testers test every build before each product release.” Unfortunately, none of these are usability tests.
How User Feedback is not Usability Testing
Getting user feedback is great but it often comes in the form of a compliment or a complaint. While this is certainly useful, there may be many other opportunities for improvements that usability testing can reveal. In addition, we’ve also seen many instances where the users may be internal employees who are cautious about complaining about a product because they either don’t want to seem like a complainer or they’re afraid to insult someone in a higher position. This is why we encourage our clients to allow us to perform both user shadowing and usability tests without anyone else involved. Sure, we’ll record sessions so the client can playback a video of exactly what we observed but, when a user knows they’re being observed by an audience, they simply behave differently.
The other issue with getting user feedback is that it is often gathered by asking for the feedback, usually in the form of a survey or a user feedback form. Unfortunately, when you ask someone “What do you think of this?” you may as well be asking them to “Find something wrong with this.” In other words, the user feels they are being tested — not the product design — and they think have to find something wrong otherwise they’ll fail the test. In usability testing, we stress how we are testing the product’s design – not the users – so there is nothing they can do “wrong”. This simple statement that is said to the user prior to testing, helps the user relax, focus on the tasks they’re being asked to perform, and behave much more naturally.
How QA and UAT Testing is not Usability Testing
There have also been times when we’ve asked about usability testing and we’re told how the product has been thoroughly tested and is fully functional and bug-free. In other words, they hear the word “usability” and they think “functionality”. However, a product can be fully functional while also being nearly impossible to use. In Don Norman’s book, The Design of Everyday Things, he discusses how there’s an epidemic of poor door designs, which has now been coined as “Norman Doors” (see video below). In one example, the doors are fully functional but because of the poor design, numerous people were getting stuck between two sets of doorways. The point is, you can have a fully “functional” product that is nearly “unusable”.
How to overcome the confusion
The solution to the confusion we continue to see regarding usability testing is for those of us in the User Experience profession to do a better job at explaining and educating people on what it is, what it is not, and why it is so valuable. Much the way we UXers try not to blame the users for not understanding how to use a product, we shouldn’t blame non-UXers for not understanding what usability testing is. We need to do our best to get on our soapboxes and teach the masses. In fact, you do your part by sharing this post!
