Can a designer build an engaging, effective user experience without usability testing? And does usability testing assure a successful user experience?
Paul Sherman‘s recent article Usability Testing ≠ A Good User Experience attempts to establish that usability testing doesn’t guarantee good user experience. Claiming that the only reliable method to developing a good user experience is “strategic design”, the author asserts that focusing on “tactical usability” is a myopic exercise.
I certainly agree that rigid, task-focused usability testing is unlikely to glean new insight into a product or service’s user experience. And a usability test can often hinge on the quality of the script and system and the personalities of the moderator and participant. But does user testing only elucidate whether people can successfully complete a given task? I have my doubts.
Dr. Sherman’s argument that usability testing doesn’t quantify user enjoyment is not new. Long before User Centered Design principles were accepted in development teams, software interface designers and developers still needed to assess the usability of the systems they unleashed to their clients.
In an effort to de-humanize usability testing and provide quantitative results, engineers often relied on GOMS (Goal, Operators, Method, Selection) and Keystroke Level Models to assess the usability of existing systems.
Often these findings would be presented in numerical reports claiming that Feature A was 1.4% more usable than Feature B under strict, predictable circumstances. Such tests proved effective enough for years of practical use, though today they almost seem quaint when compared to the user-driven research practices of the present.
Today, most software developers and their management teams need and want usability tests to reveal how people feel about their application, while also confirming their audience can successfully complete step 2 before attempting step 3.
Usability scripts can elicit how people feel about a system, and a moderator tasked with observing participants’ interactions and emotions can also glean valuable insight, at times even tabling task-based questions to prompt a particularly effusive participant to expand on how she feels about the system. Granted, you can’t “tactically” assess those emotional reactions, but it’s also safe to say it’s difficult to fundamentally quantify any user experience as well.
Dr. Sherman continues that usability testing cannot “craft a unified user experience, plan for tomorrow’s user experience, [and] create delight, loyalty, and stickiness”. Well, that’s certainly true. But a usability test or other user research can identify if the user accepts and welcomes the unified experience from screen to screen and through the system’s interactions.
And a usability test can also provide insight into how to build future releases of software or web sites as well, or what may not be important at all.
Perhaps most importantly, I could argue it’s the usability test that will prove whether the designer is creating a system that is capable of instilling delight and loyalty. I’d much rather stand behind a system that’s been put in front of my target audience and generated feedback than trust the words of my designer that he sprinkled some “strategic design” into his workflow.
Back in 2007, UX Matters published Connecting Cultures, Changing Organizations: The User Experience Practitioner As Change Agent wherein Dr. Sherman wrote
Above all, our goal is to bring the users’ wants, needs, abilities, and limitations into the organization, and ensure that the organization doesn’t forget the user during any of the design and development stages
In my experience, usability testing is a prime opportunity to confirm your system is built with an understanding of those wants, needs, abilities and limitations.
