Usability Testing Basics: What QA Engineers Need to Know

An error message that says "An error occurred. Please try again." when a user registers with an already-taken email leaves them choosing between three wrong paths: new email, different password, or just retrying the same form. That confusion is a usability defect, and it's within QA's scope to catch and report. This article covers Nielsen's five usability dimensions, four testing methods from 5-minute guerrilla sessions to moderated sessions, what QA engineers can do without a dedicated UX process, and how to write usability bugs in a way that gets them taken seriously.

What usability testing is

Usability testing is structured observation of real users attempting to accomplish real tasks with your product. You watch what they do, listen to what they say, and note where they struggle.

The output isn't a pass/fail result. It's qualitative insight: where users get confused, what they try that doesn't work, what language they use that doesn't match your UI language, how long it takes them to accomplish something that should be simple.

The five components of usability (Nielsen)

Jacob Nielsen's usability model defines five dimensions:

Learnability: How easy is it for first-time users to accomplish basic tasks? Efficiency: Once learned, how fast can experienced users accomplish tasks? Memorability: When users return after not using the product for a while, how easily do they re-establish proficiency? Errors: How many errors do users make, how severe are they, and how easily can users recover? Satisfaction: How pleasant is the experience?

Different products weight these differently. A consumer checkout flow optimizes for learnability and error recovery. A power tool for daily professional use optimizes for efficiency.

Methods: from cheap to rigorous

Heuristic evaluation (no users required)

A QA engineer or UX specialist evaluates the UI against established usability heuristics. Nielsen's 10 heuristics are the standard:

1. Visibility of system status: does the user know what's happening?

2. Match between system and real world: does the language match user expectations?

3. User control and freedom: can users undo actions and escape situations?

4. Consistency and standards: does the UI follow platform conventions?

5. Error prevention: does the design prevent mistakes before they happen?

6. Recognition rather than recall: does the UI minimize memory load?

7. Flexibility and efficiency: are there shortcuts for expert users?

8. Aesthetic and minimalist design: is the UI free of irrelevant information?

9. Help users recognize, diagnose, and recover from errors: are error messages specific and helpful?

10. Help and documentation: is help available and easy to find?

This is fast and requires no users. It finds 30–50% of usability issues in a single review session.

Guerrilla usability testing

Five minutes, any user, any location. Show someone a prototype or the actual product. Give them one task: "Can you find where to reset your password?" Watch. Don't help. Take notes.

Five users per round. You'll find the most obvious problems quickly. This is usable with zero budget.

Moderated usability sessions

A facilitator works with one participant at a time. The participant "thinks aloud" while attempting tasks. The facilitator probes and follows up.

Remote moderated testing (via Zoom + screen share) makes this accessible. Tools like UserTesting or Maze provide participant panels.

Unmoderated usability testing

Participants complete tasks independently, recorded via screen capture. You review the recordings. More scalable than moderated sessions, less nuanced: no opportunity to probe.

What QA engineers can do without a formal UX process

During exploratory testing: Note moments of confusion or frustration. "The error message says 'Invalid input' with no indication of which field is invalid." That's a usability defect. Apply the 10 heuristics: Walk through any new feature with Nielsen's list. Check each one systematically. Ask "can a new user figure this out?": When testing a feature for the first time (without reading the spec), pay attention to your own confusion. You're acting as a proxy for new users. Test error recovery: What happens when a user makes a common mistake? Is the error message helpful? Is there a clear path to fix it? Test on mobile: Many usability issues are device-specific. Touch targets too small to tap accurately. Forms that require scrolling up and down to fill. Modals that overflow the screen.

Common usability defects to report

Error messages that don't specify what went wrong or how to fix it
Form fields that reset on error (user has to re-enter everything)
Required fields not marked as required until after failed submission
Buttons or links that look clickable but aren't, or aren't clickable but look like they should be
Confirmation dialogs with no clear distinction between confirm and cancel
Loading states that give no indication of progress or expected wait time
Actions with no undo capability (especially destructive actions)
Inconsistent terminology (same feature called different names in different parts of the UI)
Pagination with no indication of total results or current position

Writing usability defects

Usability defects are harder to write than functional ones because there's no clear expected result from a spec. Frame them as impact:

Bad: "The error message is confusing." Good: "When a user submits the registration form with an already-registered email, the error message shows 'An error occurred. Please try again.' Users cannot determine whether they should change their email address, use a different password, or try logging in instead. Recommend: show 'This email is already registered. Try logging in.' with a link to the login page."

The bug report explains the user impact and proposes a specific fix. This gets usability issues taken seriously instead of dismissed as "design opinions."