How many users are enough?
Virzi (1992) found four or five participants reveal about 80% of problems, and Nielsen and Landauer (1993) modelled discovery by sample size. Nielsen (2000) therefore advises three rounds of five rather than one of fifteen. Faulkner (2003) countered that five-user samples vary widely by chance. Five suits iterative qualitative testing; comparing numbers needs larger samples or A/B testing.
Designing and running tasks
Good tasks give goals and context, such as 'You get too many notifications and want fewer', not 'Find settings'. Think-aloud (Ericsson & Simon, 1993) asks participants to verbalise while moderators intervene minimally. Hertzum and Jacobsen (2001) report a strong evaluator effect, so have designers, developers and PMs observe together. Combine success, time and errors with SUS interpreted against Bangor et al. (2008).
Tips for mobile tests
Test on participants' own phones, held one-handed as usual. In remote sessions add a camera showing finger movement alongside screen recording. Polished mock-ups win verbal praise through the aesthetic-usability effect, so focus on behaviour. An expert review with Nielsen's heuristics beforehand removes obvious issues.
FAQ
- Are five users really enough?
- For iterative qualitative rounds yes; for metrics or multiple user groups you need more.
- Remote or in-person?
- Remote is fast and diverse; in-person allows deeper observation of context and expression.
Sources reviewed: 2026-10-27
References
- Nielsen, J., & Landauer, T. K. (1993). A mathematical model of the finding of usability problems. In Proceedings of the INTERACT '93 and CHI '93 Conference on Human Factors in Computing Systems (pp. 206–213). ACM. Gives the formula for problems found by number of participants.
- Nielsen, J. (2000, March 19). Why you only need to test with 5 users. Nielsen Norman Group. Argues for many small iterative tests.
- Virzi, R. A. (1992). Refining the test phase of usability evaluation: How many subjects is enough? Human Factors, 34(4), 457–468. Four or five subjects find about 80% of problems.
- Faulkner, L. (2003). Beyond the five-user assumption: Benefits of increased sample sizes in usability testing. Behavior Research Methods, Instruments, & Computers, 35(3), 379–383. Shows five-user samples vary widely.
- Bangor, A., Kortum, P. T., & Miller, J. T. (2008). An empirical evaluation of the System Usability Scale. International Journal of Human–Computer Interaction, 24(6), 574–594. Provides large-scale norms for interpreting SUS scores.
- Hertzum, M., & Jacobsen, N. E. (2001). The evaluator effect: A chilling fact about usability evaluation methods. International Journal of Human–Computer Interaction, 13(4), 421–443. Different evaluators find very different problems.
- Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (Rev. ed.). MIT Press. Theoretical basis of the think-aloud method.