How many testers do you need before launch?
· 6 min read
"How many testers do I need before I launch?" You will hear "five" a lot. That number is real, but it comes with conditions that people skip. This post covers what the five-user finding says, where it stops applying, and how I would run small test rounds for a first app.
I checked the Nielsen Norman Group pages below in November 2026. I only use what is written on those pages, and I link each one. The short answer: for finding usability problems, a few testers per round is enough. For other goals, you need more.
What the "5 users" finding says
The source is Jakob Nielsen's article Why You Only Need to Test with 5 Users, published March 18, 2000. His claim is: "The best results come from testing no more than 5 users and running as many small tests as you can afford."
The reasoning is about repeats. He says the share of problems one user finds is, on average, "31%, averaged across a large number of projects we studied". He also says a first study with five participants "has found 85% of the usability problems". After a few users, you mostly watch the same problems again.
His advice is not to run one big test. It is to "spend this budget on 3 studies with 5 users each". You test, fix what you saw, and test again. A fix can fail to solve the problem, or create a new one, and only another round shows that.
The conditions
Two points from the same article matter for a maker:
- The users must be alike. Nielsen writes that "the formula only holds for comparable users who will be using the site in fairly similar ways".
- 31% is an average. It comes from the projects he studied. Your app can be different.
So "5 users" does not mean "5 users and you are done". It means five users per round, per group, for finding usability problems.
When NN/g says you need more
The 2000 article says: "You need to test additional users when a website has several highly distinct groups of users." For two groups, he suggests "3–4 users from each category". For three or more groups, he suggests "3 users from each category".
A later article, How Many Test Users in a Usability Study? (June 3, 2012), gives numbers for other kinds of study:
- Qualitative usability study: "test 5 users".
- Quantitative study: "Test at least 20 users to get statistically significant numbers; tight confidence intervals require even more users."
- Card sorting: "Test at least 15 users per user group."
- Eyetracking: "Test 39 users if you want stable heatmaps."
The same article says small tests are fine inside an iterative cycle: "it's okay to leave usability problems behind in any one version of the design". You catch them in a later round.
NN/g's current guide, Usability (User) Testing 101, still says: "we recommend using five participants to uncover the majority of the most common problems". It describes a typical qualitative study of one user group.
Testing is not the same as store rules
Some numbers you hear are not about finding problems at all. Google Play asks new personal developer accounts to run a closed test with a set number of testers for a set number of days. That is a rule about getting access to production. It says nothing about how many testers find your bugs. I wrote about that rule in Google Play 12 testers for 14 days, so I will not repeat it here. Treat the two goals separately. You may need twelve testers for Google and still learn most of what you need from five.
How I would run small rounds
This is my own advice, built on the iterative idea above. It is not a quote from NN/g.
- Pick 3 to 5 testers. Small enough that you actually read every report.
- Give one concrete task. "Sign up and create your first project" beats "try my app". I wrote a guide on this: how to write a test task.
- Watch for where people get stuck. Where they pause, tap the wrong thing or ask what something means. That is more useful than whether they liked it. NN/g's guide says the facilitator "observes the participant's behavior and listens for feedback", and that participants are often asked to think out loud. With remote testers you cannot watch, so ask them to write down where they hesitated.
- Fix the problems that several testers hit. One person's odd taste can wait. The same stumble twice cannot.
- Run another round. New testers if you can, because they have not seen the old version. Check that your fixes worked.
When more testers make sense
- You have different user groups. Per NN/g above, add a few testers for each group. A tool for teachers and students needs both.
- You need a spread of devices. Android runs on many phones and versions, and iOS users sit on different iOS versions. Five testers will not cover that. A crash on one device model can stay hidden.
- You handle payments or load. A checkout needs repeated tests, and a spike of traffic is not something five people can simulate.
- You want feedback on features, not usability. A beta with many users can tell you what people ask for. The five-user finding is about finding usability problems, so it does not answer this.
- You want numbers. If you need statistics, see the quantitative figure above.
A checklist for a first round
- One goal for the round, like "can a new user finish signup".
- One task of about 10 minutes, with a start link and a clear finish.
- A test login, if your app needs one. Use a throwaway account.
- 3 to 5 testers who are as close to your real users as you can get.
- A way to collect where they got stuck and what device they used.
- A list of what you will fix before round two.
Where TestSwap fits
On TestSwap you post web, iOS and Android apps with a link testers can open or install. You can add a test login that only signed-in testers see. Everyone starts with 3 credits, writing a review earns 1 and each review your app receives costs 1. The maker can rate each review 👍 or 👎, and a 👎 takes back the reviewer's credit.
The testers are other makers, not your target users. I want to be plain about that. Makers are good at spotting crashes, confusing flows and weak first impressions. They are not a replacement for your real audience. If you build for dentists, test with a few dentists too. I make no promise about how many testers you will get.
Used well, it is a cheap way to run the early rounds: find the obvious problems before you spend time recruiting your real users. For other free ways to find testers, see how to find beta testers for your app. For why the swap works as it does, here is why I built TestSwap.