All posts

Beta feedback triage: from tester notes to a fix list

· 7 min read

Your testers finally answered. Now you have a pile: a few emails, some notes from a form, a screenshot with no explanation, and "it feels slow". Some of it is a real bug. Some is a wish. Three people said the same thing in three different ways. Where do you start?

This post is a simple routine for turning that pile into a short fix list before launch. It works for 5 testers or 20. You need a spreadsheet and about an hour.

Step 1: Put everything in one list

Feedback arrives in many places: email, chat, TestSwap reviews, a testing feedback page in your store console. Don't triage it where it lives. Copy every note into one sheet first. One row per note. If one email holds three issues, make three rows.

Use these columns:

  • Who: the tester's name or handle.
  • Where: which screen or step.
  • What they did: the action, in their words if possible.
  • What happened: what they saw or felt.
  • Device: phone or computer, OS version, browser if it's a web app.
  • Type, count, severity, decision: empty for now. You fill these in below.

Don't judge yet. Don't fix anything yet. A fix you make while reading note 3 might make note 9 irrelevant, or worse, hide the pattern.

Step 2: Sort each row into a type

Give every row one of three types:

  • Bug: something is broken. A crash, an error, a button that does nothing, wrong data.
  • Usability problem: it works, but people get stuck. They can't find the button, they misread a label, they don't know what to do next.
  • Wish: a new feature. "It would be great if it had dark mode."

The line between a bug and a usability problem is not always clean, and that's fine. Pick the closer one and move on. The line that matters is the third one. Keep wishes out of your pre-launch list. Move them to a separate tab called "Later". Wishes are useful, but they are how a two-week launch becomes a two-month one.

One exception: if a wish shows up because the app can't do its main job, it's really a missing basic. Ask yourself whether a new user could finish the core task without it. If not, it's a bug in disguise.

Step 3: Group duplicates and count them

Read down the sheet and group rows that describe the same thing. "Couldn't find where to add a habit", "Plus button is hidden" and "Took me a minute to start" may all be one problem. Give each group one short name and write the number of different testers who hit it.

Count people, not notes. One tester who complains three times is still one tester. This count is your frequency. Four out of seven testers tripping on the same step is a much stronger signal than one person's strong opinion.

Be careful with small numbers too. With only a handful of testers, one report can still be real. A crash that one person saw is still a crash. You'll weigh it in step 5. If you're unsure how many testers to aim for, see how many testers you need before launch. A crash reporting tool shows how many people really hit a crash, not only the ones who wrote it down: crash reports before launch.

Step 4: Reproduce before you fix

Before you change code, make the problem happen on your own device. A fix you can't test is a guess. For each bug group, write down:

  1. Steps: the exact taps or clicks, from a clean start.
  2. Device and OS: model, OS version, browser.
  3. Expected: what should have happened.
  4. Actual: what did happen, including any error text.

Reports that already have these four, plus a screenshot or recording, save you most of this work. How to ask for them: bug reports testers can act on.

If you can't reproduce it, don't delete it. Mark it "can't reproduce" and ask the tester. Vague notes are normal. Testers are not professional QA people. Two or three short questions fix most of them:

  • "What were you trying to do right before it happened?"
  • "What device and OS version are you on, and was it the app or the browser?"
  • "Could you send a screenshot or a short screen recording, and tell me what you expected to see?"

Keep each question easy to answer in one line. The longer your message, the less likely you get a reply. You can prevent many vague notes in the first place by asking for the right feedback up front. See how to write a test task.

Step 5: Rate severity

Now you have grouped, counted problems. You can't fix them all, so rate each one. For usability problems there is a well-known method. I read the Nielsen Norman Group article How to Rate the Severity of Usability Problems on 10 November 2026. It names three factors that together decide how serious a problem is:

  • Frequency: "Is it common or rare?"
  • Impact: "Will it be easy or difficult for the users to overcome?"
  • Persistence: whether it is a one-time problem that users can overcome once they know about it, or one that keeps bothering them.

The article also gives a 0 to 4 rating scale:

  • 0: "I don't agree that this is a usability problem at all"
  • 1: "Cosmetic problem only: need not be fixed unless extra time is available on project"
  • 2: "Minor usability problem: fixing this should be given low priority"
  • 3: "Major usability problem: important to fix, so should be given high priority"
  • 4: "Usability catastrophe: imperative to fix this before product can be released"

You already have the frequency from step 3. Impact and persistence you judge from the notes and from your own reproduction. The same article says severity ratings from a single evaluator are "too unreliable to be trusted". As a solo maker you can't fully fix that. You can lower the risk by asking one friend to rate your top ten, then comparing.

For bugs, a simpler split is enough. It's a suggestion, not a standard:

  • Blocker: the user can't finish sign-up, payment or the main task, or the app crashes.
  • Major: the task can be finished, but something important is wrong or painful.
  • Minor: small glitch, easy to work around.

Step 6: Decide what to fix before launch

Use a short rule set, so you don't argue with yourself on every row. Fix it before launch if any of these is true:

  1. It blocks sign-up, payment or the main task. Blockers and NN/g 4s always go first.
  2. Several testers hit it. A count of 3 or more out of a small group is hard to ignore.
  3. It's cheap. If it takes ten minutes, just do it, even if it's minor.

Everything else goes on the "Later" tab, with the wishes. That's not ignoring it. Write the date and the count next to each item. After launch, real users will tell you which of those items matter, and your list will already be sorted.

A good pre-launch list is short. Five to ten items is normal. If you end up with thirty, your rules are too loose or the app isn't ready, and it's better to know that now.

Step 7: Close the loop with testers

This step is the one people skip, and it's the one that makes testers come back. After you fix something, reply to the person who reported it:

  • Thank them for the specific thing they found.
  • Say what you changed, in one sentence.
  • Ask the same tester to check the fix. They already know the problem, so they will spot it fast if it's still there.

If you decided not to fix something, say so briefly and say why, for example "I'm saving this for after launch". Most testers are fine with that. Silence is what feels bad.

A short template: "Thanks for catching the sign-up error on your Pixel. I changed the form so it no longer clears your email. Could you try signing up again and tell me if it works?"

Where the feedback lives in the stores

I read Google's ratings and reviews page on 10 November 2026. It says that when your app is in testing you can read and reply to feedback in Play Console, on the "Testing feedback" page under "Ratings and reviews". It also says this beta feedback "is only visible to you and can't be seen on Google Play". You can filter it by criteria such as date, language and app version. For the Google side of testing, see how to add testers to a closed test. For Apple, see the post on TestFlight external testers and the public link.

Where TestSwap fits

TestSwap is a free site where app makers review each other's apps: you test someone's app, they test yours. Each review needs a real first impression and a concrete suggestion, and you can rate each review 👍 or 👎. That makes it a good way to get a first batch of rows for your sheet.

A review has a private bug notes box that only you can see, so crashes, error text and device details can arrive already in the shape step 4 needs. TestSwap reviews are one input among several. You still do the triage yourself.

Checklist

  1. Copy all feedback into one sheet, one row per note.
  2. Add who, where, what they did, what happened and device.
  3. Mark each row as a bug, a usability problem or a wish.
  4. Move wishes to a "Later" tab.
  5. Group duplicates and count testers, not notes.
  6. Reproduce each bug: steps, device and OS, expected, actual.
  7. Ask vague reporters two or three short questions.
  8. Rate severity: the 0 to 4 scale for usability problems, blocker, major or minor for bugs.
  9. Fix blockers, problems that several testers hit, and cheap fixes. Everything else goes to "Later".
  10. Tell each tester what you changed and ask them to re-check.

Try it

If you want a fresh round of feedback to triage, post your app on TestSwap with a test task that points at the part you're least sure about. New accounts start with 3 credits, and reviewing someone else's app is the quickest way to see what a good review looks like. Tell me what's missing or broken (@dizid on X).