by myuser.ai
Get it free on Shopify or Wix
Free tool

How many session recordings should you watch?

Say how often a problem happens and how sure you want to be of seeing it, and get the number of replays to watch. Or enter the replays you plan to watch and see your chances. It's the math behind testing with five users, applied to real visits.

Your question

How sure you want to be of seeing it at least once

Count only the visits you pick from: if you open visits that reached your cart, how often the problem happens among those. Nothing you enter leaves your browser.

Replays to watch

Replays to watch22
Chance with your 1065%
Seen twice or more in 1026%
Saw nothing in 10?Under 1 in 4

To have a 90% chance of seeing a problem that happens on 1 in 10 visits at least once, watch 22 replays. With 10, you'd see it at least once 65% of the time, and twice or more 26% of the time.

The arithmetic, step by step

  1. Replays to watch: ln(1 − 0.90) ÷ ln(1 − 1/10) = 21.9, rounded up to 22.
  2. With 10: 1 − (1 − 1/10)^10 = 65% chance of seeing it at least once, and 26% of seeing it twice or more.
  3. On average: 10 × 1/10 = 1 sighting in 10 replays.
  4. Saw nothing in 10? 1 − (1 − 0.90)^(1/10) = 20.6%, so with 90% certainty it happens on fewer than about 1 in 4 of those visits.

Replays needed to see a problem at least once

The same math for common cases. Each figure is the number of replays to watch for that chance of seeing the problem at least once.

The problem happens on80% sure90% sure95% sure
1 in 2 visits (50%)345
1 in 3 visits (33.3%)468
1 in 5 visits (20%)81114
1 in 10 visits (10%)162229
1 in 20 visits (5%)324559
1 in 50 visits (2%)80114149
1 in 100 visits (1%)161230299

How does the calculator work?

If a problem happens on a share p of the visits you pick from, the chance of seeing it at least once in n replays is 1 − (1 − p)n. It's the model behind Jakob Nielsen's advice to test with five users: one tester shows almost a third of the problems, so five show about 85%1. The calculator turns it around to find n, rounds it up, and also works out:

  • Your chances with the replays you plan to watch, of seeing the problem once, and twice or more, which is when it starts to look like a pattern.
  • What seeing nothing tells you. If n replays show no sign of it, the problem likely happens on fewer than 1 − (1 − c)1/n of the visits, with the certainty c you picked.

Jeff Sauro at MeasuringU works the same sums: for an 85% chance of seeing a problem that affects 1 in 10 people, plan on 182. The calculator gives the same 18.

Pick the visits where the problem lives

The fewer unrelated visits you pick from, the more common the problem is among them. Say a cart problem hits 1 in 20 of all visits, but 1 in 4 of the visits that reach the cart. Ten replays from all visits would show it 40% of the time; ten that reached the cart, 94% of the time. Find the step that loses the most with the conversion funnel calculator, then watch visits that reached it.

The math also assumes each replay is a fair pick, such as the latest ones or every visit that reached a step, rather than the most eventful. Our guides suggest about ten to start: enough to catch a problem on 1 in 4 of them 94% of the time, and a reason to watch ten more when every visit looks different. How to watch session recordings faster keeps ten to a few minutes.

Session Recordings

Watch more replays in less time

Session Recordings lists each click and typing moment beside the replay, skips the idle parts and plays at up to 8x, so ten visits take minutes. Free forever: 2,000 recordings a month.

Questions

Watching enough replays, answered

How many session recordings should I watch?

Enough to see the problems that matter at least once. A problem on 1 in 10 of the visits you pick from needs 22 replays for a 90% chance of seeing it; one on 1 in 4 needs 8. Start with about ten visits that reached the step you're fixing, and watch more when every visit looks different.

Why do usability tests use five people?

Jakob Nielsen found that one tester shows almost a third of a design's usability problems, so five show about 85% of them. That 85% covers problems that affect about a third of people. For a problem that affects 1 in 10, Jeff Sauro at MeasuringU puts it at 18 people for an 85% chance of seeing it once.

What if I watched 20 replays and never saw the problem?

Then it's probably rare among the visits you picked from: with 90% certainty, it happens on fewer than about 1 in 9 of them. It could still be common somewhere else, such as on phones, so pick from the visits that matter.

Does it matter which replays I pick?

Yes. The math assumes each replay is a fair pick: the latest ones, or every visit that reached a step, not only the ones with lots of clicks. Picking from the visits that reached the step you're fixing makes the problem more common among them, so fewer replays find it.

See your next visit through your visitor's eyes

Install it on Shopify and switch it on in your theme, or add it to your Wix site. Recording starts with your next visitor.

Add to Shopify Add to Wix Free forever · Shopify and Wix · Form typing masked in the browser