Module 5 · Measuring and improving over time Lesson 13 of 15

A/B tests and the one-change rule

The concept

Decide the metric and the minimum sample before you start; stopping the moment one side is ahead is how random noise becomes a confident wrong decision.

Most tests end in "no difference". Write that down too - knowing what does not matter is as useful as knowing what does.

Plain textCopy

Test: CTA wording on the tool page

A (control):  "Check your score - free"
B (variant):  "See what's holding you back - free"
Change:       the CTA line only (same page, same traffic)
Metric:       tool completions per visitor
Sample:       ~400 visitors per side
Result:       B +18% completions -> keep B

Worked example

Version A and version B differ only in the call-to-action line, and the result tells you which line earns more clicks.

Your practice task

Design one A/B test with a single difference, and write the metric and the sample you would need.

Do this before moving on. Reading a solution teaches the shape of it; typing it teaches you.

Key takeaway

Decide the metric and the minimum sample before you start; stopping the moment one side is ahead is how random noise becomes a confident wrong decision.

Almost there

15 lessons left of 15

Finish every lesson to unlock the certificate. Each one ends in a practice task — do them and the final project brief writes itself.

Next recommended course UI/UX Design Fundamentals