Conversion Optimization (CRO): A/B Test Methodology

Turning the same traffic into more customers — that's the entire summary of CRO (Conversion Rate Optimization). Your revenue grows without increasing your ad budget. But CRO isn't "a matter of intuition"; it's a systematic A/B testing discipline.

01. Hypothesis-Based Testing

Every A/B test starts with a hypothesis. "If the color is green instead of blue, CTA clicks increase by 10%." A test done without a hypothesis is just a comparison of random variation.

02. Sample Size

An adequate sample is needed for statistical significance. Small-change tests (button color) require a larger sample; big changes (a whole page) require fewer.

Tools like Optimizely, VWO and Google Optimize have a calculator.

03. Control vs Variant

Once a visitor has seen one version, they should see the same one on subsequent visits. Tracked with a cookie. Mixed assignment blurs the result.

04. Statistical Significance

A 95% confidence level is standard. This says that the probability that the observed difference is due to randomness is below 5%.

The test shouldn't be stopped before hitting this threshold. A test stopped early gives a wrong result.