AB Testing in Livesport

Small changes — rewording a CTA or repositioning a badge — often moved CTR more than full redesigns did

The vast majority of traffic came from mobile, so desktop-first assumptions consistently undermined test validity

Fake-door tests repeatedly saved development capacity by disproving demand for features that looked promising internally

Each experiment began with a hypothesis grounded in observed behaviour — a heatmap anomaly, drop-off in an affiliate funnel, a pattern in session recordings. I sketched multiple variants, from microcopy changes to component rearrangements, and narrowed to the strongest candidates. Before anything was built, every variant was tied to one primary metric and a set of guardrail metrics — typically CTR and FTD revenue.

Variants were designed in Figma and specified for engineering handoff. Post-launch, I monitored results until statistical significance — applying the p-value discipline I introduced to the team — and documented every outcome, positive or negative, in a shared learnings library. Failed experiments were treated as findings, not failures; they fed the next cycle of hypotheses.

Measurable CTR and FTD conversion uplifts on key monetised placements, validated through controlled experiments

A team-wide research methodology built on Microsoft Clarity heatmaps and session recordings

Proper statistical evaluation

p-values and significance thresholds — adopted as standard team practice

On a mature, high-traffic product, small validated changes consistently outperform dramatic redesigns. The real craft is measurement discipline: one hypothesis, one primary metric, honest guardrails, and the patience to wait for significance. Some of the most valuable experiments were the ones that failed — a cheap fake-door test that kills a weak feature idea saves months of engineering.