Scorecard Comparison

SIDE-BY-SIDE IMPACT ANALYSIS ACROSS EXPERIMENTS

Select Experiments to Compare
Value Rankings
EXPERIMENTS RANKED BY OVERALL SCORE
#1
Checkout Flow RedesignShip

Growth • Projected $2.4M annual revenue increase

87

3/4 sig.

#2
Search Ranking v3Iterate

Search • Projected $800K annual revenue increase if latency fixed

72

3/4 sig.

#3
New Pricing TierKill

Revenue • Projected $1.2M annual revenue loss

45

2/4 sig.

Metric-by-Metric Comparison
LIFT AND SIGNIFICANCE ACROSS ALL EXPERIMENTS
MetricCheckout Flow RedesignSearch Ranking v3New Pricing Tier
Conversion Rate
+18.8%3.2% → 3.8%p=0.003
——
Cart Abandonment
-10.3%68% → 61%p=0.01
——
Avg Order Value
+4.7%85$ → 89$
——
Page Load Time
-14.3%2.1s → 1.8sp=0.005
——
Search CTR—
+18.2%22% → 26%p=0.008
—
Zero Results Rate—
-27.1%8.5% → 6.2%p=0.001
—
Search Latency—
+50.0%120ms → 180msp=0.001
—
Revenue per Search—
+6.7%0.45$ → 0.48$
—
Revenue per User——
-12.0%12.5$ → 11$p=0.02
Upgrade Rate——
-7.7%5.2% → 4.8%
Churn Rate——
+22.6%3.1% → 3.8%p=0.01
NPS Score——
-9.5%42 → 38
Checkout Flow Redesign
Ship
Growth • 2026-02-15

KEY FINDINGS

  • Mobile conversion improved 24%
  • Desktop neutral
  • Reduced checkout steps from 4 to 2

BUSINESS IMPACT

Projected $2.4M annual revenue increase

Search Ranking v3
Iterate
Search • 2026-02-12

KEY FINDINGS

  • Relevance improved significantly
  • Latency regression needs optimization
  • Long-tail queries benefited most

BUSINESS IMPACT

Projected $800K annual revenue increase if latency fixed

New Pricing Tier
Kill
Revenue • 2026-02-10

KEY FINDINGS

  • Enterprise segment revenue dropped 15%
  • SMB segment neutral
  • Increased churn in first 30 days

BUSINESS IMPACT

Projected $1.2M annual revenue loss