📊 Analytics — A/B Test Framework Demo

• Z-test for proportions — Industry-standard two-proportion z-test for conversion rate comparisons
• Chi-squared test — Alternative significance test (2×2 contingency table)
• Sample size calculator — Know how many visitors you need before starting
• Confidence levels — 90%, 95%, 99% with correct z-score thresholds
• Simulation mode — Generate synthetic experiments to test your setup
• Relative & absolute lift — Both metrics reported for full context

# Analyze experiment results from a JSON file
python src/ab_test_framework.py analyze --results examples/sample_experiment.json

# Analyze with 99% confidence level
python src/ab_test_framework.py analyze --results examples/sample_experiment.json --confidence 0.99

# Simulate an experiment (5000 visitors, 3.2% vs 4.1% conversion)
python src/ab_test_framework.py simulate --visitors 5000 --rate-a 0.032 --rate-b 0.041

# Calculate minimum sample size
python src/ab_test_framework.py sample-size --baseline 0.05 --mde 0.10

# Save results to JSON
python src/ab_test_framework.py analyze --results data.json --output report.json

{
    "control": {
        "name": "Original Checkout",
        "visitors": 5000,
        "conversions": 160
    },
    "treatment": {
        "name": "New Checkout",
        "visitors": 5000,
        "conversions": 205
    }
}