Contents

Chapter 1

Features

This chapter covers the core features and capabilities of A/B Test Framework.

Features

  • Z-test for proportions — Industry-standard two-proportion z-test for conversion rate comparisons
  • Chi-squared test — Alternative significance test (2×2 contingency table)
  • Sample size calculator — Know how many visitors you need before starting
  • Confidence levels — 90%, 95%, 99% with correct z-score thresholds
  • Simulation mode — Generate synthetic experiments to test your setup
  • Relative & absolute lift — Both metrics reported for full context
  • Pretty terminal output — Clear, formatted results with winner declaration
  • JSON export — Machine-readable results for automation pipelines

Requirements

  • Python 3.10+
  • No external dependencies (stdlib only)
Chapter 2

Quick Start

Follow this guide to get A/B Test Framework up and running in your environment.

Quick Start

bash
# Analyze experiment results from a JSON file
python src/ab_test_framework.py analyze --results examples/sample_experiment.json

# Analyze with 99% confidence level
python src/ab_test_framework.py analyze --results examples/sample_experiment.json --confidence 0.99

# Simulate an experiment (5000 visitors, 3.2% vs 4.1% conversion)
python src/ab_test_framework.py simulate --visitors 5000 --rate-a 0.032 --rate-b 0.041

# Calculate minimum sample size
python src/ab_test_framework.py sample-size --baseline 0.05 --mde 0.10

# Save results to JSON
python src/ab_test_framework.py analyze --results data.json --output report.json

Input Format

JSON

json
{
    "control": {
        "name": "Original Checkout",
        "visitors": 5000,
        "conversions": 160
    },
    "treatment": {
        "name": "New Checkout",
        "visitors": 5000,
        "conversions": 205
    }
}

CSV

csv
variant,visitors,conversions
Control,5000,160
Treatment,5000,205
Chapter 3
🔒 Available in full product

Output

Chapter 4
🔒 Available in full product

FAQ

You’ve reached the end of the free preview

Get the full A/B Test Framework and unlock everything.

All Chapters

Get the complete guide with every chapter unlocked, including code samples, diagrams, and best practices.

Full Tool Suite

Access all interactive tools with complete data, all workload profiles, and the full scenario library.

Source Files

Downloadable source code, configuration files, and working examples from every chapter.

Lifetime Updates

Free updates for life. Every new chapter, tool, and improvement included.

Buy Now — $29 →
📦 Free sample included — download another copy for the full product.
A/B Test Framework v1.0.0 — Free Preview