Data Cleaning Playbook

$19

Step-by-step data quality frameworks: deduplication, standardization, outlier detection, missing value strategies, and validation rules.

📁 30 files

YAMLMarkdownPython

📄 Product Preview

Try the interactive reader and demo tools below, or get the full product with all content unlocked.

📖 Interactive Reader (Free Preview) ⚙ Try Demo Tools 📦 Download Free Sample

📁 File Structure 30 files

data-cleaning-playbook/ ├── LICENSE ├── README.md ├── examples/ │ ├── __pycache__/ │ │ └── cleaning_pipeline.cpython-312.pyc │ ├── cleaning_pipeline.py │ └── messy_customers.csv ├── free-sample.zip ├── guide/ │ ├── deduplication_strategies.md │ ├── missing_data_playbook.md │ └── outlier_detection_guide.md ├── guides/ │ ├── deduplication_strategies.md │ ├── missing_data_playbook.md │ └── outlier_detection_guide.md ├── index.html ├── requirements.txt ├── rules/ │ ├── common_patterns.yaml │ └── data_quality_rules.yaml └── src/ └── cleaners/ ├── __init__.py ├── __pycache__/ │ ├── __init__.cpython-312.pyc │ ├── deduplication.cpython-312.pyc │ ├── main.cpython-312.pyc │ ├── missing_values.cpython-312.pyc │ ├── outliers.cpython-312.pyc │ ├── standardization.cpython-312.pyc │ └── validation_engine.cpython-312.pyc ├── deduplication.py ├── main.py ├── missing_values.py ├── outliers.py ├── standardization.py └── validation_engine.py

📖 Documentation Preview README excerpt

Data Cleaning Playbook

A comprehensive toolkit combining deep playbook guides with runnable Python cleaners. Covers deduplication, standardization, outlier detection, missing-value strategies, and a rule-based validation engine.

Features

Deduplication Engine — Exact-match, fuzzy-match, and time-window dedup strategies
Standardization — Date formats, addresses, names, phone numbers, categorical encoding
Outlier Detection — IQR, z-score, Modified Z-Score (MAD), and isolation-based methods
Missing Value Strategies — Pattern analysis, imputation hierarchy, and indicator generation
Validation Rule Engine — Declarative YAML rules for automated data quality checks

Quick Start


pip install -r requirements.txt
python -m src.cleaners.main

Structure


src/cleaners/
├── __init__.py            # Package exports
├── main.py                # Demo runner
├── deduplication.py       # Exact and fuzzy deduplication
├── standardization.py     # Date/name/address standardizers
├── outliers.py            # IQR, z-score, MAD outlier detection
├── missing_values.py      # Missing value analysis and imputation
├── validation_engine.py   # YAML-driven rule validation
rules/
├── data_quality_rules.yaml  # Example validation ruleset
├── common_patterns.yaml     # Regex patterns for data standardization
examples/
├── messy_customers.csv      # Sample messy data for testing
├── cleaning_pipeline.py     # End-to-end cleaning example
guides/
├── deduplication_strategies.md   # When to use which dedup approach
├── outlier_detection_guide.md    # Choosing the right detection method
├── missing_data_playbook.md      # Decision framework for missing values

Usage

Detect and Remove Duplicates


from src.cleaners.deduplication import find_duplicates, deduplicate
dupes = find_duplicates(df, subset=["email"], method="exact")
clean_df = deduplicate(df, subset=["email"], keep="latest", date_col="created_at")

Standardize Dates and Names


from src.cleaners.standardization import standardize_dates, standardize_names
df["date"] = standardize_dates(df["date_raw"], output_format="%Y-%m-%d")
df["name"] = standardize_names(df["name_raw"])

Detect Outliers



*... continues with setup instructions, usage examples, and more.*

📄 Code Sample .py preview

src/cleaners/deduplication.py """ Deduplication engine supporting exact-match, fuzzy-match, and time-window strategies. Identifies and removes duplicate records using configurable matching logic. """ from __future__ import annotations import logging import re from dataclasses import dataclass from typing import Literal import numpy as np import pandas as pd logger = logging.getLogger(__name__) @dataclass class DuplicateReport: """Summary of duplicate detection results.""" total_rows: int duplicate_groups: int duplicate_rows: int duplicate_pct: float sample_duplicates: pd.DataFrame def find_duplicates( df: pd.DataFrame, subset: list[str] | None = None, method: Literal["exact", "normalized", "fuzzy_simple"] = "exact", ) -> DuplicateReport: """Identify duplicate records in a DataFrame. Args: df: Input DataFrame. subset: Columns to check for duplicates. None means all columns. method: Matching method. - 'exact': Exact string match on raw values - 'normalized': Match after lowercasing and stripping whitespace - 'fuzzy_simple': Match after removing non-alphanumeric chars Returns: DuplicateReport with counts and sample duplicate groups. """ work_df = df.copy() if method == "normalized" and subset: # ... 120 more lines ...

Buy Now — $19 Back to Products