← Back to all products

Data Catalog Builder

$29

Automated metadata discovery, data dictionary generation, and searchable catalog for Unity Catalog.

📁 18 files🏷 v1.0.0

Production-ready

✓ Instant download✓ Lifetime updates✓ MIT licensed✓ MIT license✓ Secure checkout (Stripe)

📋 What's Inside 18 files

✓ README.md—
✓ manifest.json—
✓ LICENSE—
✓ src/catalog_scanner.py—
✓ src/metadata_enricher.py—
✓ src/lineage_mapper.py—
✓ src/search_index.py—
✓ src/quality_scorer.py—
✓ src/catalog_reporter.py—
✓ configs/catalog_config.yaml—
✓ configs/classification_rules.yaml—
✓ configs/templates/table_doc.md.j2—
✓ notebooks/scan_catalog.py—
✓ notebooks/catalog_dashboard.py—
✓ tests/conftest.py—

📁 File Structure 18 files

data-catalog-builder/
├── README.md
├── manifest.json
├── LICENSE
├── src/
│ ├── catalog_scanner.py
│ ├── metadata_enricher.py
│ ├── lineage_mapper.py
│ ├── search_index.py
│ ├── quality_scorer.py
│ ├── catalog_reporter.py
├── configs/
│ ├── catalog_config.yaml
│ ├── classification_rules.yaml
│ ├── templates/
│ │ ├── table_doc.md.j2
├── notebooks/
│ ├── scan_catalog.py
│ ├── catalog_dashboard.py
├── tests/
│ ├── conftest.py
│ ├── test_catalog_scanner.py
│ ├── test_quality_scorer.py
├── guides/
│ ├── data-catalog-guide.md

📖 Documentation Preview README excerpt

Data Catalog Builder

Automated metadata discovery, data dictionary generation, quality scoring, and searchable catalog for Databricks Unity Catalog.

> By [Datanest Digital](https://datanest.dev) | Version 1.0.0 | $39

---

What You Get

**Catalog Scanner** — Discover tables, columns, and metadata across Unity Catalog schemas

**Metadata Enricher** — Add business descriptions, owners, tags, and classification labels

**Lineage Mapper** — Trace column-level lineage from bronze to gold layers

**Search Index** — Full-text search across table/column names, descriptions, and tags

**Quality Scorer** — Score tables on completeness, freshness, documentation, and conformance

**Catalog Reporter** — Generate Markdown data dictionaries from Jinja2 templates

File Tree


data-catalog-builder/
├── README.md
├── manifest.json
├── LICENSE
├── src/
│   ├── catalog_scanner.py           # Unity Catalog metadata discovery
│   ├── metadata_enricher.py         # Business metadata enrichment
│   ├── lineage_mapper.py            # Column-level lineage tracing
│   ├── search_index.py              # Full-text catalog search
│   ├── quality_scorer.py            # Data quality scoring engine
│   └── catalog_reporter.py          # Markdown report generator
├── configs/
│   ├── catalog_config.yaml          # Scanner and enrichment settings
│   ├── classification_rules.yaml    # PII / sensitivity classification
│   └── templates/
│       └── table_doc.md.j2          # Jinja2 table documentation template
├── notebooks/
│   ├── scan_catalog.py              # Run catalog discovery scan
│   └── catalog_dashboard.py         # Catalog health dashboard
├── tests/
│   ├── conftest.py                  # Shared test fixtures
│   ├── test_catalog_scanner.py      # Scanner tests
│   └── test_quality_scorer.py       # Quality scoring tests
└── guides/
    └── data-catalog-guide.md        # Setup and usage guide


Getting Started

1. Configure the Scanner

Edit configs/catalog_config.yaml with your Unity Catalog settings:


scanner:
  catalogs: ["main"]
  schemas: ["bronze", "silver", "gold"]
  exclude_patterns: ["_tmp_", "_staging_"]


... preview truncated, see full README in product download.

`📄 Code Sample .py preview`

src/catalog_scanner.py"""
Catalog Scanner — Discover tables, columns, and metadata across Unity Catalog.

Scans specified catalogs and schemas, collecting table metadata, column
information, table properties, and basic statistics into structured
dataclasses for downstream enrichment and reporting.

Author: Datanest Digital
"""

from __future__ import annotations

import logging
import re
from dataclasses import dataclass, field
from datetime import datetime
from typing import Any, Dict, List, Optional

import yaml
from pyspark.sql import DataFrame, SparkSession
from pyspark.sql import functions as F

logger = logging.getLogger(__name__)


# ---------------------------------------------------------------------------
# Data models
# ---------------------------------------------------------------------------

@dataclass
class ColumnInfo:
    """Metadata for a single table column."""

    name: str
    data_type: str
    nullable: bool = True
    comment: Optional[str] = None
    is_partition: bool = False
    tags: List[str] = field(default_factory=list)
    classification: Optional[str] = None

`📅 Changelog`

v1.0.0 — Initial release.

Purchases include lifetime updates. Check the product page for the latest version.

`📄 Product Preview`

Try the interactive reader and demo tools below, or get the full product with all content unlocked.

📖 Interactive Reader (Free Preview)⚙ Try Demo Tools📦 Download Free Sample

`❓ Frequently Asked Questions`

What license is this under?

MIT License. You can use it in commercial, client, and internal projects. See our license terms page for details.

How do I download after purchase?

Immediately after purchase, you'll be redirected to a download page. A download link is also sent to your email.

Do I get updates?

Yes. All products include lifetime updates at no extra cost. Re-download anytime for the latest version.

What if it doesn't work for me?

DM @DatanestDigital on X and we'll make it right.

Can I get a refund?

DM @DatanestDigital on X with your order number and we'll help.

Is there support?

DM @DatanestDigital on X. We respond within 1-2 business days.



      
        Buy Now — $29
        
        Back to Products