Contents

Chapter 1

Python Packaging Guide

A comprehensive reference for packaging, versioning, and publishing Python libraries.

1. pyproject.toml Anatomy

pyproject.toml is the single source of truth for modern Python projects (PEP 621).

Build System

toml
[build-system]
requires = ["setuptools>=68.0"]
build-backend = "setuptools.build_meta"

The [build-system] table tells pip and build which backend to use. Alternatives include flit-core, hatchling, and pdm-backend.

Project Metadata

toml
[project]
name = "my-package"          # PyPI package name (use hyphens)
version = "1.0.0"            # Follow semver
requires-python = ">=3.9"    # Minimum Python version
dependencies = ["httpx"]     # Runtime dependencies

Key fields:

  • name: Must be unique on PyPI. Use lowercase with hyphens.
  • version: Semantic versioning (MAJOR.MINOR.PATCH).
  • classifiers: Trove classifiers for PyPI search/filtering.
  • optional-dependencies: Groups like [dev], [docs].

Tool Configuration

Tools like ruff, mypy, and pytest read config from [tool.*] tables, keeping everything in one file.

2. Versioning Strategies

Semantic Versioning (semver)

  • MAJOR: Breaking API changes
  • MINOR: New features, backwards-compatible
  • PATCH: Bug fixes, backwards-compatible

Single-Source Version

Keep the version in one place. Options:

1. __init__.py: __version__ = "1.0.0" + version = attr: package.__version__

2. pyproject.toml: Canonical, read at build time

3. setuptools-scm: Derive version from git tags automatically

Calendar Versioning (calver)

Some projects use YYYY.MM.DD format (e.g., pip, Ubuntu). Useful for projects without a stable API contract.

3. Wheels vs Source Distributions

Source Distribution (sdist)

  • A tarball of the source tree (.tar.gz)
  • Requires a build step on the user's machine
  • Controlled by MANIFEST.in

Wheel (bdist_wheel)

  • A pre-built .whl file (a zip with metadata)
  • Installs instantly — no build step required
  • Always preferred when available

Rule of thumb: Always publish both sdist and wheel. Use python -m build which creates both by default.

4. The src/ Layout

my-project/
├── src/
│   └── my_package/
│       ├── __init__.py
│       └── core.py
├── tests/
│   └── test_core.py
└── pyproject.toml

Why src/ layout?

  • Prevents accidental imports from the project root
  • Forces you to install the package before testing (catches packaging bugs)
  • Standard for modern Python projects

5. PEP 561: Type Stub Packages

Add an empty py.typed marker file to your package root:

src/my_package/py.typed

Then include it in your package data:

toml
[tool.setuptools.package-data]
my_package = ["py.typed"]

This tells type checkers (mypy, pyright) that your package ships inline type annotations.

6. Namespace Packages

For large organizations splitting a namespace across multiple packages:

# Package A installs: myorg/auth/...
# Package B installs: myorg/billing/...

Use implicit namespace packages (PEP 420) — omit __init__.py in the shared namespace directory. Each sub-package is an independent distribution.

7. Publishing to PyPI

Manual Publishing

bash
python -m build
python -m twine check dist/*
python -m twine upload dist/*

TestPyPI First

Always test on TestPyPI before publishing to production:

bash
python -m twine upload --repository testpypi dist/*
pip install --index-url https://test.pypi.org/simple/ my-package

CI Publishing (GitHub Actions)

yaml
- name: Publish to PyPI
  uses: pypa/gh-action-pypi-publish@release/v1
  with:
    password: ${{ secrets.PYPI_API_TOKEN }}

Use Trusted Publishers (OIDC) for keyless authentication — no API tokens needed.

8. MANIFEST.in Patterns

include LICENSE README.md
recursive-include src *.py py.typed
recursive-include tests *.py
global-exclude *.pyc __pycache__

Common mistakes:

  • Forgetting to include py.typed in the sdist
  • Including .git/, node_modules/, or CI configs
  • Not testing the sdist: pip install dist/*.tar.gz

9. Pre-commit and CI Integration

Pre-commit

yaml
repos:
  - repo: https://github.com/astral-sh/ruff-pre-commit
    hooks:
      - id: ruff
      - id: ruff-format

GitHub Actions Matrix

yaml
strategy:
  matrix:
    python-version: ["3.9", "3.10", "3.11", "3.12", "3.13"]
    os: [ubuntu-latest, macos-latest, windows-latest]

10. Checklist Before Publishing

  • [ ] Version bumped in pyproject.toml (or __init__.py)
  • [ ] CHANGELOG updated
  • [ ] All tests pass (make check)
  • [ ] py.typed marker included
  • [ ] License file included
  • [ ] README renders correctly on PyPI (check with twine check)
  • [ ] Tested install from sdist: pip install dist/*.tar.gz
  • [ ] Tested install from wheel: pip install dist/*.whl
  • [ ] Published to TestPyPI first
Python Packaging Guide v1.0.0 — Free Preview