Features

This chapter covers the core features and capabilities of Workflow Engine.

Features

• DAG execution — Tasks execute in dependency order with automatic parallelism
• Conditional branching — Skip tasks based on Python expressions
• Parallel execution — Tasks without dependencies run concurrently via thread pool
• State persistence — Save and resume workflows after failures
• Retry policies — Per-task retry limits with configurable backoff
• DAG validation — Detects cycles and missing dependencies before execution
• Completion hooks — Run commands on workflow success or failure
• Dry-run mode — Preview execution plan without running anything
• Context passing — Upstream task output available to downstream tasks via env vars

Requirements

• Python 3.10+
• No external dependencies (stdlib only)

Quick Start

Follow this guide to get Workflow Engine up and running in your environment.

Quick Start

# Run a workflow
python src/workflow_engine.py --config examples/workflow_config.json

# Preview execution plan
python src/workflow_engine.py --config examples/workflow_config.json --dry-run

# Resume a failed workflow
python src/workflow_engine.py --config examples/workflow_config.json --resume workflow_state.json

Configuration Reference

{
    "name": "my-pipeline",
    "max_workers": 4,
    "tasks": [
        {
            "name": "build",
            "command": "make build",
            "depends_on": ["lint", "test"],
            "condition": "",
            "retry_limit": 2,
            "timeout": 300
        }
    ],
    "on_complete": "echo 'Done!'",
    "on_failure": "curl https://hooks.example.com/alert"
}