Document AI Toolkit

$49

PDF/document parsing pipelines, OCR integration, table extraction, summarization chains, and structured data extraction.

📁 26 files

MarkdownYAMLPythonLLM

📄 Product Preview

Try the interactive reader and demo tools below, or get the full product with all content unlocked.

📖 Interactive Reader (Free Preview) ⚙ Try Demo Tools 📦 Download Free Sample

📁 File Structure 26 files

document-ai-toolkit/ ├── LICENSE ├── README.md ├── configs/ │ └── extraction_config.yaml ├── free-sample.zip ├── guide/ │ └── 01-document-processing.md ├── guides/ │ └── document-processing.md ├── index.html ├── requirements.txt ├── src/ │ ├── __init__.py │ ├── __pycache__/ │ │ ├── __init__.cpython-312.pyc │ │ ├── document_parser.cpython-312.pyc │ │ ├── ocr_interface.cpython-312.pyc │ │ ├── schema_extractor.cpython-312.pyc │ │ ├── summarizer.cpython-312.pyc │ │ ├── table_extractor.cpython-312.pyc │ │ └── text_chunker.cpython-312.pyc │ ├── document_parser.py │ ├── ocr_interface.py │ ├── schema_extractor.py │ ├── summarizer.py │ ├── table_extractor.py │ └── text_chunker.py └── tests/ ├── __pycache__/ │ ├── test_chunker.cpython-312.pyc │ └── test_parser.cpython-312.pyc ├── test_chunker.py └── test_parser.py

📖 Documentation Preview README excerpt

Document AI Toolkit

Multi-format document parsing, intelligent chunking, and structured data extraction for LLM pipelines.

Parse any document. Chunk it for RAG. Extract structured data. Summarize it. All in one toolkit.

What You Get

Multi-format parser with unified Document model: PDF, DOCX, HTML, Markdown, and plain text with automatic format detection
5 chunking strategies: Fixed, Sentence-aware, Paragraph-aware, Semantic (heading-based), and Recursive — each with configurable overlap
Table extractor that pulls tabular data from HTML and Markdown into structured formats (dicts, CSV, JSON)
Schema-guided extractor that pulls specific fields from unstructured text using regex patterns or LLM function calling
Summarization chains: Extractive (stdlib, no LLM needed), Direct, Map-Reduce, and Refine strategies for any document length
OCR interface with Tesseract backend, cloud OCR adapter, and mock backend for testing
Stdlib reference pipeline for .txt/.md/.html that runs without any external dependencies

Quick Start


from src.document_parser import parse_file
from src.text_chunker import TextChunker, ChunkStrategy

# Parse any document
doc = parse_file("report.pdf")
print(f"Title: {doc.title}, Words: {doc.word_count}")

# Chunk for RAG
chunker = TextChunker(strategy=ChunkStrategy.SENTENCE, chunk_size=500)
chunks = chunker.chunk_document(doc)
for chunk in chunks:
    print(f"Chunk {chunk.chunk_index}: {chunk.char_count} chars")

# Extract structured data
from src.schema_extractor import ExtractionSchema, SchemaExtractor
schema = ExtractionSchema("invoice")
schema.add_field("total", "currency", "Total amount", required=True)
schema.add_field("date", "date", "Invoice date")
extractor = SchemaExtractor(schema)
result = extractor.extract(doc.full_text)
print(result.to_json())

# Summarize
from src.summarizer import extractive_summarize
summary = extractive_summarize(doc.full_text, num_sentences=5)
print(summary.summary)

File Tree


document-ai-toolkit/
├── README.md
├── LICENSE
├── requirements.txt
├── src/
│   ├── __init__.py
│   ├── document_parser.py        # Multi-format parsing (PDF/DOCX/HTML/MD/TXT)
│   ├── text_chunker.py           # 5 chunking strategies with overlap

*... continues with setup instructions, usage examples, and more.*

📄 Code Sample .py preview

src/document_parser.py """ Document Parser — Document AI Toolkit ======================================= Multi-format document parser that extracts structured text from PDF, DOCX, HTML, Markdown, and plain text files. Provides a unified Document model regardless of the input format. For PDF and DOCX, this module wraps pdfplumber and python-docx respectively. For HTML and Markdown, it uses stdlib html.parser and regex-based parsing that works without any external dependencies. The parser preserves document structure: headings, paragraphs, lists, tables, and metadata — because downstream tasks (chunking, summarization, extraction) need to know "this is a heading" vs "this is body text." """ from __future__ import annotations import html import json import logging import mimetypes import os import re from dataclasses import dataclass, field from enum import Enum from html.parser import HTMLParser from pathlib import Path from typing import Any, Dict, List, Optional, Tuple logger = logging.getLogger(__name__) class BlockType(Enum): """Types of content blocks within a parsed document.""" HEADING = "heading" PARAGRAPH = "paragraph" LIST_ITEM = "list_item" TABLE = "table" CODE_BLOCK = "code_block" BLOCKQUOTE = "blockquote" IMAGE_REF = "image_ref" METADATA = "metadata" @dataclass class ContentBlock: """A single structural block extracted from a document.""" block_type: BlockType text: str # ... 452 more lines ...

Buy Now — $49 Back to Products