LLM Application Framework

LLM Application Framework

Build production LLM applications with prompt management, RAG pipelines, guardrails, evaluation harnesses, and cost tracking — using a mock LLM client that runs without API keys.

Assemble LLM apps the way an ML engineer would: modular pipeline stages, versioned prompts, measurable output quality, and explicit cost control. Every component works standalone or composes into a full application pipeline.

What You Get

• Mock LLM client — deterministic LLM simulator for testing without API keys, with token counting, streaming simulation, and error injection for resilience testing
• Prompt registry — versioned prompt templates with {{variable}} substitution, validation, fingerprinting, bulk loading from YAML, and composition (system + few-shot + context + query)
• RAG pipeline — document chunking with configurable overlap, in-memory vector store with cosine similarity, token-budget-aware context assembly, and source citation
• Guardrails — input validation (prompt injection detection, content length, topic filtering) and output validation (PII detection/redaction, format compliance) composable into chains
• Evaluation harness — ROUGE-L, BLEU, word overlap, factual consistency scoring, answer relevance, format compliance, output length checks, and A/B comparison
• Cost tracker — per-model pricing, running cost accumulation, budget alerts with callbacks, cost breakdown by model and prompt, and monthly cost estimation
• App builder — pipeline orchestrator that assembles stages (guardrails → retrieval → prompt assembly → generation → output validation → cost tracking) into a complete application
• Pre-built prompts — YAML prompt templates for QA, summarization, classification, entity extraction, code generation, chain-of-thought reasoning, and conversational assistants
• Few-shot example sets — ready-to-use examples for sentiment classification, intent detection, entity extraction, summarization, and code review

File Tree

llm-application-framework/
├── README.md
├── LICENSE
├── requirements.txt
├── src/
│   └── llm_framework/
│       ├── __init__.py              # Package exports
│       ├── mock_client.py           # Deterministic LLM simulator
│       ├── prompt_registry.py       # Prompt templates and versioning
│       ├── rag_pipeline.py          # Document retrieval and context assembly
│       ├── guardrails.py            # Input/output validation and PII detection
│       ├── evaluation.py            # Output quality scoring (ROUGE-L, BLEU, etc.)
│       ├── cost_tracker.py          # Token usage and API cost tracking
│       └── app_builder.py           # Pipeline orchestrator
├── prompts/
│   ├── system_prompts.yaml          # Reusable prompt templates
│   └── few_shot_examples.yaml       # In-context learning examples
├── configs/
│   ├── app_config.yaml              # Application configuration
│   └── guardrails_config.yaml       # Guardrail rules and thresholds
├── tests/
│   ├── test_prompt_registry.py      # Template rendering and registry tests
│   ├── test_guardrails.py           # PII, injection, and chain tests
│   └── test_rag.py                  # Chunking, vector store, pipeline tests
└── guides/
    ├── app-assembly.md              # Step-by-step app building guide
    └── guardrails-guide.md          # Guardrails configuration and tuning

Quick Start

1. Install Dependencies

pip install -r requirements.txt

... continues with setup instructions, usage examples, and more.