AI API Gateway

Unified LLM client with RetryRouter, circuit breaker, token-bucket rate limiter, SHA-256 response cache, and structured JSONL logging.

Configure providers, fallback strategy, rate limits, and test with a prompt.

Providers

OpenAI (gpt-4o) Anthropic (claude-3-5-sonnet) Local (Ollama/vLLM)

Fallback Rate Limit (req/s) Cache Temperature 0.3

Test Prompt

Gateway Response

Configure and click to test.

Full version includes OpenAI/Anthropic/Local adapters, retry router, circuit breaker, rate limiter, cache, request logger, and analytics.

🤖 AI/LLM Toolkit — AI API Gateway Demo