**Deploy ML models with FastAPI, A/B testing, canary routing, and multi-backend support.**
Browse the actual product documentation and code examples included in this toolkit.
Key features of Model Serving Toolkit
• **FastAPI serving application** with single and batch prediction endpoints • **Model loader abstraction** supporting pickle, JSON, and ONNX formats with hot-swap capability • **A/B testing router** with weighted traffic splits, sticky sessions, and header-based overrides • **Canary deployment** support with dynamic weight updates for gradual rollouts • **Request validation** with schema-based type checking, range validation, and allowed-value constraints • **Health and readiness probes** following Kubernetes patterns with serving metrics
**FastAPI serving application** with single and batch prediction endpoints
**Model loader abstraction** supporting pickle, JSON, and ONNX formats with hot-swap capability
**A/B testing router** with weighted traffic splits, sticky sessions, and header-based overrides
**Canary deployment** support with dynamic weight updates for gradual rollouts
**Request validation** with schema-based type checking, range validation, and allowed-value constraints
**Health and readiness probes** following Kubernetes patterns with serving metrics
Configure Model Serving Toolkit parameters to see how the product works.
pip install -r requirements.txt uvicorn src.serving_app:create_app --factory --reload --port 8000