LLM Evaluation Framework

13+ built-in metrics (BLEU, ROUGE-1/2/L, Exact Match, Token F1, Cosine Sim), LLM-as-Judge, regression tester, and HTML report generator.

Select metrics, choose an LLM-as-judge rubric, and run a simulated evaluation.

Model Under Test Metrics

Exact Match Token F1 BLEU ROUGE-L LLM-as-Judge (helpfulness) JSON Validity

LLM-as-Judge Rubric

Evaluation Results

Configure and run evaluation.

Full version includes 13+ metrics, eval harness, dataset runner, regression tester, LLM-as-Judge with 4 rubrics, and HTML report generator.

🤖 AI/LLM Toolkit — LLM Evaluation Framework Demo