Catalogue tag

#llm-evaluation-framework

Every repository in the public record carrying this tag — from its GitHub topics or the keywords its package registries publish. Health is measured under the same versioned methodology as the rest of the record.

5 records

Tagged “llm-evaluation-framework”Ranked by health index

npm

97Exceptionalhealth index

promptfoo/promptfoo

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

TypeScript★ 25.2K↓ 2.6M/moSep 16, 2026

MITSep 16, 2026 · metrics 2.10.0

PyPI · npm

94Exceptionalhealth index

confident-ai/deepeval

The LLM Evaluation Framework

Python · TypeScript★ 18.3K↓ 6.3M/moSep 16, 2026

Apache-2.0Sep 16, 2026 · metrics 2.10.0

RubyGems

69Goodhealth index

homemade-software-inc/completion-kit

Your prompts need tests too. Run prompts against real datasets, score outputs with LLM judges, version everything, and compare runs to see what got better.

Ruby · HTML★ 3Sep 10, 2026

Custom licenseSep 10, 2026 · metrics 2.10.0

PyPI

59Moderatehealth index

dalmia/calibrate-python-sdk

Python SDK for Calibrate's API

Python★ 0↓ 3,013/moSep 20, 2026

MITSep 20, 2026 · metrics 2.10.0

59Moderatehealth index

petmal/MindTrial

MindTrial: Evaluate and compare AI language models (LLMs) on text-based tasks with optional file/image attachments and tool use. Supports multiple providers (OpenAI, Google, Anthropic, DeepSeek, Mistral AI, xAI, Alibaba, Moonshot AI, OpenRouter), custom tasks in YAML, and HTML/CSV/JSON reports.

Go★ 21Oct 21, 2026

MPL-2.0Oct 21, 2026 · metrics 2.10.0