PyPI · npm
Arize-ai/phoenix
AI Observability & Evaluation
Власна ліцензія28 серп. 2026 р. · метрики 2.10.0
Усі теги
Тег каталогу
#llm-evaluation
Усі репозиторії публічного реєстру з цим тегом — із тем GitHub або ключових слів, які публікують їхні реєстри пакетів. Здоров'я вимірюється за тією ж версіонованою методологією, що й решта реєстру.
25 записів
PyPI
Giskard-AI/giskard-oss
🐢 Open-Source Evaluation & Testing library for LLM Agents
Apache-2.028 серп. 2026 р. · метрики 2.10.0
npm · PyPI
comet-ml/opik
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
Apache-2.05 серп. 2026 р. · метрики 2.10.0
npm
promptfoo/promptfoo
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
MIT5 серп. 2026 р. · метрики 2.10.0
PyPI
Q00/ouroboros
Agent OS: the agent gets smarter on its own. We just hold the line: the grading command and expected result never make it into the success contract we hand it. Interview-gated, staged evaluation, budgeted evolution loop. MCP server, 13 runtimes: Claude Code, Codex CLI, Gemini CLI, OpenCode, Copilot, Kiro and more.
MIT13 серп. 2026 р. · метрики 2.10.0
PyPI · npm
confident-ai/deepeval
The LLM Evaluation Framework
Apache-2.05 серп. 2026 р. · метрики 2.10.0
npm
langfuse/langfuse
🪢 Open source AI engineering platform: LLM evals, observability, metrics, prompt management, playground, datasets. Integrates with OpenTelemetry, LangChain, OpenAI SDK, LiteLLM, and more. 🍊YC W23
Власна ліцензія5 серп. 2026 р. · метрики 2.10.0
PyPI
JudgmentLabs/judgeval
The Continuous-Improvement Stack for Agents. Our environment data and evals power agent improvement and monitoring.
Apache-2.013 серп. 2026 р. · метрики 2.10.0
Go
praetorian-inc/julius
Simple LLM service identification - translate IP:Port to Ollama, vLLM, LiteLLM, or 60+ other AI services in seconds
Apache-2.08 серп. 2026 р. · метрики 2.10.0
npm · PyPI
Marker-Inc-Korea/AutoRAG
AutoRAG: Now your agent can find anything in your computer. It gets smarter if you are using it frequently.
Власна ліцензія2 серп. 2026 р. · метрики 2.10.0
PyPI
truera/trulens
Evaluation and Tracking for LLM Experiments and AI Agents
MIT6 серп. 2026 р. · метрики 2.10.0
PyPI · npm
UiPath/coder_eval
Test that your Claude Code skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, activation checks, A/B experiments, CI gates.
Apache-2.019 серп. 2026 р. · метрики 2.10.0
npm
microsoft/prompty
Prompty makes it easy to create, manage, debug, and evaluate LLM prompts for your AI applications. Prompty is an asset class and format for LLM prompts designed to enhance observability, understandability, and portability for developers.
MIT13 серп. 2026 р. · метрики 2.10.0
PyPI
alizahidraja/isnad
Grade every agent, scraper and model in a claim's chain — provenance, trust scoring and audit evidence for LLM pipelines
Apache-2.029 серп. 2026 р. · метрики 2.10.0
Go · Maven · npm
hugalafutro/model-hotel
Multi-Provider AI Gateway - No personal logs by design. Model autodiscovery, Failover groups, High availabilty, Android companion app, and more. "Because we have LiteLLM at home"
MIT18 лип. 2026 р. · метрики 2.10.0
PyPI
b7n0de/proofbundle
Offline cryptographic receipts for AI evaluation results — Ed25519 + RFC 6962 Merkle + optional SD-JWT. Integrity, not truth
MIT23 лип. 2026 р. · метрики 2.10.0
Hex · npm
ccarvalho-eng/aludel
LLM Evaluation for Phoenix Apps
Apache-2.017 лип. 2026 р. · метрики 2.10.0
PyPI
phierceweb/pf-core
Python foundation for LLM apps whose prompts and spend you can actually see — versioned prompts, every call recorded and replayable, budgets, evals, jobs.
MIT5 вер. 2026 р. · метрики 2.10.0
RubyGems
homemade-software-inc/completion-kit
Your prompts need tests too. Run prompts against real datasets, score outputs with LLM judges, version everything, and compare runs to see what got better.
Власна ліцензія17 лип. 2026 р. · метрики 2.10.0
PyPI · npm
fastaifoundry/fastaiagent-sdk
Опис репозиторію не опубліковано.
Apache-2.017 лип. 2026 р. · метрики 2.10.0
Go
tamnd/taocp-solver
A Go library and CLI for complete TAOCP solutions, fast and audited solving modes, reproducible model evaluation, and detailed token and list-cost accounting.
MIT28 лип. 2026 р. · метрики 2.10.0
npm
mykim-aus/hey-llm-you-okay
Hey LLM, you okay? — pyramid-ordered LLM testing CLI for CI/CD. One YAML for every layer, LLM-as-a-judge gates, and A/B triage that tells prompt regressions from model drift.
MIT30 лип. 2026 р. · метрики 2.10.0
PyPI
JarJarBeatyourattitude/evalt
Budget-bounded LLM routing that finds the cheapest model and prompt meeting your accuracy target.
MIT22 лип. 2026 р. · метрики 2.10.0
PyPI
waybarrios/crystal
CRYSTAL: Beyond Final Answers: Benchmark for Transparent Multimodal Reasoning Evaluation | arXiv 2603.13099
Без ліцензії17 лип. 2026 р. · метрики 2.10.0
PyPI · npm
mlflow/mlflow
The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.
Apache-2.05 серп. 2026 р. · метрики 2.10.0