PyPI
BrainLesion/panoptica
panoptica -- instance-wise evaluation of 3D semantic and instance segmentation maps
Apache-2.0Sep 14, 2026 · metrics 2.10.0
All tags
Catalogue tag
#evaluation
Every repository in the public record carrying this tag — from its GitHub topics or the keywords its package registries publish. Health is measured under the same versioned methodology as the rest of the record.
93 records
npm · crates.io · PyPI
PSU3D0/formualizer
Embeddable spreadsheet engine - parse, evaluate & mutate Excel workbooks from Rust, Python, or the browser. Arrow-powered, 400+ functions.
Apache-2.0Sep 5, 2026 · metrics 2.10.0
PyPI
Toloka/tolokaforge
Universal LLM benchmarking harness for tool use, browser, mobile, coding, and long-horizon evals
Custom licenseSep 25, 2026 · metrics 2.10.0
PyPI
VectorInstitute/cyclops
A toolkit for evaluating and monitoring AI models in clinical settings
Custom licenseOct 19, 2026 · metrics 2.10.0
PyPI · crates.io
derekallman/hotcoco
Perception evaluation toolkit in Rust with Python bindings. A pycocotools drop-in for COCO, LVIS, and Open Images detection metrics, up to 36× faster, plus TIDE error analysis, confusion matrices, calibration, model comparison, and a dataset browser.
MITOct 7, 2026 · metrics 2.10.0
PyPI
SuperagenticAI/superqode
The harness interoperability layer for coding agent. Agent to Agent connect over ACP, A2A and UHP
Apache-2.0Sep 11, 2026 · metrics 2.10.0
PyPI · npm
awslabs/llm-evaluation-system
Agentic AI-guided evaluation system for comparing LLMs with multi-judge jury scoring
Apache-2.0Sep 19, 2026 · metrics 2.10.0
npm
mattapperson/noetic
No repository description published.
Apache-2.0Sep 18, 2026 · metrics 2.10.0
npm
Kromatic-Innovation/panelist
Synthetic user panels for any artifact, run across multiple model providers to correct for self-preference bias — models the exact point a persona quits, dismisses, or refuses to click, not a warmth score.
Apache-2.0Oct 20, 2026 · metrics 2.10.0
npm · PyPI · Go
memoturn/memoturn
Open-source AI engineering platform — LLM observability, evals, metrics, prompt management, playground, and datasets. OpenTelemetry-native, self-hostable. Because "it worked when I tried it" is not an eval.
Apache-2.0Oct 3, 2026 · metrics 2.10.0
npm
Precisa-Saude/medbench-brasil
Leaderboard reproduzível de LLMs em provas médicas brasileiras (Revalida, INEP) — zero-shot, 3 execuções, análise de contaminação
Custom licenseOct 18, 2026 · metrics 2.10.0
PyPI
digline/digline
Regression testing for LLM applications — with the baseline in your repository, not on someone's server.
Apache-2.0Sep 18, 2026 · metrics 2.10.0
NuGet
elbruno/ElBruno.MarkItDotNet
.NET library that converts file formats to Markdown for AI pipelines
MITOct 19, 2026 · metrics 2.10.0
npm · PyPI
hue-run/hue-sdk
Hue TypeScript and Python SDKs for OpenTelemetry tracing and local evaluations
MITSep 23, 2026 · metrics 2.10.0
PyPI
OpenAdaptAI/openadapt-evals
Windows Agent Arena and GUI-agent evaluation for OpenAdapt. Live support surface. Install the product with pip install openadapt; this repo is the eval harness, including WAA.
MITSep 14, 2026 · metrics 2.10.0
npm
crewhaus/factory
Open-source compiler for AI agents. Write one crewhaus.yaml; compile it to a CLI, a Slack bot, and an eval harness from the same spec. Apache-2.0.
Apache-2.0Sep 13, 2026 · metrics 2.10.0
PyPI
huggingface/evaluate
🤗 Evaluate: A library for easily evaluating machine learning models and datasets.
Apache-2.0Sep 10, 2026 · metrics 2.10.0
npm
o-stepper/graphorin
Project Graphorin is a TypeScript framework for personal AI assistants and long-living agents with rich memory, durable workflow, and observability out of the box.
MITSep 15, 2026 · metrics 2.10.0
npm
JudgmentLabs/judgeval-js
The open source post-building layer for agents.
No licenseSep 15, 2026 · metrics 2.10.0
npm
axl-sdk/axl
TypeScript SDK for orchestrating Agentic Systems — concurrency, structured output, cost control, and consensus as first-class primitives.
Apache-2.0Sep 14, 2026 · metrics 2.10.0
npm
baabakk/llm-ports
TypeScript LLM Abstraction Layer for Multi-Provider AI Systems. Multi-provider routing, fallback chains, USD cost gating, capability factories, tool-use security.
MITSep 15, 2026 · metrics 2.10.0
PyPI
ianarawjo/ChainForge
An open-source visual programming environment for battle-testing prompts to LLMs.
MITOct 1, 2026 · metrics 2.10.0
PyPI
mjpost/sacrebleu
Reference BLEU implementation that auto-downloads test sets and reports a version string to facilitate cross-lab comparisons
Apache-2.0Aug 27, 2026 · metrics 2.10.0
PyPI
AgentX-ai/AgentX-Python
AgentX python SDK. Build multi-agent AI workforce. Run evaluation. Trace your agent. Full Observability.
Apache-2.0Sep 16, 2026 · metrics 2.10.0
npm
agenticklabs/agentick
The component framework for AI.
MITSep 18, 2026 · metrics 2.10.0
PyPI
bullpeng72/Agent-Evaluator
LLM agent evaluation framework with 7 Harness Gates (A–G) and 58 metrics. Supports LangChain, CrewAI, AutoGen, DSPy, PydanticAI. Native LLM-as-Judge, OTEL tracing, FastAPI dashboard.
MITOct 18, 2026 · metrics 2.10.0
PyPI
danthedeckie/simpleeval
Simple Safe Sandboxed Extensible Expression Evaluator for Python
Custom licenseSep 12, 2026 · metrics 2.10.0
PyPI
HarperZ9/flywheel
Flywheel runs AI tasks with a model you choose and keeps a receipt anyone can recheck offline.
Custom licenseOct 15, 2026 · metrics 2.10.0
PyPI
datalab-to/omni_extract_bench
A consistent, fair, benchmark for structured extraction.
Apache-2.0Oct 16, 2026 · metrics 2.10.0
PyPI · crates.io · npm +1
nickderobertis/onejudge
A simulated interaction and evaluation loop over oneharness: drive a harness through a multi-turn conversation and score the transcript.
MITOct 20, 2026 · metrics 2.10.0