PyPI
BrainLesion/panoptica
panoptica -- instance-wise evaluation of 3D semantic and instance segmentation maps
Apache-2.014. Sept. 2026 · Metriken 2.10.0
Alle Tags
Katalog-Tag
#evaluation
Alle Repositories im öffentlichen Register, die dieses Tag tragen — aus ihren GitHub-Topics oder den von ihren Paket-Registries veröffentlichten Schlagwörtern. Die Gesundheit wird nach derselben versionierten Methodik gemessen wie im übrigen Register.
93 Einträge
PyPI · npm · crates.io
PSU3D0/formualizer
The fastest embeddable spreadsheet engine - parse, evaluate & mutate Excel workbooks from Rust, Python, or the browser. Arrow-powered, 400+ functions.
Apache-2.021. Okt. 2026 · Metriken 2.10.0
PyPI
Toloka/tolokaforge
Universal LLM benchmarking harness for tool use, browser, mobile, coding, and long-horizon evals
Eigene Lizenz25. Sept. 2026 · Metriken 2.10.0
PyPI
VectorInstitute/cyclops
A toolkit for evaluating and monitoring AI models in clinical settings
Eigene Lizenz19. Okt. 2026 · Metriken 2.10.0
PyPI · crates.io
derekallman/hotcoco
Perception evaluation toolkit in Rust with Python bindings. A pycocotools drop-in for COCO, LVIS, and Open Images detection metrics, up to 36× faster, plus TIDE error analysis, confusion matrices, calibration, model comparison, and a dataset browser.
MIT7. Okt. 2026 · Metriken 2.10.0
PyPI
SuperagenticAI/superqode
The harness interoperability layer for coding agent. Agent to Agent connect over ACP, A2A and UHP
Apache-2.011. Sept. 2026 · Metriken 2.10.0
PyPI · npm
awslabs/llm-evaluation-system
Agentic AI-guided evaluation system for comparing LLMs with multi-judge jury scoring
Apache-2.019. Sept. 2026 · Metriken 2.10.0
npm
mattapperson/noetic
Keine Repository-Beschreibung veröffentlicht.
Apache-2.018. Sept. 2026 · Metriken 2.10.0
npm
Kromatic-Innovation/panelist
Synthetic user panels for any artifact, run across multiple model providers to correct for self-preference bias — models the exact point a persona quits, dismisses, or refuses to click, not a warmth score.
Apache-2.020. Okt. 2026 · Metriken 2.10.0
npm · PyPI · Go
memoturn/memoturn
Open-source AI engineering platform — LLM observability, evals, metrics, prompt management, playground, and datasets. OpenTelemetry-native, self-hostable. Because "it worked when I tried it" is not an eval.
Apache-2.03. Okt. 2026 · Metriken 2.10.0
npm
Precisa-Saude/medbench-brasil
Leaderboard reproduzível de LLMs em provas médicas brasileiras (Revalida, INEP) — zero-shot, 3 execuções, análise de contaminação
Eigene Lizenz18. Okt. 2026 · Metriken 2.10.0
PyPI
digline/digline
Regression testing for LLM applications — with the baseline in your repository, not on someone's server.
Apache-2.018. Sept. 2026 · Metriken 2.10.0
NuGet
elbruno/ElBruno.MarkItDotNet
.NET library that converts file formats to Markdown for AI pipelines
MIT19. Okt. 2026 · Metriken 2.10.0
npm · PyPI
hue-run/hue-sdk
Hue TypeScript and Python SDKs for OpenTelemetry tracing and local evaluations
MIT23. Sept. 2026 · Metriken 2.10.0
PyPI
OpenAdaptAI/openadapt-evals
Windows Agent Arena and GUI-agent evaluation for OpenAdapt. Live support surface. Install the product with pip install openadapt; this repo is the eval harness, including WAA.
MIT14. Sept. 2026 · Metriken 2.10.0
npm
crewhaus/factory
Open-source compiler for AI agents. Write one crewhaus.yaml; compile it to a CLI, a Slack bot, and an eval harness from the same spec. Apache-2.0.
Apache-2.013. Sept. 2026 · Metriken 2.10.0
PyPI
huggingface/evaluate
🤗 Evaluate: A library for easily evaluating machine learning models and datasets.
Apache-2.010. Sept. 2026 · Metriken 2.10.0
npm
o-stepper/graphorin
Project Graphorin is a TypeScript framework for personal AI assistants and long-living agents with rich memory, durable workflow, and observability out of the box.
MIT15. Sept. 2026 · Metriken 2.10.0
npm
JudgmentLabs/judgeval-js
The open source post-building layer for agents.
Keine Lizenz15. Sept. 2026 · Metriken 2.10.0
npm
axl-sdk/axl
TypeScript SDK for orchestrating Agentic Systems — concurrency, structured output, cost control, and consensus as first-class primitives.
Apache-2.014. Sept. 2026 · Metriken 2.10.0
npm
baabakk/llm-ports
TypeScript LLM Abstraction Layer for Multi-Provider AI Systems. Multi-provider routing, fallback chains, USD cost gating, capability factories, tool-use security.
MIT15. Sept. 2026 · Metriken 2.10.0
PyPI
ianarawjo/ChainForge
An open-source visual programming environment for battle-testing prompts to LLMs.
MIT1. Okt. 2026 · Metriken 2.10.0
PyPI
mjpost/sacrebleu
Reference BLEU implementation that auto-downloads test sets and reports a version string to facilitate cross-lab comparisons
Apache-2.027. Aug. 2026 · Metriken 2.10.0
PyPI
AgentX-ai/AgentX-Python
AgentX python SDK. Build multi-agent AI workforce. Run evaluation. Trace your agent. Full Observability.
Apache-2.016. Sept. 2026 · Metriken 2.10.0
npm
agenticklabs/agentick
The component framework for AI.
MIT18. Sept. 2026 · Metriken 2.10.0
PyPI
bullpeng72/Agent-Evaluator
LLM agent evaluation framework with 7 Harness Gates (A–G) and 58 metrics. Supports LangChain, CrewAI, AutoGen, DSPy, PydanticAI. Native LLM-as-Judge, OTEL tracing, FastAPI dashboard.
MIT18. Okt. 2026 · Metriken 2.10.0
PyPI
danthedeckie/simpleeval
Simple Safe Sandboxed Extensible Expression Evaluator for Python
Eigene Lizenz12. Sept. 2026 · Metriken 2.10.0
PyPI
HarperZ9/flywheel
Flywheel runs AI tasks with a model you choose and keeps a receipt anyone can recheck offline.
Eigene Lizenz15. Okt. 2026 · Metriken 2.10.0
PyPI
datalab-to/omni_extract_bench
A consistent, fair, benchmark for structured extraction.
Apache-2.016. Okt. 2026 · Metriken 2.10.0
PyPI · crates.io · npm +1
nickderobertis/onejudge
A simulated interaction and evaluation loop over oneharness: drive a harness through a multi-turn conversation and score the transcript.
MIT20. Okt. 2026 · Metriken 2.10.0