PyPI · npm
langchain-ai/langsmith-sdk
LangSmith Client SDK Implementations
MITAug 27, 2026 · metrics 2.10.0
All tags
Catalogue tag
#evaluation
Every repository in the public record carrying this tag — from its GitHub topics or the keywords its package registries publish. Health is measured under the same versioned methodology as the rest of the record.
58 records
PyPI · npm
Agenta-AI/agenta
Agenta is a workspace where you and your team build agents and automations.
Custom licenseAug 28, 2026 · metrics 2.10.0
npm · Go
langwatch/langwatch
The platform for LLM evaluations and AI agent testing
Apache-2.0Aug 13, 2026 · metrics 2.10.0
npm · PyPI
comet-ml/opik
Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.
Apache-2.0Aug 5, 2026 · metrics 2.10.0
npm
promptfoo/promptfoo
Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.
MITAug 5, 2026 · metrics 2.10.0
Go · PyPI · npm
Tencent/WeKnora
Open-source LLM knowledge platform: turn raw documents into a queryable RAG, an autonomous reasoning agent, and a self-maintaining Wiki.
Custom licenseAug 5, 2026 · metrics 2.10.0
PyPI
embeddings-benchmark/mteb
MTEB: State-of-the-art evaluation of embeddings across languages and modalities
Apache-2.0Jul 19, 2026 · metrics 2.10.0
Go
trpc-group/trpc-agent-go
A Go framework for building production agent systems with graph workflows, tools, memory, A2A, AG-UI, MCP, evaluation, and observability.
Apache-2.0Jul 19, 2026 · metrics 2.10.0
PyPI
langchain-ai/deepagents
The batteries-included agent harness.
MITAug 5, 2026 · metrics 2.10.0
npm
langfuse/langfuse
🪢 Open source AI engineering platform: LLM evals, observability, metrics, prompt management, playground, datasets. Integrates with OpenTelemetry, LangChain, OpenAI SDK, LiteLLM, and more. 🍊YC W23
Custom licenseAug 5, 2026 · metrics 2.10.0
PyPI
modelscope/evalscope
A streamlined and customizable framework for efficient large model (LLM, VLM, AIGC) evaluation and performance benchmarking.
Apache-2.0Aug 1, 2026 · metrics 2.10.0
PyPI · npm
NVIDIA-NeMo/Gym
Evaluate and improve models and agents using environments
Apache-2.0Jul 18, 2026 · metrics 2.10.0
npm · PyPI
joshuaswarren/remnic
Open-source memory and context for user-aware agents: scoped memory, provenance, retrieval quality, correction, boundaries, evals, and MCP/HTTP access.
MITAug 22, 2026 · metrics 2.10.0
npm
MCPJam/inspector
Testing and evaluation platform to chat, inspect, and debug MCP servers, MCP apps, and ChatGPT apps.
Custom licenseJul 18, 2026 · metrics 2.10.0
npm · PyPI
Marker-Inc-Korea/AutoRAG
AutoRAG: Now your agent can find anything in your computer. It gets smarter if you are using it frequently.
Custom licenseAug 2, 2026 · metrics 2.10.0
PyPI · npm
UiPath/coder_eval
Test that your Claude Code skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, activation checks, A/B experiments, CI gates.
Apache-2.0Aug 19, 2026 · metrics 2.10.0
PyPI · Go · npm
gooddata/gooddata-python-sdk
GoodData Cloud Python SDK
Custom licenseAug 22, 2026 · metrics 2.10.0
PyPI
vibrantlabsai/ragas
Supercharge Your LLM Application Evaluations 🚀
Apache-2.0Aug 8, 2026 · metrics 2.10.0
PyPI · npm
hidai25/eval-view
Regression testing for AI agents. Snapshot behavior,diff tool calls,catch regressions in CI. Works with LangGraph, CrewAI, OpenAI, Anthropic.
Apache-2.0Jul 26, 2026 · metrics 2.10.0
PyPI
BrainLesion/panoptica
panoptica -- instance-wise evaluation of 3D semantic and instance segmentation maps
Apache-2.0Jul 31, 2026 · metrics 2.10.0
PyPI
MichaelGrupp/evo
Python package for the evaluation of odometry and SLAM
GPL-3.0Aug 28, 2026 · metrics 2.10.0
NuGet
asc-community/AngouriMath
Open-source cross-platform symbolic algebra library for C# and F#. Can be used for both production and research purposes.
MITAug 23, 2026 · metrics 2.10.0
PyPI · npm
robocurve/inspect-robots
Open source evals for physical AI. Run any LLM/VLA on any arm/humanoid against any real/sim benchmark.
MITSep 6, 2026 · metrics 2.10.0
npm · crates.io · PyPI
PSU3D0/formualizer
Embeddable spreadsheet engine - parse, evaluate & mutate Excel workbooks from Rust, Python, or the browser. Arrow-powered, 400+ functions.
Apache-2.0Sep 5, 2026 · metrics 2.10.0
npm · Go · crates.io +2
ops-ai/Toggly.FeatureManagement
Enables teams to release software faster and safer, and with better results.
MITSep 3, 2026 · metrics 2.10.0
npm · PyPI
aikdna/kdna
KDNA protocol and Core runtime for versioned, verifiable, encrypted, authorized judgment assets.
Apache-2.0Jul 22, 2026 · metrics 2.10.0
npm
o-stepper/graphorin
Project Graphorin is a TypeScript framework for personal AI assistants and long-living agents with rich memory, durable workflow, and observability out of the box.
MITAug 1, 2026 · metrics 2.10.0
PyPI
huggingface/evaluate
🤗 Evaluate: A library for easily evaluating machine learning models and datasets.
Apache-2.0Jul 18, 2026 · metrics 2.10.0
PyPI
mjpost/sacrebleu
Reference BLEU implementation that auto-downloads test sets and reports a version string to facilitate cross-lab comparisons
Apache-2.0Aug 27, 2026 · metrics 2.10.0
npm
JudgmentLabs/judgeval-js
The open source post-building layer for agents.
No licenseAug 1, 2026 · metrics 2.10.0