#evals

Playwright for coding agents. Test that your skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, A/B experiments, CI gates.

RL environments + evals for AI agents. Define once, train anything.

Build, Evaluate, and Optimize AI Systems. Includes evals, RAG, agents, fine-tuning, synthetic data generation, dataset management, MCP, and more.

Ridiculous speed. Readable code.

Observability and enforcement for AI agent harnesses. Capture every run and runtime reliability with policy enforcement. 40 built-in policies, a local dashboard, no account required with a generous free cloud plan

Testing and evaluation platform to chat, inspect, and debug MCP servers, MCP apps, and ChatGPT apps.

Official CLI and Python SDK for Prime Intellect - access GPU compute, remote sandboxes, RL environments, and distributed training infrastructure for AI development at scale.

AI evaluators CLI for your AI apps and Agents - Dynatrace AI Observability

Self-hostable, MIT Licensed, Enterprise AI Software Factory

Research infra for creating RL environments, post-training, and evals.

Give an agent a job, not your keys. Open-source control plane for AI agent work: each agent gets its own identity, a bounded job, and a signed record of what it did.

Composable Pi coding agent with MCP, LSP, agent chains, prompt presets, and local eval telemetry

Phoenix-native LLM evaluation, prompt testing, model comparison, red teaming, and observability for Elixir

Zero-config LLM cost & quality monitoring for Ruby apps - automatically log AI API calls and collect user feedback for analysis using Coolhand.

Python SDK for AI agent monitoring, LLM cost tracking, benchmarking, and more. Integrates with most LLMs and agent frameworks including CrewAI, Agno, OpenAI Agents SDK, Langchain, Autogen, AG2, and CamelAI