#llmops

AI Observability & Evaluation

The open source AI engineering platform for agents, LLMs, and ML models. MLflow enables teams of all sizes to debug, evaluate, monitor, and optimize production-quality AI applications while controlling costs and managing access to models and data.

Agenta is a workspace where you and your team build agents and automations.

Composio powers 1000+ toolkits, tool search, context management, authentication, and a sandboxed workbench to help you build AI agents that turn intent into action.

Build, Manage and Deploy AI/ML Systems

Debug, evaluate, and monitor your LLM applications, RAG systems, and agentic workflows with comprehensive tracing, automated evaluations, and production-ready dashboards.

The platform for LLM evaluations and AI agent testing

ZenML 🙏: One AI Platform from Pipelines to Agents. https://zenml.io.

🐢 Open-Source Evaluation & Testing library for LLM Agents

Fastest enterprise AI gateway (50x faster than LiteLLM) with adaptive load balancer, cluster mode, guardrails, 1000+ models support & <100 µs overhead at 5k RPS.

Test your prompts, agents, and RAGs. Red teaming/pentesting/vulnerability scanning for AI. Compare performance of GPT, Claude, Gemini, DeepSeek, and more. Simple declarative configs with command line and CI/CD integration. Used by OpenAI and Anthropic.

Community maintained hardware plugin for vLLM on Huawei Ascend

The fastest, litest AI Gateway. Rust core with Python SDK. Call 100+ LLM APIs in OpenAI (or native) format with cost tracking, guardrails, load balancing, and logging [Bedrock, Azure, OpenAI, Anthropic, OpenAI, VertexAI, vLLM, Nvidia NIM]

Build applications that make decisions (chatbots, agents, simulations, etc...). Monitor, trace, persist, and execute on your own infrastructure.

The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!

Dynamiq is an orchestration framework for agentic AI and LLM applications

🪢 Open source agent evals & observability: Trace, evaluate, and improve LLM applications with one open platform.

Fine-tune LLMs from one YAML. Layer streaming trains an 8B model on a 4 GB laptop GPU.

⚙️🦀 Build modular and scalable LLM Applications in Rust

OpenTelemetry Instrumentation for AI Observability

Open-source observability & evaluation platform for AI agents and coding agents. Trace LLMs, tools, prompts, costs & agent workflows with OpenTelemetry.

Open-source observability for your GenAI or LLM application, based on OpenTelemetry

Evaluation and Tracking for LLM Experiments and AI Agents

ClearML - Auto-Magical CI/CD to streamline your AI workload. Experiment Management, Data Management, Pipeline, Orchestration, Scheduling & Serving in one MLOps/LLMOps solution

The platform for continuously improving AI agents.