#llm-evaluation

Quantize an LLM and check it still refuses what it should — a GPU-aware quantization CLI that measures the safety drift of the quantization it just performed.

Creating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: artifacts, environments tools, dynamism of the environment, reproducibility, and more. There is no open source framework for building these environments effectively. Until now.

Leaderboard reproduzível de LLMs em provas médicas brasileiras (Revalida, INEP) — zero-shot, 3 execuções, análise de contaminação

Build, enrich, and transform datasets using AI models with no code

Keine Repository-Beschreibung veröffentlicht.

Your prompts need tests too. Run prompts against real datasets, score outputs with LLM judges, version everything, and compare runs to see what got better.

Flywheel runs AI tasks with a model you choose and keeps a receipt anyone can recheck offline.

Python foundation for LLM apps whose prompts and spend you can actually see — versioned prompts, every call recorded and replayable, budgets, evals, jobs.