# AI verify loop

> How inspect.software measures the agent verify loop — one-command bootstrap, tests, lint, type checking, reproducible environments. The heaviest AI Readiness metric.


**AI verify loop** measures whether an AI coding agent can set up the
project, run it, and verify its own change without human help. This is the
crux of autonomous agent work — an agent that can check its work compounds;
one that cannot merely generates plausible text — and it carries the
heaviest weight in the [AI Readiness](/wiki/ai-readiness) badge.

- **Category:** [AI Readiness](/wiki/ai-readiness) (40% within category)
- **Weight in overall index:** 0% — part of the independent AI Readiness badge
- **Metric key:** `ai_verify_loop`

## How the value is computed

| Component | Weight | Evidence |
| --------- | ------ | -------- |
| One-command bootstrap | 18 | Makefile, Taskfile, justfile, mise, or noxfile earns full credit; a toolchain that defines the command itself (`Cargo.toml`, `go.mod`, `mix.exs`, Maven, Gradle, `.csproj`) earns most of it |
| Automated tests | 22 | a test suite the agent can run to self-check (shared with [engineering practices](/wiki/engineering-practices)) |
| Lint / format config | 11 | shared with the engineering linter signal |
| Static type checking | 11 | a statically typed language, or a type-check configuration (mypy, pyright, tsconfig, `py.typed`) |
| Reproducible environment | 10 | devcontainer, Dockerfile, Nix, or a dependency lockfile |
| Demonstrated agent practice | 10 | share of recent commits authored or co-authored by a coding agent; full marks from 5% upward |
| Automated maintenance | 8 | dependency-update bot commits observed in the sample; a configuration with nothing observed earns partial credit |
| OpenSSF Scorecard: Pinned-Dependencies | 10 | Scorecard's 0–10 result, scaled to 10 points |

`Pinned-Dependencies` is shared evidence: it adds supply-chain reproducibility
information to the agent verification loop while remaining a full Security
component. It is excluded when Scorecard is unavailable or reports `n/a`.

## Evidence, not only equipment

Every other component here reads the repository's files: a task runner exists,
a lockfile exists. Each is a proxy for a loop nobody has watched run.

Commits authored by a coding agent — or credited to one in a `Co-authored-by`
trailer — are the one available record that the loop was actually closed. A
change was proposed, verified, and merged with an agent in the loop, and the
project's own history says so.

A dependency bot's pull request is machine-authored too, and it must clear the
same gates: the tests run, the checks pass, it merges. A project already
absorbing that traffic has shown the pathway an agent needs, which is why
**automated maintenance** is scored beside agent practice rather than folded
into it — either can be present without the other.

What counts is the observed commits, not the configuration file. A
`dependabot.yml` can sit in a repository with the integration switched off, and
Renovate is commonly configured outside the repository altogether: two widely
used projects run it across roughly a third of their commits while carrying no
recognizable config file at all. Only bots that author their own content
qualify — a robot that merges other people's work is not writing changes.

The same commits count *against* a repository in
[development activity](/wiki/development-activity), where automation crowding
out human work is a warning sign. That is deliberate. The two metrics ask
different questions: whether people still maintain a project, and whether
machines can contribute to it. Both answers can be true at once.

This evidences **adoption, not autonomy**. A maintainer working interactively
with an agent leaves the same trace as an unattended run, and public data
cannot tell the two apart, which is why the component carries a modest weight
and makes no claim about how much of the work the agent did. Detection also
recognizes only the agents it knows about, so a project using an unrecognized
tool simply reads as having no evidence — never as failing.

The component is excluded rather than scored as zero when no commit sample was
collected.

## Why the toolchain counts as a bootstrap

`cargo test` and `go test ./...` are the canonical verify loops of their
languages. Crediting only task runners charged those ecosystems for having
better defaults — well-run Rust projects scored zero on this component for
shipping no Makefile. `package.json` and `pyproject.toml` are deliberately not
credited: npm and Python define no universal test command, and whether a given
project defines one lives inside file contents this metric does not read.

## Why this loop decides agent usefulness

Each component removes one failure mode of autonomous work:

- **Bootstrap** — the agent can go from clone to running without archaeology.
- **Tests** — the agent can prove a change did what was intended.
- **Lint and types** — entire classes of error are caught mechanically,
  before review.
- **Reproducible environment** — "works in the agent's sandbox" means works
  elsewhere too.

Notably, this is the same infrastructure that serves human contributors —
the metric rewards no agent-specific exotica beyond what a disciplined
project already has. A strong [Engineering Quality](/wiki/engineering-quality)
project typically starts strong here.

## Reading the result

- Signals confirm the loop's *existence*, not its speed or coverage — the
  presence-based honesty rule of the whole badge.
- A high verify loop with a stub
  [agent context](/wiki/ai-agent-context) usually means a well-engineered
  project that simply has not written agent guidance yet — the cheapest
  possible improvement.

## Improving the value

- Add a Makefile (or justfile/Taskfile) exposing `install`, `test`, and
  `lint` targets.
- Keep an automated test suite runnable locally in one command.
- Adopt type checking — even a minimal mypy/tsconfig counts.
- Commit a lockfile, Dockerfile, or devcontainer for environment
  reproducibility.
- Where agents are already used, keep the `Co-authored-by` trailer the tools
  add by default. Stripping it removes the only public evidence that the loop
  closes in practice.

Related: [AI code legibility](/wiki/ai-code-legibility) ·
[engineering practices](/wiki/engineering-practices)