Public record
Software health reportschema 0.34.0 · metrics 2.10.0 · 2026-09-17 09:35 UTC

benchflow-ai / benchflow

Research infra for creating RL environments, post-training, and evals.

PythonApache-2.0★ 349 stars⑂ 65 forkssince Jan 2025View on GitHub ↗
KindCommand-line toolLibraryhow this is determined
#benchmark#evals

benchflow-ai/benchflow holds a health index of 84 out of 100, placing it in the Excellent band. It scores highest on Engineering Quality (89/100) and lowest on Security (47/100). It was last updated 1 day ago. A single contributor accounts for most of its recent work.

84
overall / 100
Excellent

Software health index

Metrics are grouped into weighted categories on one standardized 1–100 scale. Overall starts as their weighted mean, calibrated against the distribution of the public record so bands carry percentile meaning; when public evidence triggers the High-Risk Jurisdiction Policy, the rating is adjusted and receives an At Risk ceiling of 34.

84
Exceptional93-100The record's top tier (≈ top 5%); essentially all checked criteria met
Excellent80-92Strong across the board; minor gaps
Good65-79Healthy; gaps are limited and manageable
Moderate50-64Acceptable with notable gaps; review recommended
Weak35-49Material weaknesses across several areas
At Risk20-34Significant weaknesses; adoption warrants caution
Critical1-19Severe problems (abandoned, single-maintainer, no hygiene)
VitalityCommunity &AdoptionSustainability &GovernanceEngineeringQualitySecurityAI Readiness

Score profile

Each axis is a category. The shape matters more than the average — a healthy subject fills the whole shape, while a spike-and-crater profile means strength in one dimension is masking risk in another.

The weighted overall 71 is calibrated to 84 on the published index scale (record calibration 2026-08-02).

Ownership

BenchFlowOrganization
143 followers27 public repossince Dec 2024

This repository is backed by an organization — shared, accountable stewardship that can outlive any single maintainer.

Package ecosystems

RegistryPackageVersionDownloads / moVersionsLast publishTags
PyPIbenchflow0.7.814,2881461 day agoacpagent-evaluationbenchmarkllm-agentsmulti-turnskillsbench

Metrics by category

Vitality

Is the project alive — is code being written and are releases shipping?

86Excellent · 21% of overall

Development activity

83Excellent
How it's scored
36/36Push recency — last push 1 days ago
18.7/36Commit cadence — 27/52 weeks with commits
18/18Commit volume — 1,316 commits in the last year
10/10OpenSSF Scorecard: Maintained — 30 commit(s) and 10 issue activity found in the last 90 days -- score normalized to 10
Inputs used
commits_last_year1,316
human_commit_share1
days_since_last_push1
active_weeks_last_year27

Release discipline

90Excellent
How it's scored
27/27Ships releases — 36 releases published
36/36Release recency — latest release 1 days ago
27/27Release cadence — a release every ~3.4 days
0/10OpenSSF Scorecard: Signed-Releases — Project has not signed or included provenance with any releases.
Inputs used
releases_count36
latest_release_tagv0.7.8
releases_from_tagsno
days_since_latest_release1
mean_days_between_releases3.4

Community & Adoption

Does the project have users, downloads, attention, and a welcoming setup for contributors?

57Moderate · 17% of overall

Popularity & adoption

56Moderate
How it's scored
41.2/60Stars — 349 stars
15.1/25Forks — 65 forks
0/15Watchers — 2 watchers
Inputs used
forks65
stars349
watchers2
growth_stateunverified
growth_factor_pct100
growth_unverified_reasonno_history

Community health

50Moderate
How it's scored
22.5/22.5README
22.5/22.5License — recognized license (Apache-2.0)
0/18CONTRIBUTING guide
0/13.5Code of conduct
0/7.2Issue template
0/6.3PR template
Inputs used
has_readmeyes
has_licenseyes
readme_badges2
has_contributingno
has_issue_templateno
has_code_of_conductno
readme_badge_servicesshields.io
has_pull_request_templateno

Ecosystem adoption (downloads)

69Good
How it's scored
55.4/80Monthly downloads — 14,288 downloads/month across pypi
0/20Registry dependents — not reported by this ecosystem
Inputs used
packagesbenchflow
dependents—
ecosystemspypi
total_downloads—
monthly_downloads14,288
unverified_packages_excluded—
Excluded from scoring (no data or not applicable): Registry dependents. Remaining weights renormalized.

Sustainability & Governance

Will the project survive its people — bus factor, responsiveness, who backs it, and package upkeep?

67Good · 23% of overall

Maintainer resilience (bus factor)

42Weak
How it's scored
9/54Bus factor — 1 contributor(s) cover half of all commits
10/22.5Commit distribution — top contributor authored 56% of commits
13.5/13.5Contributor breadth — 20 contributors
10/10OpenSSF Scorecard: Contributors — project has 14 contributing companies or organizations
Inputs used
bus_factor1
contributors_sampled20
top_contributor_share0.557

Issue & PR responsiveness

79Good
How it's scored
36.6/42Issue resolution — 87% of issues closed
26.2/30PR acceptance — 684/782 decided PRs merged
12.1/13Newcomer PR acceptance — 13/14 first-time contributors' PRs merged in 30d
4.5/15OpenSSF Scorecard: Code-Review — Found 8/26 approved changesets -- score normalized to 3
Inputs used
merged_prs684
open_issues31
closed_issues210
prs_merged_7d11
prs_decided_7d11
prs_merged_30d38
prs_decided_30d43
issue_closed_ratio0.871
closed_unmerged_prs98
first_time_authors_30d7
first_time_prs_merged_30d13
first_time_prs_decided_30d14

Ownership & stewardship

60Moderate
How it's scored
30/30Ownership backing — organization-owned
0/20Verified domain
15.5/25Owner reach — 143 followers of benchflow-ai
14.1/25Track record — 27 public repos, account ~1 yr old
Inputs used
followers143
owner_typeOrganization
is_verifiedno
owner_loginbenchflow-ai
public_repos27
account_age_days654

Package maintenance

100Exceptional
How it's scored
25/25Published & resolvable — 1 package(s) on pypi
35/35Publish recency — latest publish 1 days ago
20/20Version history — 146 published versions
20/20Not deprecated — active, not deprecated or yanked
Inputs used
packagesbenchflow
ecosystemspypi
any_deprecatedno
min_days_since_publish1

Engineering Quality

Are baseline engineering and documentation practices in place?

89Excellent · 19% of overall

Engineering practices

82Excellent
How it's scored
24/24CI workflows — 8 workflow(s)
24/24Tests present
16/16Linter config — pyproject.toml ([tool.ruff])
0/9.6Pre-commit hooks
0/6.4.editorconfig
18/20OpenSSF Scorecard: CI-Tests — 25 out of 26 merged PRs checked by a CI test -- score normalized to 9
Inputs used
has_ciyes
has_testsyes
has_editorconfigno
has_linter_configyes
has_precommit_configno

Documentation

100Exceptional
How it's scored
30/30README
25/25Documentation directory
15/15Documentation / homepage site — https://benchflow.ai/
10/10Repository description
10/10Topics — 2 topics
10/10Wiki
Inputs used
topicsbenchmark, evals
has_wikiyes
homepagehttps://benchflow.ai/
docs_sitehttps://benchflow.ai/
has_readmeyes
has_docs_diryes
has_descriptionyes

Security

Are visible security and supply-chain practices strong, without unresolved high-risk jurisdiction exposure?

47Weak · 16% of overall

Security posture

40Weak
How it's scored
7.5/7.5Binary-Artifacts — no binaries found in the repo
0/7.5Branch-Protection — no data
2.2/2.5CI-Tests — 25 out of 26 merged PRs checked by a CI test -- score normalized to 9
0/2.5CII-Best-Practices — no effort to earn an OpenSSF best practices badge detected
2.2/7.5Code-Review — Found 8/26 approved changesets -- score normalized to 3
2.5/2.5Contributors — project has 14 contributing companies or organizations
10/10Dangerous-Workflow — no dangerous workflow patterns detected
0/7.5Dependency-Update-Tool — no update tool detected
0/5Fuzzing — project is not fuzzed
2.5/2.5License — license file detected
7.5/7.5Maintained — 30 commit(s) and 10 issue activity found in the last 90 days -- score normalized to 10
0/5Packaging — no data
1/5Pinned-Dependencies — dependency not pinned by hash detected -- score normalized to 2
0/5SAST — SAST tool is not run on all commits -- score normalized to 0
0/5Security-Policy — security policy file not detected
0/7.5Signed-Releases — Project has not signed or included provenance with any releases.
0/7.5Token-Permissions — detected GitHub workflow tokens with excessive permissions
1.5/7.5Vulnerabilities — 8 existing vulnerabilities detected
Inputs used
sourceopenssf_scorecard
checks_evaluated16
scorecard_versionv5.5.0
checks_inconclusive2
scorecard_aggregate4
Excluded from scoring (no data or not applicable): Branch-Protection, Packaging. Remaining weights renormalized.

Dependency advisories

77Good
How it's scored
12/35Direct dependencies free of known advisories — 1 affected: cryptography 48.0.1 (high 8.2)
25/25Indirect dependencies free of known advisories — no indirect dependency carries a known advisory
40/40No advisories left outstanding — no advisory has been public longer than 90 days
Inputs used
sourceosv
advisories6
affected_packages1
assessed_packages125
unassessed_packages0
affected_by_severityhigh 1
direct_affected_packages1
Matched the pypi:benchflow@0.7.8 runtime dependency closure — what installing the published package pulls in — 125 packages. Reachability is not analyzed.

AI Readiness

How well is the repo equipped to be developed and maintained with AI coding agents? Carries a deliberately small weight (4%): agent tooling is a real maintenance signal, but a repository with none can still reach 100/100.

84Excellent · 4% of overall

Agent context & guidance

100Exceptional
How it's scored
45/45Agent instructions — AGENTS.md, CLAUDE.md
15/15Machine-readable docs (llms.txt) — llms.txt served by the project website (https://benchflow.ai/llms.txt)
40/40Legible commit history — 95 of 100 human commits state their intent (structured subject or explanatory body)
Inputs used
has_llms_txtyes
llms_txt_urlhttps://benchflow.ai/llms.txt
legible_history_share0.95
agent_instruction_filesAGENTS.md, CLAUDE.md
agent_instruction_max_bytes6,509

Verify loop (build / test / typecheck)

66Good
How it's scored
0/18One-command bootstrap
22/22Automated tests
11/11Lint / format config — pyproject.toml ([tool.ruff])
11/11Static type checking — src/benchflow/py.typed
10/10Reproducible environment — devcontainer, Dockerfile, lockfile
10/10Demonstrated agent practice — 6 of the last 100 commits agent-authored or agent-credited
0/8Automated maintenance — no automated dependency updates observed
2/10OpenSSF Scorecard: Pinned-Dependencies — dependency not pinned by hash detected -- score normalized to 2
Inputs used
has_nixno
has_testsyes
lockfilesuv.lock
has_dockerfileyes
typed_languageno
bootstrap_files—
has_devcontaineryes
has_linter_configyes
typecheck_configssrc/benchflow/py.typed
agent_commit_share0.06
toolchain_manifests—
dependency_bot_commit_share0

Code legibility for models

81Excellent
How it's scored
27/45Type-checkable code — Python with type-check config (src/benchflow/py.typed)
54.2/55Manageable file sizes — 12/786 source files over 60KB
Inputs used
primary_languagePython
largest_source_bytes121,773
source_files_sampled786
oversized_source_files12

Machine-readable interfaces

100Exceptional
How it's scored
0/40API schema (OpenAPI/GraphQL/proto) — not applicable to this kind of software
20/20MCP server
40/40Runnable examples — examples, notebooks
Inputs used
example_dirsexamples, notebooks
has_mcp_signalyes
api_schema_files—
interfaces_expected_of—
Excluded from scoring (no data or not applicable): API schema (OpenAPI/GraphQL/proto). Remaining weights renormalized.

Key facts

349GitHub stars
20contributors
1,316commits, last 12 months
1days since last push
36releases
1bus factor
31open issues
PyPIpackage ecosystems

Data collection warnings

  • Star history unavailable: GitHub GraphQL error: Resource not accessible by personal access token

More detail

Star and fork history 0 ★ / 65 ⇿
0Stars
65Forks
35Releases

When each star and fork was added, collected from GitHub and bucketed by day. Cumulative growth sits directly above the daily additions it is made of, so the two read against each other: steady organic accretion looks nothing like an abrupt, short-lived burst. Where that difference is measurable, it is reported as growth authenticity.

01325385063756542025-022025-112026-09
Major 0Minor 5Patch 24

Each point covers 2 days.

OpenSSF Scorecard 4.0 / 10
4.0aggregate

Independent, tool-agnostic security assessment from the open-source OpenSSF Scorecard. Each check rewards a security practice, not a specific vendor's tool. Checks Scorecard could not determine are marked n/a and excluded from the security score (never counted as zero).Scorecard v5.5.0 · 2026-09-17 09:34 UTC

10Binary-Artifactsno binaries found in the repo
n/aBranch-Protectioninternal error: error during branchesHandler.setup: internal error: some github tokens can't read classic branch protection rules: https://github.com/ossf/scorecard-action/blob/main/docs/authentication/fine-grained-auth-token.md
9CI-Tests25 out of 26 merged PRs checked by a CI test -- score normalized to 9
0CII-Best-Practicesno effort to earn an OpenSSF best practices badge detected
3Code-ReviewFound 8/26 approved changesets -- score normalized to 3
10Contributorsproject has 14 contributing companies or organizations
10Dangerous-Workflowno dangerous workflow patterns detected
0Dependency-Update-Toolno update tool detected
0Fuzzingproject is not fuzzed
10Licenselicense file detected
10Maintained30 commit(s) and 10 issue activity found in the last 90 days -- score normalized to 10
n/aPackagingpackaging workflow not detected
2Pinned-Dependenciesdependency not pinned by hash detected -- score normalized to 2
0SASTSAST tool is not run on all commits -- score normalized to 0
0Security-Policysecurity policy file not detected
0Signed-ReleasesProject has not signed or included provenance with any releases.
0Token-Permissionsdetected GitHub workflow tokens with excessive permissions
2Vulnerabilities8 existing vulnerabilities detected
Direct dependencies 12
RegistryPackageVersion constraintManifest
PyPIagent-client-protocol>=0.10pyproject.toml
PyPIhttpx>=0.27.0pyproject.toml
PyPIanyio>=4.0pyproject.toml
PyPIpydantic>=2.7pyproject.toml
PyPIpyyaml>=6.0pyproject.toml
PyPIrich>=13.0pyproject.toml
PyPIcryptography>=44pyproject.toml
PyPIpackaging>=24pyproject.toml
PyPIlitellm==1.91.0pyproject.toml
PyPIgoogle-cloud-aiplatform>=1.133.0,<2.0pyproject.toml
PyPItomli-w>=1.0pyproject.toml
PyPItyper>=0.9pyproject.toml
All dependencies 259

Full resolved dependency set from the GitHub dependency graph: 14 direct and 245 indirect (transitive) packages. The transitive closure is complete when the repository commits a lockfile.

RegistryPackageVersionRelation
PyPIagent-client-protocol0.10.0direct
PyPIanyio4.8.0direct
PyPIcryptography48.0.1direct
PyPIgoogle-cloud-aiplatform1.163.0direct
PyPIhttpx—direct
PyPIhttpx0.28.1direct
PyPIlitellm1.91.0direct
PyPIpackaging26.0direct
PyPIpydantic—direct
PyPIpydantic2.12.5direct
PyPIpyyaml6.0.3direct
PyPIrich13.9.4direct
PyPItomli-w1.2.0direct
PyPItyper0.24.1direct
PyPIaccelerate1.14.0indirect
PyPIaiofiles24.1.0indirect
PyPIaiohappyeyeballs2.6.1indirect
PyPIaiohttp3.14.3indirect
PyPIaiohttp-retry2.9.1indirect
PyPIaiosignal1.4.0indirect
PyPIannotated-doc0.0.4indirect
PyPIannotated-types0.7.0indirect
PyPIanthropic0.104.0indirect
PyPIapscheduler3.11.2indirect
PyPIattrs26.1.0indirect
PyPIazure-core1.41.0indirect
PyPIazure-data-tables12.7.0indirect
PyPIazure-identity1.25.3indirect
PyPIazure-storage-blob12.29.0indirect
PyPIazure-storage-queue12.17.0indirect
PyPIbackoff2.2.1indirect
PyPIbedrock-agentcore1.18.1indirect
PyPIbenchflow0.7.9.dev0indirect
PyPIboto31.43.56indirect
PyPIbotocore1.43.56indirect
PyPIbracex2.6indirect
PyPIcbor26.1.1indirect
PyPIcertifi2024.12.14indirect
PyPIcffi2.0.0indirect
PyPIcfgv3.5.0indirect
PyPIcharset-normalizer3.4.7indirect
PyPIclaw-gmail—indirect
PyPIclick—indirect
PyPIclick8.4.2indirect
PyPIcolorama0.4.6indirect
PyPIcroniter6.2.2indirect
PyPIcuda-bindings13.3.1indirect
PyPIcuda-pathfinder1.5.6indirect
PyPIcuda-toolkit13.0.3.0indirect
PyPIdatasets5.0.0indirect
PyPIdaytona0.184.0indirect
PyPIdaytona-api-client0.184.0indirect
PyPIdaytona-api-client-async0.184.0indirect
PyPIdaytona-toolbox-api-client0.184.0indirect
PyPIdaytona-toolbox-api-client-async0.184.0indirect
PyPIdeepagents0.6.8indirect
PyPIdeprecated1.3.1indirect
PyPIdill0.4.1indirect
PyPIdistlib0.4.0indirect
PyPIdistro1.9.0indirect
PyPIdnspython2.8.0indirect
PyPIdocstring-parser0.18.0indirect
PyPIemail-validator2.3.0indirect
PyPIexpression5.6.0indirect
PyPIfaker—indirect
PyPIfastapi—indirect
PyPIfastapi0.136.3indirect
PyPIfastapi-mcp—indirect
PyPIfastapi-sso0.21.0indirect
PyPIfastuuid0.14.0indirect
PyPIfilelock3.25.2indirect
PyPIfiletype1.2.0indirect
PyPIfrozenlist1.8.0indirect
PyPIfsspec2026.4.0indirect
PyPIgoogle-api-core2.34.0indirect
PyPIgoogle-auth2.53.0indirect
PyPIgoogle-cloud-bigquery3.43.0indirect
PyPIgoogle-cloud-core2.6.1indirect
PyPIgoogle-cloud-resource-manager1.18.0indirect
PyPIgoogle-cloud-storage3.13.1indirect
PyPIgoogle-crc32c1.8.0indirect
PyPIgoogle-genai2.5.0indirect
PyPIgoogle-resumable-media2.10.1indirect
PyPIgoogleapis-common-protos1.75.0indirect
PyPIgranian2.7.5indirect
PyPIgreenlet3.5.5indirect
PyPIgrpc-google-iam-v10.14.5indirect
PyPIgrpcio1.83.0indirect
PyPIgrpcio-status1.83.0indirect
PyPIgrpclib0.4.9indirect
PyPIgunicorn23.0.0indirect
PyPIgymnasium—indirect
PyPIh110.16.0indirect
PyPIh24.4.1indirect
PyPIhf-xet1.5.0indirect
PyPIhpack4.2.0indirect
PyPIhttpcore1.0.9indirect
PyPIhttpx-sse0.4.3indirect
PyPIhttpx-ws0.9.0indirect
PyPIhuggingface-hub1.16.1indirect
PyPIhyperframe6.1.0indirect
PyPIidentify2.6.18indirect
PyPIidna3.16indirect
PyPIimportlib-metadata8.7.1indirect
PyPIiniconfig2.0.0indirect
PyPIisodate0.7.2indirect
PyPIjinja2—indirect
PyPIjinja23.1.6indirect
PyPIjiter0.15.0indirect
PyPIjmespath1.1.0indirect
PyPIjsonpatch1.33indirect
PyPIjsonpointer3.1.1indirect
PyPIjsonschema4.26.0indirect
PyPIjsonschema-specifications2025.9.1indirect
PyPIlangchain1.3.7indirect
PyPIlangchain-anthropic1.4.5indirect
PyPIlangchain-core1.4.6indirect
PyPIlangchain-google-genai4.2.5indirect
PyPIlangchain-openai1.3.0indirect
PyPIlangchain-protocol0.0.16indirect
PyPIlanggraph1.2.4indirect
PyPIlanggraph-checkpoint4.1.1indirect
PyPIlanggraph-prebuilt1.1.0indirect
PyPIlanggraph-sdk0.4.2indirect
PyPIlangsmith0.8.15indirect
PyPIlitellm-enterprise0.1.44indirect
PyPIlitellm-proxy-extras0.4.74indirect
PyPImarkdown-it-py4.0.0indirect
PyPImarkupsafe3.0.3indirect
PyPImcp1.28.1indirect
PyPImdurl0.1.2indirect
PyPImodal1.4.2indirect
PyPImpmath1.3.0indirect
PyPImsal1.37.0indirect
PyPImsal-extensions1.3.1indirect
PyPImultidict6.7.1indirect
PyPImultiprocess0.70.19indirect
PyPInetworkx3.6.1indirect
PyPInodeenv1.10.0indirect
PyPInumpy2.4.6indirect
PyPInvidia-cublas13.1.1.3indirect
PyPInvidia-cuda-cupti13.0.85indirect
PyPInvidia-cuda-nvrtc13.0.88indirect
PyPInvidia-cuda-runtime13.0.96indirect
PyPInvidia-cudnn-cu139.20.0.48indirect
PyPInvidia-cufft12.0.0.61indirect
PyPInvidia-cufile1.15.1.6indirect
PyPInvidia-curand10.4.0.35indirect
PyPInvidia-cusolver12.0.4.66indirect
PyPInvidia-cusparse12.6.3.3indirect
PyPInvidia-cusparselt-cu130.8.1indirect
PyPInvidia-nccl-cu132.29.7indirect
PyPInvidia-nvjitlink13.3.33indirect
PyPInvidia-nvshmem-cu133.4.5indirect
PyPInvidia-nvtx13.0.85indirect
PyPIoauthlib3.3.1indirect
PyPIobstore0.8.2indirect
PyPIopenai2.38.0indirect
PyPIopentelemetry-api1.41.1indirect
PyPIopentelemetry-exporter-otlp-proto-common1.41.1indirect
PyPIopentelemetry-exporter-otlp-proto-http1.41.1indirect
PyPIopentelemetry-instrumentation0.62b1indirect
PyPIopentelemetry-instrumentation-aiohttp-client0.62b1indirect
PyPIopentelemetry-proto1.41.1indirect
PyPIopentelemetry-sdk1.41.1indirect
PyPIopentelemetry-semantic-conventions0.62b1indirect
PyPIopentelemetry-util-http0.62b1indirect
PyPIorjson3.11.9indirect
PyPIormsgpack1.12.2indirect
PyPIpandas3.0.3indirect
PyPIpathspec1.1.1indirect
PyPIplatformdirs4.9.6indirect
PyPIplaywright1.62.0indirect
PyPIpluggy1.5.0indirect
PyPIpolars1.41.2indirect
PyPIpolars-runtime-321.41.2indirect
PyPIpre-commit4.5.1indirect
PyPIpropcache0.5.2indirect
PyPIproto-plus1.28.3indirect
PyPIprotobuf6.33.6indirect
PyPIpsutil7.2.2indirect
PyPIpyarrow24.0.0indirect
PyPIpyasn10.6.4indirect
PyPIpyasn1-modules0.4.2indirect
PyPIpycparser3.0indirect
PyPIpydantic-core2.41.5indirect
PyPIpydantic-settings2.14.2indirect
PyPIpyee13.0.1indirect
PyPIpygments2.20.0indirect
PyPIpyjwt2.13.0indirect
PyPIpynacl1.6.2indirect
PyPIpyopenssl26.2.0indirect
PyPIpyroscope-io0.8.16indirect
PyPIpytest—indirect
PyPIpytest9.0.3indirect
PyPIpytest-asyncio—indirect
PyPIpytest-asyncio1.3.0indirect
PyPIpytest-json-report—indirect
PyPIpython-dateutil2.9.0.post0indirect
PyPIpython-discovery1.2.2indirect
PyPIpython-dotenv1.2.2indirect
PyPIpython-multipart—indirect
PyPIpython-multipart0.0.32indirect
PyPIpywin32311indirect
PyPIredis8.0.0indirect
PyPIreferencing0.37.0indirect
PyPIregex2026.5.9indirect
PyPIrequests2.34.2indirect
PyPIrequests-toolbelt1.0.0indirect
PyPIrestrictedpython8.3indirect
PyPIrpds-py2026.5.1indirect
PyPIrq2.9.0indirect
PyPIruff—indirect
PyPIruff0.15.7indirect
PyPIs3transfer0.19.2indirect
PyPIsafetensors0.8.0indirect
PyPIsetuptools—indirect
PyPIsetuptools83.0.0indirect
PyPIshellingham1.5.4indirect
PyPIsix1.17.0indirect
PyPIsniffio1.3.1indirect
PyPIsoundfile0.13.1indirect
PyPIsqlalchemy—indirect
PyPIsse-starlette3.4.4indirect
PyPIstarlette1.3.1indirect
PyPIsympy1.14.0indirect
PyPIsynchronicity0.12.2indirect
PyPItenacity9.1.4indirect
PyPItiktoken0.13.0indirect
PyPItokenizers0.22.2indirect
PyPItoml0.10.2indirect
PyPItorch2.13.0indirect
PyPItqdm4.67.3indirect
PyPItransformers5.6.2indirect
PyPItriton3.7.1indirect
PyPItrl1.8.0indirect
PyPIty0.0.29indirect
PyPItypes-certifi2021.10.8.3indirect
PyPItypes-toml0.10.8.20260518indirect
PyPItyping-extensions4.15.0indirect
PyPItyping-inspection0.4.2indirect
PyPItzdata2026.2indirect
PyPItzlocal5.3.1indirect
PyPIurllib32.7.0indirect
PyPIuuid-utils0.16.0indirect
PyPIuvicorn—indirect
PyPIuvicorn0.49.0indirect
PyPIuvloop0.22.1indirect
PyPIvirtualenv21.2.1indirect
PyPIwatchfiles1.2.0indirect
PyPIwcmatch10.1indirect
PyPIwebsockets15.0.1indirect
PyPIwheel—indirect
PyPIwrapt2.1.2indirect
PyPIwsproto1.3.2indirect
PyPIxxhash3.7.0indirect
PyPIyarl1.23.0indirect
PyPIzipp3.23.1indirect
PyPIzstandard0.25.0indirect
Dependency advisories 1

Installing pypi:benchflow@0.7.8 pulls in 125 packages, direct and transitive: 1 carry known advisories, of which 1 are direct dependencies.

PackageVersionRelationSeverityAdvisoriesFixed in
cryptography48.0.1directhigh650.0.0

An advisory means the version recorded in the dependency graph falls inside an advisory’s affected range. Reachability is not analysed, and the graph includes development and test pins — a finding may concern tooling rather than shipped software.

Raw JSON report machine-readable

Feedback

Spotted something off in this report, or have thoughts to share? Wrong measurements, missed tooling, ideas, questions — anything is welcome. Every message is read and gets a response.

The message is kept through sign-in.

Related records

Inspected repositories sharing catalogue tags or ecosystems with benchflow-ai/benchflow. Read them side by side →

92UiPath/coder_evalPlaywright for coding agents. Test that your skills, MCP servers, and CLIs actually work when an agent uses them — sandboxed YAML suites, A/B experiments, CI gates.Excellent94truera/trulensEvaluation and Tracking for LLM Experiments and AI AgentsExceptional94langchain-ai/deepagentsThe batteries-included agent harness.Exceptional80alizahidraja/isnadGrade every agent, scraper and model in a claim's chain — provenance, trust scoring and audit evidence for LLM pipelinesExcellent78scaleapi/agentenv-frameworkCreating realistic RL environments requires collaboration between researchers, engineers, and domain experts across many dimensions: artifacts, environments tools, dynamism of the environment, reproducibility, and more. There is no open source framework for building these environments effectively. Until now.Good62inferock/inferock-benchLocal LLM cost-tracking proxy for OpenAI, Anthropic, Gemini, and pinned OpenRouter calls with token usage, failure, and billing-integrity receipts.Moderate

Scores are signals, not warranties. They reflect publicly visible practices on GitHub — not a code audit, and not a security guarantee.

Missing data is excluded and weights renormalized, never scored as zero. Methodology is versioned and open: metrics v2.10.0, schema v0.34.0 — full methodology · metrics wiki.

How one result sits in the wider record: aggregate statistics — PyPI.