Public record
Software health reportschema 0.34.0 · metrics 2.10.0 · 2026-10-19 04:50 UTC

mlcommons / medperf

An open benchmarking platform for medical artificial intelligence using Federated Evaluation.

Python · HTMLApache-2.0★ 172 stars⑂ 36 forkssince Oct 2021View on GitHub ↗
KindCommand-line toolhow this is determined

mlcommons/medperf holds a health index of 65 out of 100, placing it in the Good band. It scores highest on Sustainability & Governance (80/100) and lowest on Security (36/100). It was last updated 68 days ago. 2 contributors account for most of its recent work.

65
overall / 100
Good

Software health index

Metrics are grouped into weighted categories on one standardized 1–100 scale. Overall starts as their weighted mean, calibrated against the distribution of the public record so bands carry percentile meaning; when public evidence triggers the High-Risk Jurisdiction Policy, the rating is adjusted and receives an At Risk ceiling of 34.

65
Exceptional93-100The record's top tier (≈ top 5%); essentially all checked criteria met
Excellent80-92Strong across the board; minor gaps
Good65-79Healthy; gaps are limited and manageable
Moderate50-64Acceptable with notable gaps; review recommended
Weak35-49Material weaknesses across several areas
At Risk20-34Significant weaknesses; adoption warrants caution
Critical1-19Severe problems (abandoned, single-maintainer, no hygiene)
VitalityCommunity &AdoptionSustainability &GovernanceEngineeringQualitySecurityAI Readiness

Score profile

Each axis is a category. The shape matters more than the average — a healthy subject fills the whole shape, while a spike-and-crater profile means strength in one dimension is masking risk in another.

The weighted overall 60 is calibrated to 65 on the published index scale (record calibration 2026-08-02).

Ownership

MLCommonsOrganization · verified domain
1,498 followers139 public repossince Jul 2020

This repository is backed by an organization — shared, accountable stewardship that can outlive any single maintainer.

Metrics by category

Vitality

Is the project alive — is code being written and are releases shipping?

44Weak · 21% of overall

Development activity

44Weak
How it's scored
18/36Push recency — last push 68 days ago
9/36Commit cadence — 13/52 weeks with commits
12.4/18Commit volume — 23 commits in the last year
0/10OpenSSF Scorecard: Maintained — no data
Inputs used
commits_last_year23
human_commit_share1
days_since_last_push68
active_weeks_last_year13

Release discipline

44Weak
How it's scored
27/27Ships releases — 1 releases published
0/36Release recency — latest release 1,312 days ago
12.6/27Release cadence — cadence unknown (single release)
0/10OpenSSF Scorecard: Signed-Releases — no data
Inputs used
releases_count1
latest_release_tagv0.1.0
releases_from_tagsno
days_since_latest_release1,312
mean_days_between_releases—

Community & Adoption

Does the project have users, downloads, attention, and a welcoming setup for contributors?

62Moderate · 17% of overall

Popularity & adoption

55Moderate
How it's scored
36.2/60Stars — 172 stars
12.9/25Forks — 36 forks
5.8/15Watchers — 12 watchers
Inputs used
forks36
stars172
watchers12
growth_stateunverified
growth_factor_pct100
growth_unverified_reasonno_history

Community health

70Good
How it's scored
22.5/22.5README
22.5/22.5License — recognized license (Apache-2.0)
18/18CONTRIBUTING guide
0/13.5Code of conduct
0/7.2Issue template
0/6.3PR template
Inputs used
has_readmeyes
has_licenseyes
readme_badges0
has_contributingyes
has_issue_templateno
has_code_of_conductno
readme_badge_services—
has_pull_request_templateno

Sustainability & Governance

Will the project survive its people — bus factor, responsiveness, who backs it, and package upkeep?

80Excellent · 23% of overall

Maintainer resilience (bus factor)

58Moderate
How it's scored
25.2/54Bus factor — 2 contributor(s) cover half of all commits
13.7/22.5Commit distribution — top contributor authored 39% of commits
13.5/13.5Contributor breadth — 16 contributors
0/10OpenSSF Scorecard: Contributors — no data
Inputs used
bus_factor2
contributors_sampled16
top_contributor_share0.39

Issue & PR responsiveness

88Excellent
How it's scored
39.7/42Issue resolution — 95% of issues closed
23.7/30PR acceptance — 296/375 decided PRs merged
0/13Newcomer PR acceptance — no first-time contributor's PR decided in 30d
0/15OpenSSF Scorecard: Code-Review — no data
Inputs used
merged_prs296
open_issues16
closed_issues282
prs_merged_7d0
prs_decided_7d0
prs_merged_30d0
prs_decided_30d0
issue_closed_ratio0.946
closed_unmerged_prs79
first_time_authors_30d0
first_time_prs_merged_30d0
first_time_prs_decided_30d0
Excluded from scoring (no data or not applicable): Newcomer PR acceptance. Remaining weights renormalized.

Ownership & stewardship

98Exceptional
How it's scored
30/30Ownership backing — organization-owned
20/20Verified domain
22.8/25Owner reach — 1,498 followers of mlcommons
25/25Track record — 139 public repos, account ~6 yr old
Inputs used
followers1,498
owner_typeOrganization
is_verifiedyes
owner_loginmlcommons
public_repos139
account_age_days2,299

Engineering Quality

Are baseline engineering and documentation practices in place?

72Good · 19% of overall

Engineering practices

60Moderate
How it's scored
24/24CI workflows — 10 workflow(s)
24/24Tests present
0/16Linter config
0/9.6Pre-commit hooks
0/6.4.editorconfig
0/20OpenSSF Scorecard: CI-Tests — no data
Inputs used
has_ciyes
has_testsyes
has_editorconfigno
has_linter_configno
has_precommit_configno

Documentation

90Excellent
How it's scored
30/30README
25/25Documentation directory
15/15Documentation / homepage site — https://www.medperf.org
10/10Repository description
0/10Topics
10/10Wiki
Inputs used
topics—
has_wikiyes
homepagehttps://www.medperf.org
docs_sitehttps://www.medperf.org
has_readmeyes
has_docs_diryes
has_descriptionyes

Security

Are visible security and supply-chain practices strong, without unresolved high-risk jurisdiction exposure?

36Weak · 16% of overall

Security posture

20At Risk
How it's scored
0/30Security policy (SECURITY.md)
0/25Dependabot config
0/25Dependency lockfiles
20/20CodeQL workflow
Inputs used
sourcefile_signals
lockfiles—
manifestscli/requirements.txt, cli/setup.py, docs/requirements.txt, flca/setup.py, server/requirements.txt
has_codeql_workflowyes
has_security_policyno
has_dependabot_configno

Dependency advisories

100Exceptional
How it's scored
35/35Direct dependencies free of known advisories — no direct dependency carries a known advisory
0/25Indirect dependencies free of known advisories — transitive set not separable from development and test dependencies in this scope
0/40No advisories left outstanding — no advisory carries a publication date
Inputs used
sourceosv
advisories290
affected_packages19
assessed_packages94
unassessed_packages24
affected_by_severitycritical 7, moderate 5, low 2, unknown 5
direct_affected_packages0
Excluded from scoring (no data or not applicable): Indirect dependencies free of known advisories, No advisories left outstanding. Remaining weights renormalized. Matched 94 resolved dependencies against OSV. 24 could not be assessed — no resolved version, an unsupported ecosystem, or beyond the reported package list. This repository publishes no package the index resolves, so the repository dependency graph was assessed instead. That graph mixes development and test pins with shipped dependencies, so only the declared runtime dependencies are scored; transitive findings are reported as context and excluded from the score. Reachability is not analyzed.

AI Readiness

How well is the repo equipped to be developed and maintained with AI coding agents? Carries a deliberately small weight (4%): agent tooling is a real maintenance signal, but a repository with none can still reach 100/100.

50Moderate · 4% of overall

Agent context & guidance

40Weak
How it's scored
0/45Agent instructions — no CLAUDE.md / AGENTS.md / editor rules
0/15Machine-readable docs (llms.txt)
40/40Legible commit history — 96 of 100 human commits state their intent (structured subject or explanatory body)
Inputs used
has_llms_txtno
llms_txt_url—
legible_history_share0.96
agent_instruction_files—
agent_instruction_max_bytes—

Verify loop (build / test / typecheck)

36Weak
How it's scored
0/18One-command bootstrap
22/22Automated tests
0/11Lint / format config
0/11Static type checking
10/10Reproducible environment — devcontainer, Dockerfile
0/10Demonstrated agent practice — no agent-authored commits among the last 100
0/8Automated maintenance — no automated dependency updates observed
0/10OpenSSF Scorecard: Pinned-Dependencies — no data
Inputs used
has_nixno
has_testsyes
lockfiles—
has_dockerfileyes
typed_languageno
bootstrap_files—
has_devcontaineryes
has_linter_configno
typecheck_configs—
agent_commit_share0
toolchain_manifests—
dependency_bot_commit_share0

Code legibility for models

55Moderate
How it's scored
0/45Type-checkable code — Python without a type-check config
55/55Manageable file sizes — 0/812 source files over 60KB
Inputs used
primary_languagePython
largest_source_bytes47,038
source_files_sampled812
oversized_source_files0

Machine-readable interfaces

100Exceptional
How it's scored
0/40API schema (OpenAPI/GraphQL/proto) — not applicable to this kind of software
0/20MCP server — not applicable to this kind of software
40/40Runnable examples — examples, notebooks
Inputs used
example_dirsexamples, notebooks
has_mcp_signalno
api_schema_files—
interfaces_expected_of—
Excluded from scoring (no data or not applicable): API schema (OpenAPI/GraphQL/proto), MCP server. Remaining weights renormalized.

Key facts

172GitHub stars
16contributors
23commits, last 12 months
68days since last push
1releases
2bus factor
16open issues
PyPIpackage ecosystems

Data collection warnings

  • Star history unavailable: GitHub GraphQL error: Resource not accessible by personal access token
  • Could not fetch pypi package 'medperf' from its registry
  • Could not fetch pypi package 'rano-monitor' from its registry
  • Advisory severity resolved for 120 of 175 advisories (lookup cap); the remainder are reported as unknown severity
  • OpenSSF Scorecard timed out after 240s; skipping Scorecard checks

More detail

Star and fork history 0 ★ / 36 ⇿
0Stars
36Forks
1Releases

When each star and fork was added, collected from GitHub and bucketed by day. Cumulative growth sits directly above the daily additions it is made of, so the two read against each other: steady organic accretion looks nothing like an abrupt, short-lived burst. Where that difference is measurable, it is reported as growth authenticity.

01325383422022-042024-042026-05
Major 0Minor 1Patch 0

Each point covers 4 days.

All dependencies 118

Full resolved dependency set from the GitHub dependency graph: 0 direct and 118 indirect (transitive) packages. The transitive closure is complete when the repository commits a lockfile.

RegistryPackageVersionRelation
PyPIargparse—indirect
PyPIauth0-python4.3.0indirect
PyPIclick8.1.8indirect
PyPIcnb-tools0.3.2indirect
PyPIcolorama0.4.4indirect
PyPIcookiecutter2.1.1indirect
PyPIcryptography—indirect
PyPIcryptography46.0.3indirect
PyPIcurlify—indirect
PyPIdash2.16indirect
PyPIdash-bootstrap-components1.5indirect
PyPIdicom—indirect
PyPIdjango4.2.26indirect
PyPIdjango-cors-headers4.3.1indirect
PyPIdjango-environ0.11.2indirect
PyPIdjango-extensions3.2.3indirect
PyPIdjango-filter24.3indirect
PyPIdjango-storages1.14.2indirect
PyPIdjangorestframework3.14.0indirect
PyPIdjangorestframework-simplejwt5.3.1indirect
PyPIdocker7.1.0indirect
PyPIdrf-spectacular0.27.1indirect
PyPIdrf-spectacular-sidecar2024.3.4indirect
PyPIemail-validator2.0.0indirect
PyPIfastapi0.111.1indirect
PyPIfastapi-login1.10.2indirect
PyPIgitpython3.1.41indirect
PyPIgoogle-api-python-client2.192.0indirect
PyPIgoogle-auth—indirect
PyPIgoogle-auth2.49.1indirect
PyPIgoogle-cloud-compute1.46.0indirect
PyPIgoogle-cloud-iam2.21.0indirect
PyPIgoogle-cloud-kms—indirect
PyPIgoogle-cloud-kms3.11.0indirect
PyPIgoogle-cloud-secret-manager2.19.0indirect
PyPIgoogle-cloud-storage—indirect
PyPIgoogle-cloud-storage3.9.0indirect
PyPIgriffe0.32.3indirect
PyPIgunicorn23.0.0indirect
PyPImedpy—indirect
PyPImerge-args0.1.5indirect
PyPImkdocs1.4.3indirect
PyPImkdocs-autorefs0.4.1indirect
PyPImkdocs-gen-files0.5.0indirect
PyPImkdocs-literate-nav0.6.0indirect
PyPImkdocs-macros-plugin1.0.2indirect
PyPImkdocs-material9.1.18indirect
PyPImkdocs-material-extensions1.1.1indirect
PyPImkdocstrings0.22.0indirect
PyPImkdocstrings-python1.2.0indirect
PyPInibabel—indirect
PyPInumpy—indirect
PyPInumpy1.24.3indirect
PyPInumpy1.26.0indirect
PyPInumpy1.26.4indirect
PyPInumpy2.4.3indirect
PyPIonnx1.13.0indirect
PyPIonnx1.16.2indirect
PyPIpandas—indirect
PyPIpandas2.1.0indirect
PyPIpandas2.2.1indirect
PyPIparameterized0.9.0indirect
PyPIpexpect4.8.0indirect
PyPIpillow—indirect
PyPIpillow10.2.0indirect
PyPIpillow9.5.0indirect
PyPIplotly5.20indirect
PyPIpsutil5.9.8indirect
PyPIpsycopg2-binary2.9.9indirect
PyPIpydantic1.10.13indirect
PyPIpyfakefs5.8.0indirect
PyPIpygments2.18indirect
PyPIpyopenssl25.3.0indirect
PyPIpyperclip1.8.2indirect
PyPIpytest-dependency0.6.0indirect
PyPIpytest-mock3.14.0indirect
PyPIpyyaml—indirect
PyPIpyyaml6.0indirect
PyPIpyyaml6.0.1indirect
PyPIpyyaml6.0.3indirect
PyPIrequests—indirect
PyPIrich—indirect
PyPIschema0.7.5indirect
PyPIscikit-learn—indirect
PyPIscikit-learn0.24.2indirect
PyPIscikit-learn1.2.2indirect
PyPIscikit-learn1.8.0indirect
PyPIscipy—indirect
PyPIselenium4.23.1indirect
PyPIsemver3.0.4indirect
PyPIsimpleitk—indirect
PyPIsynapseclient4.1.1indirect
PyPItabulate0.9.0indirect
PyPItblib3.0.0indirect
PyPItensorflow2.12.0indirect
PyPItextual0.46.0indirect
PyPItime-machine2.4.0indirect
PyPItorch—indirect
PyPItorch2.0.1indirect
PyPItorch2.8.0indirect
PyPItorchvision—indirect
PyPItorchvision0.15.2indirect
PyPItorchxrayvision—indirect
PyPItorchxrayvision0.0.39indirect
PyPItqdm—indirect
PyPItqdm4.65.0indirect
PyPItqdm4.67.3indirect
PyPItyper—indirect
PyPItyper0.20.1indirect
PyPItyper0.9.0indirect
PyPItyping-extensions—indirect
PyPIuvicorn0.30.1indirect
PyPIvalidators0.18.2indirect
PyPIwatchdog3.0.0indirect
PyPIwebdriver-manager4.0.2indirect
PyPIwerkzeug3.0.6indirect
PyPIwhitenoise6.6.0indirect
PyPIyaspin2.1.0indirect
Dependency advisories 19

This repository publishes no package the index resolves, so its own dependency graph was assessed — 94 packages, which also include development and test pins that never ship: 19 carry known advisories, of which 0 are direct. 24 could not be assessed — no resolved version, an unsupported ecosystem, or beyond the reported package list.

PackageVersionRelationSeverityAdvisoriesFixed in
cryptography46.0.3indirectcritical1350.0.0
django4.2.26indirectcritical406.1.1
gitpython3.1.41indirectcritical593.1.62
onnx1.13.0indirectcritical241.22.0
onnx1.16.2indirectcritical181.22.0
pillow10.2.0indirectcritical3012.3.0
pillow9.5.0indirectcritical3712.3.0
djangorestframework3.14.0indirectmoderate63.17.2
mkdocs-material9.1.18indirectmoderate29.7.7
scikit-learn0.24.2indirectmoderate31.5.0
scikit-learn1.2.2indirectmoderate21.5.0
werkzeug3.0.6indirectmoderate83.1.9
djangorestframework-simplejwt5.3.1indirectlow25.5.1
pygments2.18indirectlow22.20.0
pyopenssl25.3.0indirectunknown4—
tensorflow2.12.0indirectunknown2—
torch2.0.1indirectunknown28—
torch2.8.0indirectunknown8—
tqdm4.65.0indirectunknown2—

An advisory means the version recorded in the dependency graph falls inside an advisory’s affected range. Reachability is not analysed, and the graph includes development and test pins — a finding may concern tooling rather than shipped software.

Raw JSON report machine-readable

Feedback

Spotted something off in this report, or have thoughts to share? Wrong measurements, missed tooling, ideas, questions — anything is welcome. Every message is read and gets a response.

The message is kept through sign-in.

Related records

Inspected repositories sharing catalogue tags or ecosystems with mlcommons/medperf. Read them side by side →

100wandb/wandbThe AI developer platform. Use Weights & Biases to train and fine-tune models, and manage models from experimentation to production.Exceptional100deepset-ai/haystackOpen-source AI orchestration framework for building context-engineered, production-ready LLM applications. Design modular pipelines and agent workflows with explicit control over retrieval, routing, memory, and generation. Built for scalable agents, RAG, multimodal applications, semantic search, and conversational systems.Exceptional100mlco2/codecarbonTrack emissions from Compute and recommend ways to reduce their impact on the environment.Exceptional100psf/blackThe uncompromising Python code formatterExceptional100numpy/numpyThe fundamental package for scientific computing with Python.Exceptional100optuna/optunaA hyperparameter optimization frameworkExceptional

Scores are signals, not warranties. They reflect publicly visible practices on GitHub — not a code audit, and not a security guarantee.

Missing data is excluded and weights renormalized, never scored as zero. Methodology is versioned and open: metrics v2.10.0, schema v0.34.0 — full methodology · metrics wiki.

How one result sits in the wider record: aggregate statistics.