Public record
Software health reportschema 0.34.0 · metrics 2.10.0 · 2026-08-22 17:24 UTC

cpllab / syntaxgym-core

DEPRECATED: Command-line tool and Python API for targeted syntactic evaluation of language models

PythonMIT★ 12 stars⑂ 1 forksince Jun 2020archivedView on GitHub ↗

cpllab/syntaxgym-core holds a health index of 10 out of 100, placing it in the Critical band. It scores highest on AI Readiness (48/100) and lowest on Vitality (20/100). The repository is archived, so no further maintenance is expected.

10
overall / 100
Critical

Software health index

Metrics are grouped into weighted categories on one standardized 1–100 scale. Overall starts as their weighted mean, calibrated against the distribution of the public record so bands carry percentile meaning; when public evidence triggers the High-Risk Jurisdiction Policy, the rating is adjusted and receives an At Risk ceiling of 34.

10
Exceptional93-100The record's top tier (≈ top 5%); essentially all checked criteria met
Excellent80-92Strong across the board; minor gaps
Good65-79Healthy; gaps are limited and manageable
Moderate50-64Acceptable with notable gaps; review recommended
Weak35-49Material weaknesses across several areas
At Risk20-34Significant weaknesses; adoption warrants caution
Critical1-19Severe problems (abandoned, single-maintainer, no hygiene)
VitalityCommunity &AdoptionSustainability &GovernanceEngineeringQualitySecurityAI Readiness

Score profile

Each axis is a category. The shape matters more than the average — a healthy subject fills the whole shape, while a spike-and-crater profile means strength in one dimension is masking risk in another.

The weighted overall 32 is calibrated to 26 on the published index scale (record calibration 2026-08-02).

Ownership

MIT Computational Psycholinguistics LabOrganization
11 followers11 public repossince Sep 2015

This repository is backed by an organization — shared, accountable stewardship that can outlive any single maintainer.

Package ecosystems

RegistryPackageVersionDownloads / moVersionsLast publishTags
PyPIsyntaxgympoints to another repo — not scored0.6.043171753 days agolanguage-modelsnlpai

Metrics by category

Vitality

Is the project alive — is code being written and are releases shipping?

20At Risk · 21% of overall

Development activity

1Critical
How it's scored
0/36Push recency — last push 921 days ago
0/36Commit cadence — 0/52 weeks with commits
0/18Commit volume — 0 commits in the last year
0/10OpenSSF Scorecard: Maintained — project is archived
Inputs used
commits_last_year0
human_commit_share1
days_since_last_push921
active_weeks_last_year0

Release discipline

48Weak
How it's scored
16.2/27Ships releases — 4 version tags (no GitHub releases)
0/36Release recency — latest release 1,975 days ago
27/27Release cadence — a release every ~14.3 days
0/10OpenSSF Scorecard: Signed-Releases — no data
Inputs used
releases_count4
latest_release_tag0.5.8
releases_from_tagsyes
days_since_latest_release1,975
mean_days_between_releases14.3
Excluded from scoring (no data or not applicable): OpenSSF Scorecard: Signed-Releases. Remaining weights renormalized.

Community & Adoption

Does the project have users, downloads, attention, and a welcoming setup for contributors?

33At Risk · 17% of overall

Popularity & adoption

19Critical
How it's scored
16.9/60Stars — 12 stars
0/25Forks — 1 forks
1.7/15Watchers — 3 watchers
Inputs used
forks1
stars12
watchers3
growth_stateunverified
growth_factor_pct100
growth_unverified_reasonno_history

Community health

50Moderate
How it's scored
22.5/22.5README
22.5/22.5License — recognized license (MIT)
0/18CONTRIBUTING guide
0/13.5Code of conduct
0/7.2Issue template
0/6.3PR template
Inputs used
has_readmeyes
has_licenseyes
readme_badges1
has_contributingno
has_issue_templateno
has_code_of_conductno
readme_badge_servicesshields.io
has_pull_request_templateno

Sustainability & Governance

Will the project survive its people — bus factor, responsiveness, who backs it, and package upkeep?

35Weak · 23% of overall

Maintainer resilience (bus factor)

20At Risk
How it's scored
9/54Bus factor — 1 contributor(s) cover half of all commits
3.5/22.5Commit distribution — top contributor authored 85% of commits
4.1/13.5Contributor breadth — 3 contributors
3/10OpenSSF Scorecard: Contributors — project has 1 contributing companies or organizations -- score normalized to 3
Inputs used
bus_factor1
contributors_sampled3
top_contributor_share0.846

Issue & PR responsiveness

30At Risk
How it's scored
14/42Issue resolution — 33% of issues closed
12/30PR acceptance — 2/5 decided PRs merged
0/13Newcomer PR acceptance — no first-time contributor's PR decided in 30d
0/15OpenSSF Scorecard: Code-Review — Found 0/22 approved changesets -- score normalized to 0
Inputs used
merged_prs2
open_issues4
closed_issues2
prs_merged_7d0
prs_decided_7d0
prs_merged_30d0
prs_decided_30d0
issue_closed_ratio0.333
closed_unmerged_prs3
first_time_authors_30d0
first_time_prs_merged_30d0
first_time_prs_decided_30d0
Excluded from scoring (no data or not applicable): Newcomer PR acceptance. Remaining weights renormalized.

Ownership & stewardship

58Moderate
How it's scored
30/30Ownership backing — organization-owned
0/20Verified domain
7.8/25Owner reach — 11 followers of cpllab
19.9/25Track record — 11 public repos, account ~10 yr old
Inputs used
followers11
owner_typeOrganization
is_verifiedno
owner_logincpllab
public_repos11
account_age_days3,978

Engineering Quality

Are baseline engineering and documentation practices in place?

40Weak · 19% of overall

Engineering practices

24At Risk
How it's scored
0/24CI workflows
24/24Tests present
0/16Linter config
0/9.6Pre-commit hooks
0/6.4.editorconfig
0/20OpenSSF Scorecard: CI-Tests — 0 out of 1 merged PRs checked by a CI test -- score normalized to 0
Inputs used
has_cino
has_testsyes
has_editorconfigno
has_linter_configno
has_precommit_configno

Documentation

65Good
How it's scored
30/30README
25/25Documentation directory
0/15Documentation / homepage site
10/10Repository description
0/10Topics
0/10Wiki
Inputs used
topics—
has_wikino
homepage—
docs_site—
has_readmeyes
has_docs_diryes
has_descriptionyes

Security

Are visible security and supply-chain practices strong, without unresolved high-risk jurisdiction exposure?

31At Risk · 16% of overall

Security posture

14Critical
How it's scored
7.5/7.5Binary-Artifacts — no binaries found in the repo
0/7.5Branch-Protection — branch protection not enabled on development/release branches
0/2.5CI-Tests — 0 out of 1 merged PRs checked by a CI test -- score normalized to 0
0/2.5CII-Best-Practices — no effort to earn an OpenSSF best practices badge detected
0/7.5Code-Review — Found 0/22 approved changesets -- score normalized to 0
0.8/2.5Contributors — project has 1 contributing companies or organizations -- score normalized to 3
0/10Dangerous-Workflow — no data
0/7.5Dependency-Update-Tool — no update tool detected
0/5Fuzzing — project is not fuzzed
2.5/2.5License — license file detected
0/7.5Maintained — project is archived
0/5Packaging — no data
0/5Pinned-Dependencies — dependency not pinned by hash detected -- score normalized to 0
0/5SAST — SAST tool is not run on all commits -- score normalized to 0
0/5Security-Policy — security policy file not detected
0/7.5Signed-Releases — no data
0/7.5Token-Permissions — no data
0/7.5Vulnerabilities — 49 existing vulnerabilities detected
Inputs used
sourceopenssf_scorecard
checks_evaluated14
scorecard_versionv5.5.0
checks_inconclusive4
scorecard_aggregate1.4
Excluded from scoring (no data or not applicable): Dangerous-Workflow, Packaging, Signed-Releases, Token-Permissions. Remaining weights renormalized.

Dependency advisories

100Exceptional
How it's scored
35/35Direct dependencies free of known advisories — no direct dependency carries a known advisory
0/25Indirect dependencies free of known advisories — transitive set not separable from development and test dependencies in this scope
0/40No advisories left outstanding — no advisory carries a publication date
Inputs used
sourceosv
advisories86
affected_packages15
assessed_packages92
unassessed_packages11
affected_by_severitycritical 1, high 7, moderate 6, low 1
direct_affected_packages0
Excluded from scoring (no data or not applicable): Indirect dependencies free of known advisories, No advisories left outstanding. Remaining weights renormalized. Matched 92 resolved dependencies against OSV. 11 could not be assessed — no resolved version, an unsupported ecosystem, or beyond the reported package list. This repository publishes no package the index resolves, so the repository dependency graph was assessed instead. That graph mixes development and test pins with shipped dependencies, so only the declared runtime dependencies are scored; transitive findings are reported as context and excluded from the score. Reachability is not analyzed.

AI Readiness

How well is the repo equipped to be developed and maintained with AI coding agents? Carries a deliberately small weight (4%): agent tooling is a real maintenance signal, but a repository with none can still reach 100/100.

48Weak · 4% of overall

Agent context & guidance

19Critical
How it's scored
0/45Agent instructions — no CLAUDE.md / AGENTS.md / editor rules
15/15Machine-readable docs (llms.txt) — llms.txt served by the project website (https://docs.docker.com/llms.txt)
4.3/40Legible commit history — 8 of 100 human commits state their intent (structured subject or explanatory body)
Inputs used
has_llms_txtyes
llms_txt_urlhttps://docs.docker.com/llms.txt
legible_history_share0.08
agent_instruction_files—
agent_instruction_max_bytes—

Verify loop (build / test / typecheck)

61Moderate
How it's scored
18/18One-command bootstrap — docs/Makefile
22/22Automated tests
0/11Lint / format config
11/11Static type checking — mypy.ini
10/10Reproducible environment — Dockerfile, lockfile
0/10Demonstrated agent practice — no agent-authored commits among the last 100
0/8Automated maintenance — no automated dependency updates observed
0/10OpenSSF Scorecard: Pinned-Dependencies — dependency not pinned by hash detected -- score normalized to 0
Inputs used
has_nixno
has_testsyes
lockfilesPipfile.lock
has_dockerfileyes
typed_languageno
bootstrap_filesdocs/Makefile
has_devcontainerno
has_linter_configno
typecheck_configsmypy.ini
agent_commit_share0
toolchain_manifests—
dependency_bot_commit_share0

Code legibility for models

74Good
How it's scored
27/45Type-checkable code — Python with type-check config (mypy.ini)
46.7/55Manageable file sizes — 5/33 source files over 60KB
Inputs used
primary_languagePython
largest_source_bytes287,630
source_files_sampled33
oversized_source_files5

Key facts

12GitHub stars
3contributors
0commits, last 12 months
921days since last push
4releases
1bus factor
4open issues
PyPIpackage ecosystems

Data collection warnings

  • Star history unavailable: GitHub GraphQL error: Resource not accessible by personal access token
  • pypi package 'syntaxgym' points at a different repository (https://syntaxgym.org); excluded from ecosystem scoring

More detail

OpenSSF Scorecard 1.4 / 10
1.4aggregate

Independent, tool-agnostic security assessment from the open-source OpenSSF Scorecard. Each check rewards a security practice, not a specific vendor's tool. Checks Scorecard could not determine are marked n/a and excluded from the security score (never counted as zero).Scorecard v5.5.0 · 2026-08-22 17:24 UTC

10Binary-Artifactsno binaries found in the repo
0Branch-Protectionbranch protection not enabled on development/release branches
0CI-Tests0 out of 1 merged PRs checked by a CI test -- score normalized to 0
0CII-Best-Practicesno effort to earn an OpenSSF best practices badge detected
0Code-ReviewFound 0/22 approved changesets -- score normalized to 0
3Contributorsproject has 1 contributing companies or organizations -- score normalized to 3
n/aDangerous-Workflowno workflows found
0Dependency-Update-Toolno update tool detected
0Fuzzingproject is not fuzzed
10Licenselicense file detected
0Maintainedproject is archived
n/aPackagingpackaging workflow not detected
0Pinned-Dependenciesdependency not pinned by hash detected -- score normalized to 0
0SASTSAST tool is not run on all commits -- score normalized to 0
0Security-Policysecurity policy file not detected
n/aSigned-Releasesno releases found
n/aToken-PermissionsNo tokens found
0Vulnerabilities49 existing vulnerabilities detected
All dependencies 103

Full resolved dependency set from the GitHub dependency graph: 0 direct and 103 indirect (transitive) packages. The transitive closure is complete when the repository commits a lockfile.

RegistryPackageVersionRelation
PyPIalabaster0.7.12indirect
PyPIappdirs1.4.4indirect
PyPIattrs21.2.0indirect
PyPIbabel2.9.1indirect
PyPIblack19.10b0indirect
PyPIbleach4.1.0indirect
PyPIcached-property1.5.2indirect
PyPIcerberus1.3.4indirect
PyPIcertifi2020.12.5indirect
PyPIcffi1.15.0indirect
PyPIchardet4.0.0indirect
PyPIcharset-normalizer2.0.9indirect
PyPIclick—indirect
PyPIclick7.1.2indirect
PyPIcolorama0.4.4indirect
PyPIcoverage6.2indirect
PyPIcrayons0.4.0indirect
PyPIcryptography36.0.1indirect
PyPIdecopatch1.4.8indirect
PyPIdistlib0.3.4indirect
PyPIdocker—indirect
PyPIdocker4.4.4indirect
PyPIdocutils0.17.1indirect
PyPIh5py—indirect
PyPIh5py3.2.1indirect
PyPIidna2.10indirect
PyPIimagesize1.3.0indirect
PyPIimportlib-metadata4.8.3indirect
PyPIimportlib-resources5.4.0indirect
PyPIiniconfig1.1.1indirect
PyPIjeepney0.7.1indirect
PyPIjinja23.0.3indirect
PyPIjsonpointer2.2indirect
PyPIjsonschema4.3.0indirect
PyPIkeyring23.4.0indirect
PyPIlm-zoo—indirect
PyPIlm-zoo1.3indirect
PyPImakefun1.12.1indirect
PyPImarkupsafe2.0.1indirect
PyPInumpy—indirect
PyPInumpy1.20.1indirect
PyPIorderedmultidict1.0.1indirect
PyPIpackaging20.9indirect
PyPIpandas—indirect
PyPIpandas1.2.3indirect
PyPIpathspec0.9.0indirect
PyPIpep5170.12.0indirect
PyPIpip-shims0.6.0indirect
PyPIpipenv-setup3.1.4indirect
PyPIpipfile0.0.2indirect
PyPIpkginfo1.8.2indirect
PyPIplatformdirs2.4.0indirect
PyPIplette0.2.3indirect
PyPIpluggy1.0.0indirect
PyPIpy1.11.0indirect
PyPIpycparser2.21indirect
PyPIpygments2.10.0indirect
PyPIpyparsing—indirect
PyPIpyparsing3.0.6indirect
PyPIpyrsistent0.18.0indirect
PyPIpytest6.2.5indirect
PyPIpytest-cases3.6.5indirect
PyPIpython-dateutil2.8.1indirect
PyPIpytz2021.1indirect
PyPIpyyaml6.0indirect
PyPIreadme-renderer32.0indirect
PyPIregex2021.11.10indirect
PyPIrequests—indirect
PyPIrequests2.25.1indirect
PyPIrequests-toolbelt0.9.1indirect
PyPIrequirementslib1.6.1indirect
PyPIrfc39861.5.0indirect
PyPIsecretstorage3.3.1indirect
PyPIsemver2.13.0indirect
PyPIsix1.15.0indirect
PyPIsnowballstemmer2.2.0indirect
PyPIsphinx4.3.1indirect
PyPIsphinx-click3.0.2indirect
PyPIsphinx-jsonschema1.17.1indirect
PyPIsphinx-rtd-theme1.0.0indirect
PyPIsphinxcontrib-applehelp1.0.2indirect
PyPIsphinxcontrib-devhelp1.0.2indirect
PyPIsphinxcontrib-htmlhelp2.0.0indirect
PyPIsphinxcontrib-jsmath1.0.1indirect
PyPIsphinxcontrib-qthelp1.0.3indirect
PyPIsphinxcontrib-serializinghtml1.1.5indirect
PyPIspython—indirect
PyPIspython0.1.11indirect
PyPItoml0.10.2indirect
PyPItomli2.0.0indirect
PyPItomlkit0.7.2indirect
PyPItqdm—indirect
PyPItqdm4.59.0indirect
PyPItwine3.7.1indirect
PyPItyped-ast1.5.1indirect
PyPItyping-extensions4.0.1indirect
PyPIurllib3—indirect
PyPIurllib31.26.7indirect
PyPIvistir0.5.2indirect
PyPIwebencodings0.5.1indirect
PyPIwebsocket-client0.58.0indirect
PyPIwheel0.37.0indirect
PyPIzipp3.6.0indirect
Dependency advisories 15

This repository publishes no package the index resolves, so its own dependency graph was assessed — 92 packages, which also include development and test pins that never ship: 15 carry known advisories, of which 0 are direct. 11 could not be assessed — no resolved version, an unsupported ecosystem, or beyond the reported package list.

PackageVersionRelationSeverityAdvisoriesFixed in
black19.10b0indirectcritical426.3.1
certifi2020.12.5indirecthigh52024.7.4
click7.1.2indirecthigh18.3.3
cryptography36.0.1indirecthigh22300.0.12
idna2.10indirecthigh43.15
jinja23.0.3indirecthigh103.1.6
urllib31.26.7indirecthigh162.7.0
wheel0.37.0indirecthigh20.38.1
bleach4.1.0indirectmoderate26.4.0
numpy1.20.1indirectmoderate21.22
pygments2.10.0indirectmoderate42.20.0
pytest6.2.5indirectmoderate29.0.3
requests2.25.1indirectmoderate82.33.0
zipp3.6.0indirectmoderate23.19.1
tqdm4.59.0indirectlow24.66.3

An advisory means the version recorded in the dependency graph falls inside an advisory’s affected range. Reachability is not analysed, and the graph includes development and test pins — a finding may concern tooling rather than shipped software.

Raw JSON report machine-readable

Feedback

Spotted something off in this report, or have thoughts to share? Wrong measurements, missed tooling, ideas, questions — anything is welcome. Every message is read and gets a response.

The message is kept through sign-in.

Related records

Inspected repositories sharing catalogue tags or ecosystems with cpllab/syntaxgym-core. Read them side by side →

100apache/arrowApache Arrow is the universal columnar format and multi-language toolbox for fast data interchange and in-memory analyticsExceptional100onnx/onnxOpen standard for machine learning interoperabilityExceptional100reflex-dev/reflex🕸️ Web apps in pure Python 🐍Exceptional100scikit-learn/scikit-learnscikit-learn: machine learning in PythonExceptional100psf/blackThe uncompromising Python code formatterExceptional100uxlfoundation/oneDNNoneAPI Deep Neural Network Library (oneDNN)Exceptional

Scores are signals, not warranties. They reflect publicly visible practices on GitHub — not a code audit, and not a security guarantee.

Missing data is excluded and weights renormalized, never scored as zero. Methodology is versioned and open: metrics v2.10.0, schema v0.34.0 — full methodology · metrics wiki.

How one result sits in the wider record: aggregate statistics — PyPI.