Public record
Software health reportschema 0.34.0 · metrics 2.10.0 · 2026-09-29 13:59 UTC

sbintuitions / JMTEB

The evaluation scripts of JMTEB (Japanese Massive Text Embedding Benchmark)

PythonCC-BY-SA-4.0★ 93 stars⑂ 22 forkssince Mar 2024View on GitHub ↗

sbintuitions/JMTEB holds a health index of 50 out of 100, placing it in the Moderate band. It scores highest on Engineering Quality (80/100) and lowest on Security (37/100). It was last updated 60 days ago. A single contributor accounts for most of its recent work.

50
overall / 100
Moderate

Software health index

Metrics are grouped into weighted categories on one standardized 1–100 scale. Overall starts as their weighted mean, calibrated against the distribution of the public record so bands carry percentile meaning; when public evidence triggers the High-Risk Jurisdiction Policy, the rating is adjusted and receives an At Risk ceiling of 34.

50
Exceptional93-100The record's top tier (≈ top 5%); essentially all checked criteria met
Excellent80-92Strong across the board; minor gaps
Good65-79Healthy; gaps are limited and manageable
Moderate50-64Acceptable with notable gaps; review recommended
Weak35-49Material weaknesses across several areas
At Risk20-34Significant weaknesses; adoption warrants caution
Critical1-19Severe problems (abandoned, single-maintainer, no hygiene)
VitalityCommunity &AdoptionSustainability &GovernanceEngineeringQualitySecurityAI Readiness

Score profile

Each axis is a category. The shape matters more than the average — a healthy subject fills the whole shape, while a spike-and-crater profile means strength in one dimension is masking risk in another.

The weighted overall 56 is calibrated to 59 on the published index scale (record calibration 2026-08-02).

Ownership

sbintuitionsOrganization
48 followers17 public repossince Oct 2023

This repository is backed by an organization — shared, accountable stewardship that can outlive any single maintainer.

Metrics by category

Vitality

Is the project alive — is code being written and are releases shipping?

56Moderate · 21% of overall
How it's scored
18/36Push recency — last push 60 days ago
5.5/36Commit cadence — 8/52 weeks with commits
14.4/18Commit volume — 39 commits in the last year
0/10OpenSSF Scorecard: Maintained — 0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0
Inputs used
commits_last_year39
human_commit_share1
days_since_last_push60
active_weeks_last_year8
How it's scored
27/27Ships releases — 5 releases published
36/36Release recency — latest release 60 days ago
12.6/27Release cadence — a release every ~152.8 days
0/10OpenSSF Scorecard: Signed-Releases — no data
Inputs used
releases_count5
latest_release_tagv1.4.1
releases_from_tagsno
days_since_latest_release60
mean_days_between_releases152.8
Excluded from scoring (no data or not applicable): OpenSSF Scorecard: Signed-Releases. Remaining weights renormalized.

Community & Adoption

Does the project have users, downloads, attention, and a welcoming setup for contributors?

51Moderate · 17% of overall
How it's scored
31.9/60Stars — 93 stars
11/25Forks — 22 forks
2.7/15Watchers — 4 watchers
Inputs used
forks22
stars93
watchers4
growth_stateunverified
growth_factor_pct100
growth_unverified_reasonno_history
How it's scored
22.5/22.5README
22.5/22.5License — recognized license (CC-BY-SA-4.0)
0/18CONTRIBUTING guide
0/13.5Code of conduct
0/7.2Issue template
6.3/6.3PR template
Inputs used
has_readmeyes
has_licenseyes
readme_badges0
has_contributingno
has_issue_templateno
has_code_of_conductno
readme_badge_services—
has_pull_request_templateyes

Sustainability & Governance

Will the project survive its people — bus factor, responsiveness, who backs it, and package upkeep?

54Moderate · 23% of overall
How it's scored
9/54Bus factor — 1 contributor(s) cover half of all commits
5.5/22.5Commit distribution — top contributor authored 75% of commits
9.5/13.5Contributor breadth — 7 contributors
6/10OpenSSF Scorecard: Contributors — project has 2 contributing companies or organizations -- score normalized to 6
Inputs used
bus_factor1
contributors_sampled7
top_contributor_share0.754
How it's scored
29.6/42Issue resolution — 70% of issues closed
28.8/30PR acceptance — 69/72 decided PRs merged
0/13Newcomer PR acceptance — no first-time contributor's PR decided in 30d
12/15OpenSSF Scorecard: Code-Review — Found 7/8 approved changesets -- score normalized to 8
Inputs used
merged_prs69
open_issues8
closed_issues19
prs_merged_7d0
prs_decided_7d0
prs_merged_30d0
prs_decided_30d0
issue_closed_ratio0.704
closed_unmerged_prs3
first_time_authors_30d0
first_time_prs_merged_30d0
first_time_prs_decided_30d0
Excluded from scoring (no data or not applicable): Newcomer PR acceptance. Remaining weights renormalized.
How it's scored
30/30Ownership backing — organization-owned
0/20Verified domain
12.2/25Owner reach — 48 followers of sbintuitions
15/25Track record — 17 public repos, account ~2 yr old
Inputs used
followers48
owner_typeOrganization
is_verifiedno
owner_loginsbintuitions
public_repos17
account_age_days1,077

Engineering Quality

Are baseline engineering and documentation practices in place?

80Excellent · 19% of overall
How it's scored
24/24CI workflows — 1 workflow(s)
24/24Tests present
16/16Linter config — .flake8, pyproject.toml ([tool.black], [tool.isort])
0/9.6Pre-commit hooks
0/6.4.editorconfig
20/20OpenSSF Scorecard: CI-Tests — 7 out of 7 merged PRs checked by a CI test -- score normalized to 10
Inputs used
has_ciyes
has_testsyes
has_editorconfigno
has_linter_configyes
has_precommit_configno
How it's scored
30/30README
25/25Documentation directory
0/15Documentation / homepage site
10/10Repository description
0/10Topics
10/10Wiki
Inputs used
topics—
has_wikiyes
homepage—
docs_site—
has_readmeyes
has_docs_diryes
has_descriptionyes

Security

Are visible security and supply-chain practices strong, without unresolved high-risk jurisdiction exposure?

37Weak · 16% of overall
How it's scored
7.5/7.5Binary-Artifacts — no binaries found in the repo
0/7.5Branch-Protection — no data
2.5/2.5CI-Tests — 7 out of 7 merged PRs checked by a CI test -- score normalized to 10
0/2.5CII-Best-Practices — no effort to earn an OpenSSF best practices badge detected
6/7.5Code-Review — Found 7/8 approved changesets -- score normalized to 8
1.5/2.5Contributors — project has 2 contributing companies or organizations -- score normalized to 6
10/10Dangerous-Workflow — no dangerous workflow patterns detected
0/7.5Dependency-Update-Tool — no update tool detected
0/5Fuzzing — project is not fuzzed
2.5/2.5License — license file detected
0/7.5Maintained — 0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0
0/5Packaging — no data
0/5Pinned-Dependencies — dependency not pinned by hash detected -- score normalized to 0
0/5SAST — SAST tool is not run on all commits -- score normalized to 0
0/5Security-Policy — security policy file not detected
0/7.5Signed-Releases — no data
0/7.5Token-Permissions — detected GitHub workflow tokens with excessive permissions
0/7.5Vulnerabilities — 93 existing vulnerabilities detected
Inputs used
sourceopenssf_scorecard
checks_evaluated15
scorecard_versionv5.5.0
checks_inconclusive3
scorecard_aggregate3.5
Excluded from scoring (no data or not applicable): Branch-Protection, Packaging, Signed-Releases. Remaining weights renormalized.
How it's scored
5.5/35Direct dependencies free of known advisories — 4 affected: transformers 4.57.1 (critical 9.6), accelerate 0.31.0 (high 7.1), torch 2.9.0 (high 8.8), +1 more
0/25Indirect dependencies free of known advisories — transitive set not separable from development and test dependencies in this scope
27.4/40No advisories left outstanding — 2 advisory-carrying package(s) unaddressed past 90 days; oldest published 546 days ago
Inputs used
sourceosv
advisories172
affected_packages21
assessed_packages125
unassessed_packages0
affected_by_severitycritical 6, high 8, moderate 5, low 1, unknown 1
direct_affected_packages4
Excluded from scoring (no data or not applicable): Indirect dependencies free of known advisories. Remaining weights renormalized. Matched 125 resolved dependencies against OSV. This repository publishes no package the index resolves, so the repository dependency graph was assessed instead. That graph mixes development and test pins with shipped dependencies, so only the declared runtime dependencies are scored; transitive findings are reported as context and excluded from the score. Reachability is not analyzed.

AI Readiness

How well is the repo equipped to be developed and maintained with AI coding agents? Carries a deliberately small weight (4%): agent tooling is a real maintenance signal, but a repository with none can still reach 100/100.

46Weak · 4% of overall
How it's scored
0/45Agent instructions — no CLAUDE.md / AGENTS.md / editor rules
0/15Machine-readable docs (llms.txt)
17.6/40Legible commit history — 33 of 100 human commits state their intent (structured subject or explanatory body)
Inputs used
has_llms_txtno
llms_txt_url—
legible_history_share0.33
agent_instruction_files—
agent_instruction_max_bytes—
How it's scored
0/18One-command bootstrap
22/22Automated tests
11/11Lint / format config — .flake8, pyproject.toml ([tool.black], [tool.isort])
0/11Static type checking
10/10Reproducible environment — Dockerfile, lockfile
0/10Demonstrated agent practice — no agent-authored commits among the last 100
0/8Automated maintenance — no automated dependency updates observed
0/10OpenSSF Scorecard: Pinned-Dependencies — dependency not pinned by hash detected -- score normalized to 0
Inputs used
has_nixno
has_testsyes
lockfilespoetry.lock
has_dockerfileyes
typed_languageno
bootstrap_files—
has_devcontainerno
has_linter_configyes
typecheck_configs—
agent_commit_share0
toolchain_manifests—
dependency_bot_commit_share0
How it's scored
0/45Type-checkable code — Python without a type-check config
55/55Manageable file sizes — 0/64 source files over 60KB
Inputs used
primary_languagePython
largest_source_bytes17,845
source_files_sampled64
oversized_source_files0
How it's scored
0/40API schema (OpenAPI/GraphQL/proto) — not applicable to this kind of software
0/20MCP server — not applicable to this kind of software
40/40Runnable examples — examples
Inputs used
example_dirsexamples
has_mcp_signalno
api_schema_files—
interfaces_expected_of—
Excluded from scoring (no data or not applicable): API schema (OpenAPI/GraphQL/proto), MCP server. Remaining weights renormalized.

Key facts

93GitHub stars
7contributors
39commits, last 12 months
60days since last push
5releases
1bus factor
8open issues
PyPIpackage ecosystems

Data collection warnings

  • Star history unavailable: GitHub GraphQL error: Resource not accessible by personal access token
  • Could not fetch pypi package 'JMTEB' from its registry

More detail

Star and fork history 0 ★ / 22 ⇿
0Stars
22Forks
5Releases

When each star and fork was added, collected from GitHub and bucketed by day. Cumulative growth sits directly above the daily additions it is made of, so the two read against each other: steady organic accretion looks nothing like an abrupt, short-lived burst. Where that difference is measurable, it is reported as growth authenticity.

048121620242212024-032025-062026-08
Major 1Minor 1Patch 2

Each point covers 3 days.

OpenSSF Scorecard 3.5 / 10
3.5aggregate

Independent, tool-agnostic security assessment from the open-source OpenSSF Scorecard. Each check rewards a security practice, not a specific vendor's tool. Checks Scorecard could not determine are marked n/a and excluded from the security score (never counted as zero).Scorecard v5.5.0 · 2026-09-29 13:58 UTC

10Binary-Artifactsno binaries found in the repo
n/aBranch-Protectioninternal error: error during branchesHandler.setup: internal error: some github tokens can't read classic branch protection rules: https://github.com/ossf/scorecard-action/blob/main/docs/authentication/fine-grained-auth-token.md
10CI-Tests7 out of 7 merged PRs checked by a CI test -- score normalized to 10
0CII-Best-Practicesno effort to earn an OpenSSF best practices badge detected
8Code-ReviewFound 7/8 approved changesets -- score normalized to 8
6Contributorsproject has 2 contributing companies or organizations -- score normalized to 6
10Dangerous-Workflowno dangerous workflow patterns detected
0Dependency-Update-Toolno update tool detected
0Fuzzingproject is not fuzzed
10Licenselicense file detected
0Maintained0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0
n/aPackagingpackaging workflow not detected
0Pinned-Dependenciesdependency not pinned by hash detected -- score normalized to 0
0SASTSAST tool is not run on all commits -- score normalized to 0
0Security-Policysecurity policy file not detected
n/aSigned-Releasesno releases found
0Token-Permissionsdetected GitHub workflow tokens with excessive permissions
0Vulnerabilities93 existing vulnerabilities detected
Direct dependencies 19
RegistryPackageVersion constraintManifest
PyPImteb>=2.4.2pyproject.toml
PyPIsentence-transformers>=5.1.1pyproject.toml
PyPItorch>=2.6.0pyproject.toml
PyPInumpy>=1.26,<3.0.0pyproject.toml
PyPIpyyaml^6.0pyproject.toml
PyPIloguru>=0.7.3pyproject.toml
PyPIscikit-learn>=1.4.0pyproject.toml
PyPItransformers>=4.57.1pyproject.toml
PyPIaccelerate>=0.31.0pyproject.toml
PyPItqdm>=4.0.0pyproject.toml
PyPIrich>=13.0.0pyproject.toml
PyPIopenai>=1.41.0pyproject.toml
PyPItiktoken>=0.8.0pyproject.toml
PyPIdatasets>=2.19.0pyproject.toml
PyPIjsonargparse>=4.36.0pyproject.toml
PyPIpydantic>=2.6.0pyproject.toml
PyPIeval-type-backport^0.1.3pyproject.toml
PyPIsmart-open>=7.0.0pyproject.toml
PyPItabulate>=0.9.0pyproject.toml
All dependencies 125

Full resolved dependency set from the GitHub dependency graph: 19 direct and 106 indirect (transitive) packages. The transitive closure is complete when the repository commits a lockfile.

RegistryPackageVersionRelation
PyPIaccelerate0.31.0direct
PyPIdatasets2.19.2direct
PyPIeval-type-backport0.1.3direct
PyPIjsonargparse4.44.0direct
PyPIloguru0.7.3direct
PyPImteb2.4.2direct
PyPInumpy1.26.4direct
PyPIopenai2.14.0direct
PyPIpydantic2.7.3direct
PyPIpyyaml6.0.1direct
PyPIrich14.2.0direct
PyPIscikit-learn1.5.0direct
PyPIsentence-transformers5.1.1direct
PyPIsmart-open7.0.4direct
PyPItabulate0.9.0direct
PyPItiktoken0.12.0direct
PyPItorch2.9.0direct
PyPItqdm4.66.4direct
PyPItransformers4.57.1direct
PyPIaiohttp3.9.5indirect
PyPIaiosignal1.3.1indirect
PyPIannotated-types0.7.0indirect
PyPIanyio4.4.0indirect
PyPIasync-timeout4.0.3indirect
PyPIattrs23.2.0indirect
PyPIblack23.12.1indirect
PyPIcertifi2024.6.2indirect
PyPIcharset-normalizer3.3.2indirect
PyPIclick8.1.7indirect
PyPIcolorama0.4.6indirect
PyPIdill0.3.8indirect
PyPIdistro1.9.0indirect
PyPIexceptiongroup1.2.1indirect
PyPIfilelock3.14.0indirect
PyPIflake87.0.0indirect
PyPIfrozenlist1.4.1indirect
PyPIfsspec2024.3.1indirect
PyPIfugashi1.5.2indirect
PyPIh110.14.0indirect
PyPIhf-xet1.2.0indirect
PyPIhttpcore1.0.5indirect
PyPIhttpx0.27.0indirect
PyPIhuggingface-hub0.36.0indirect
PyPIidna3.7indirect
PyPIiniconfig2.0.0indirect
PyPIipadic1.0.0indirect
PyPIisort5.13.2indirect
PyPIjinja23.1.4indirect
PyPIjiter0.12.0indirect
PyPIjoblib1.4.2indirect
PyPIjsonnet0.21.0indirect
PyPImarkdown-it-py4.0.0indirect
PyPImarkupsafe2.1.5indirect
PyPImccabe0.7.0indirect
PyPImdurl0.1.2indirect
PyPImpmath1.3.0indirect
PyPImultidict6.0.5indirect
PyPImultiprocess0.70.16indirect
PyPImypy1.10.0indirect
PyPImypy-extensions1.0.0indirect
PyPInetworkx3.3indirect
PyPInvidia-cublas-cu1212.8.4.1indirect
PyPInvidia-cuda-cupti-cu1212.8.90indirect
PyPInvidia-cuda-nvrtc-cu1212.8.93indirect
PyPInvidia-cuda-runtime-cu1212.8.90indirect
PyPInvidia-cudnn-cu129.10.2.21indirect
PyPInvidia-cufft-cu1211.3.3.83indirect
PyPInvidia-cufile-cu121.13.1.3indirect
PyPInvidia-curand-cu1210.3.9.90indirect
PyPInvidia-cusolver-cu1211.7.3.90indirect
PyPInvidia-cusparse-cu1212.5.8.93indirect
PyPInvidia-cusparselt-cu120.7.1indirect
PyPInvidia-nccl-cu122.27.5indirect
PyPInvidia-nvjitlink-cu1212.8.93indirect
PyPInvidia-nvshmem-cu123.3.20indirect
PyPInvidia-nvtx-cu1212.8.90indirect
PyPIpackaging24.0indirect
PyPIpandas2.2.2indirect
PyPIpathspec0.12.1indirect
PyPIpillow10.3.0indirect
PyPIplac1.4.5indirect
PyPIplatformdirs4.2.2indirect
PyPIpluggy1.5.0indirect
PyPIpolars1.36.1indirect
PyPIpolars-runtime-321.36.1indirect
PyPIprotobuf6.33.0indirect
PyPIpsutil6.0.0indirect
PyPIpy1.11.0indirect
PyPIpyarrow16.1.0indirect
PyPIpyarrow-hotfix0.6indirect
PyPIpycodestyle2.11.1indirect
PyPIpydantic-core2.18.4indirect
PyPIpyflakes3.2.0indirect
PyPIpygments2.19.2indirect
PyPIpytest7.1.3indirect
PyPIpytest-mock3.14.0indirect
PyPIpython-dateutil2.9.0.post0indirect
PyPIpytrec-eval-terrier0.5.10indirect
PyPIpytz2024.1indirect
PyPIregex2024.5.15indirect
PyPIrequests2.32.3indirect
PyPIrhoknp1.3.0indirect
PyPIsafetensors0.4.3indirect
PyPIscipy1.13.1indirect
PyPIsentencepiece0.2.1indirect
PyPIsetuptools80.9.0indirect
PyPIsix1.16.0indirect
PyPIsniffio1.3.1indirect
PyPIsudachidict-core20251022indirect
PyPIsudachipy0.6.10indirect
PyPIsympy1.14.0indirect
PyPIthreadpoolctl3.5.0indirect
PyPItokenizers0.22.1indirect
PyPItomli2.0.1indirect
PyPItriton3.5.0indirect
PyPItyping-extensions4.12.1indirect
PyPItzdata2024.1indirect
PyPIunidic1.1.0indirect
PyPIunidic-lite1.0.8indirect
PyPIurllib32.2.1indirect
PyPIwasabi0.10.1indirect
PyPIwin32-setctime1.1.0indirect
PyPIwrapt1.16.0indirect
PyPIxxhash3.4.1indirect
PyPIyarl1.9.4indirect
Dependency advisories 21

This repository publishes no package the index resolves, so its own dependency graph was assessed — 125 packages, which also include development and test pins that never ship: 21 carry known advisories, of which 4 are direct.

PackageVersionRelationSeverityAdvisoriesFixed in
transformers4.57.1directcritical105.10.0
aiohttp3.9.5indirectcritical683.14.3
anyio4.4.0indirectcritical24.14.2
black23.12.1indirectcritical426.3.1
h110.14.0indirectcritical20.16.0
pillow10.3.0indirectcritical3412.3.0
accelerate0.31.0directhigh2—
torch2.9.0directhigh52.13.0
certifi2024.6.2indirecthigh22024.7.4
click8.1.7indirecthigh18.3.3
jinja23.1.4indirecthigh63.1.6
protobuf6.33.0indirecthigh26.33.5
pyarrow16.1.0indirecthigh323.0.1
urllib32.2.1indirecthigh142.7.0
filelock3.14.0indirectmoderate43.20.3
idna3.7indirectmoderate23.15
pytest7.1.3indirectmoderate29.0.3
requests2.32.3indirectmoderate42.33.0
setuptools80.9.0indirectmoderate283.0.0
pygments2.19.2indirectlow22.20.0
datasets2.19.2directunknown15.0.1

An advisory means the version recorded in the dependency graph falls inside an advisory’s affected range. Reachability is not analysed, and the graph includes development and test pins — a finding may concern tooling rather than shipped software.

Raw JSON report machine-readable

Feedback

Spotted something off in this report, or have thoughts to share? Wrong measurements, missed tooling, ideas, questions — anything is welcome. Every message is read and gets a response.

The message is kept through sign-in.

Scores are signals, not warranties. They reflect publicly visible practices on GitHub — not a code audit, and not a security guarantee.

Missing data is excluded and weights renormalized, never scored as zero. Methodology is versioned and open: metrics v2.10.0, schema v0.34.0 — full methodology · metrics wiki.

How one result sits in the wider record: aggregate statistics.