bytedance / xpu-perf
[HPCA 2026] AI Accelerator Benchmark focuses on evaluating AI Accelerators from a practical production perspective, including the ease of use and versatility of software and hardware.
bytedance/xpu-perf holds a health index of 50 out of 100, placing it in the Moderate band. It scores highest on Sustainability & Governance (69/100) and lowest on Security (16/100). It was last updated 145 days ago. 2 contributors account for most of its recent work.
Software health index
Metrics are grouped into weighted categories on one standardized 1–100 scale. Overall starts as their weighted mean, calibrated against the distribution of the public record so bands carry percentile meaning; when public evidence triggers the High-Risk Jurisdiction Policy, the rating is adjusted and receives an At Risk ceiling of 34.
Score profile
Each axis is a category. The shape matters more than the average — a healthy subject fills the whole shape, while a spike-and-crater profile means strength in one dimension is masking risk in another.
The weighted overall 50 is calibrated to 50 on the published index scale (record calibration 2026-08-02).
Ownership
This repository is backed by an organization — shared, accountable stewardship that can outlive any single maintainer.
Metrics by category
Vitality
Is the project alive — is code being written and are releases shipping?
Development activity
How it's scored
| 9.9/36 | Push recency — last push 145 days ago |
| 4.2/36 | Commit cadence — 6/52 weeks with commits |
| 12.2/18 | Commit volume — 22 commits in the last year |
| 0/10 | OpenSSF Scorecard: Maintained — no data |
Inputs used
| commits_last_year | 22 |
| human_commit_share | 1 |
| days_since_last_push | 145 |
| active_weeks_last_year | 6 |
Release discipline
How it's scored
| 27/27 | Ships releases — 1 releases published |
| 0/36 | Release recency — latest release 749 days ago |
| 12.6/27 | Release cadence — cadence unknown (single release) |
| 0/10 | OpenSSF Scorecard: Signed-Releases — no data |
Inputs used
| releases_count | 1 |
| latest_release_tag | v1.0.0 |
| releases_from_tags | no |
| days_since_latest_release | 749 |
| mean_days_between_releases | — |
Community & Adoption
Does the project have users, downloads, attention, and a welcoming setup for contributors?
Popularity & adoption
How it's scored
| 41.9/60 | Stars — 383 stars |
| 17.6/25 | Forks — 130 forks |
| 5.8/15 | Watchers — 12 watchers |
Inputs used
| forks | 130 |
| stars | 383 |
| watchers | 12 |
| growth_state | unverified |
| growth_factor_pct | 100 |
| growth_unverified_reason | no_history |
Community health
How it's scored
| 22.5/22.5 | README |
| 22.5/22.5 | License — recognized license (Apache-2.0) |
| 18/18 | CONTRIBUTING guide |
| 0/13.5 | Code of conduct |
| 0/7.2 | Issue template |
| 0/6.3 | PR template |
Inputs used
| has_readme | yes |
| has_license | yes |
| readme_badges | 0 |
| has_contributing | yes |
| has_issue_template | no |
| has_code_of_conduct | no |
| readme_badge_services | — |
| has_pull_request_template | no |
Sustainability & Governance
Will the project survive its people — bus factor, responsiveness, who backs it, and package upkeep?
Maintainer resilience (bus factor)
How it's scored
| 25.2/54 | Bus factor — 2 contributor(s) cover half of all commits |
| 13.9/22.5 | Commit distribution — top contributor authored 38% of commits |
| 13.5/13.5 | Contributor breadth — 15 contributors |
| 0/10 | OpenSSF Scorecard: Contributors — no data |
Inputs used
| bus_factor | 2 |
| contributors_sampled | 15 |
| top_contributor_share | 0.382 |
How it's scored
| 27.8/42 | Issue resolution — 66% of issues closed |
| 23.4/30 | PR acceptance — 88/113 decided PRs merged |
| 0/13 | Newcomer PR acceptance — no first-time contributor's PR decided in 30d |
| 0/15 | OpenSSF Scorecard: Code-Review — no data |
Inputs used
| merged_prs | 88 |
| open_issues | 26 |
| closed_issues | 51 |
| prs_merged_7d | 0 |
| prs_decided_7d | 0 |
| prs_merged_30d | 0 |
| prs_decided_30d | 0 |
| issue_closed_ratio | 0.662 |
| closed_unmerged_prs | 25 |
| first_time_authors_30d | 0 |
| first_time_prs_merged_30d | 0 |
| first_time_prs_decided_30d | 0 |
Ownership & stewardship
How it's scored
| 30/30 | Ownership backing — organization-owned |
| 0/20 | Verified domain |
| 25/25 | Owner reach — 19,001 followers of bytedance |
| 25/25 | Track record — 418 public repos, account ~13 yr old |
Inputs used
| followers | 19,001 |
| owner_type | Organization |
| is_verified | no |
| owner_login | bytedance |
| public_repos | 418 |
| account_age_days | 4,900 |
Engineering Quality
Are baseline engineering and documentation practices in place?
Engineering practices
How it's scored
| 0/24 | CI workflows |
| 24/24 | Tests present |
| 16/16 | Linter config — .flake8, .pylintrc |
| 0/9.6 | Pre-commit hooks |
| 0/6.4 | .editorconfig |
| 0/20 | OpenSSF Scorecard: CI-Tests — no data |
Inputs used
| has_ci | no |
| has_tests | yes |
| has_editorconfig | no |
| has_linter_config | yes |
| has_precommit_config | no |
Documentation
How it's scored
| 30/30 | README |
| 0/25 | Documentation directory |
| 15/15 | Documentation / homepage site — https://xpu-perf.ai/ |
| 10/10 | Repository description |
| 10/10 | Topics — 1 topics |
| 0/10 | Wiki |
Inputs used
| topics | python |
| has_wiki | no |
| homepage | https://xpu-perf.ai/ |
| docs_site | https://xpu-perf.ai/ |
| has_readme | yes |
| has_docs_dir | no |
| has_description | yes |
Security
Are visible security and supply-chain practices strong, without unresolved high-risk jurisdiction exposure?
Security posture
How it's scored
| 0/30 | Security policy (SECURITY.md) |
| 0/25 | Dependabot config |
| 0/25 | Dependency lockfiles |
| 0/20 | CodeQL workflow |
Inputs used
| source | file_signals |
| lockfiles | — |
| manifests | pyproject.toml |
| has_codeql_workflow | no |
| has_security_policy | no |
| has_dependabot_config | no |
Dependency advisories
How it's scored
| 20.2/35 | Direct dependencies free of known advisories — 1 affected: numpy 1.19.2 (moderate 5.3) |
| 0/25 | Indirect dependencies free of known advisories — transitive set not separable from development and test dependencies in this scope |
| 35.3/40 | No advisories left outstanding — 1 advisory-carrying package(s) unaddressed past 90 days; oldest published 1,731 days ago |
Inputs used
| source | osv |
| advisories | 1,859 |
| affected_packages | 18 |
| assessed_packages | 38 |
| unassessed_packages | 52 |
| affected_by_severity | critical 12, high 5, moderate 1 |
| direct_affected_packages | 1 |
AI Readiness
How well is the repo equipped to be developed and maintained with AI coding agents? Carries a deliberately small weight (4%): agent tooling is a real maintenance signal, but a repository with none can still reach 100/100.
Agent context & guidance
How it's scored
| 0/45 | Agent instructions — no CLAUDE.md / AGENTS.md / editor rules |
| 0/15 | Machine-readable docs (llms.txt) |
| 19.7/40 | Legible commit history — 37 of 100 human commits state their intent (structured subject or explanatory body) |
Inputs used
| has_llms_txt | no |
| llms_txt_url | — |
| legible_history_share | 0.37 |
| agent_instruction_files | — |
| agent_instruction_max_bytes | — |
Verify loop (build / test / typecheck)
How it's scored
| 18/18 | One-command bootstrap — projects/train_perf/Megatron-LM/megatron/core/datasets/Makefile |
| 22/22 | Automated tests |
| 11/11 | Lint / format config — .flake8, .pylintrc |
| 11/11 | Static type checking — projects/train_perf/Megatron-LM/mypy.ini |
| 10/10 | Reproducible environment — Dockerfile |
| 0/10 | Demonstrated agent practice — no agent-authored commits among the last 100 |
| 0/8 | Automated maintenance — no automated dependency updates observed |
| 0/10 | OpenSSF Scorecard: Pinned-Dependencies — no data |
Inputs used
| has_nix | no |
| has_tests | yes |
| lockfiles | — |
| has_dockerfile | yes |
| typed_language | no |
| bootstrap_files | projects/train_perf/Megatron-LM/megatron/core/datasets/Makefile |
| has_devcontainer | no |
| has_linter_config | yes |
| typecheck_configs | projects/train_perf/Megatron-LM/mypy.ini |
| agent_commit_share | 0 |
| toolchain_manifests | — |
| dependency_bot_commit_share | 0 |
Code legibility for models
How it's scored
| 27/45 | Type-checkable code — Python with type-check config (projects/train_perf/Megatron-LM/mypy.ini) |
| 54.4/55 | Manageable file sizes — 8/757 source files over 60KB |
Inputs used
| primary_language | Python |
| largest_source_bytes | 132,791 |
| source_files_sampled | 757 |
| oversized_source_files | 8 |
Machine-readable interfaces
How it's scored
| 40/40 | API schema (OpenAPI/GraphQL/proto) — projects/infer_perf/llm_perf/server/server.proto |
| 0/20 | MCP server |
| 40/40 | Runnable examples — examples, recipes |
Inputs used
| example_dirs | examples, recipes |
| has_mcp_signal | no |
| api_schema_files | projects/infer_perf/llm_perf/server/server.proto |
| interfaces_expected_of | network-service |
Key facts
Data collection warnings
- Star history unavailable: GitHub GraphQL error: Resource not accessible by personal access token
- Could not fetch pypi package 'xpu-perf' from its registry
- Advisory severity resolved for 120 of 573 advisories (lookup cap); the remainder are reported as unknown severity
- OpenSSF Scorecard did not return a usable result (killed by SIGKILL — most likely the container's memory limit; 2026/09/14 18:46:43 Warning: PATs stored in env variables GITHUB_AUTH_TOKEN and GITHUB_TOKEN differ. Scorecard will use the former.); skipping Scorecard checks
More detail
Star and fork history 0 ★ / 130 ⇿
When each star and fork was added, collected from GitHub and bucketed by day. Cumulative growth sits directly above the daily additions it is made of, so the two read against each other: steady organic accretion looks nothing like an abrupt, short-lived burst. Where that difference is measurable, it is reported as growth authenticity.
Each point covers 4 days.
Direct dependencies 5
| Registry | Package | Version constraint | Manifest |
|---|---|---|---|
| PyPI | prettytable | — | pyproject.toml |
| PyPI | jsonlines | — | pyproject.toml |
| PyPI | flask | — | pyproject.toml |
| PyPI | requests | — | pyproject.toml |
| PyPI | numpy | <=1.26.4 | pyproject.toml |
All dependencies 90
Full resolved dependency set from the GitHub dependency graph: 8 direct and 82 indirect (transitive) packages. The transitive closure is complete when the repository commits a lockfile.
| Registry | Package | Version | Relation |
|---|---|---|---|
| PyPI | flask | — | direct |
| PyPI | jsonlines | — | direct |
| PyPI | numpy | — | direct |
| PyPI | numpy | 1.19.2 | direct |
| PyPI | numpy | 1.23.0 | direct |
| PyPI | numpy | 1.23.4 | direct |
| PyPI | prettytable | — | direct |
| PyPI | requests | — | direct |
| PyPI | accelerate | — | indirect |
| PyPI | attrs | 21.2.0 | indirect |
| PyPI | backoff | — | indirect |
| PyPI | bert | — | indirect |
| PyPI | bert-tensorflow | 1.0.1 | indirect |
| PyPI | black | 23.1.0 | indirect |
| PyPI | decorator | — | indirect |
| PyPI | einops | — | indirect |
| PyPI | flask-restful | — | indirect |
| PyPI | fpdf | — | indirect |
| PyPI | google-api-python-client | — | indirect |
| PyPI | google-cloud-core | 2.3.2 | indirect |
| PyPI | graphviz | 0.8.4 | indirect |
| PyPI | grpcio | — | indirect |
| PyPI | isort | — | indirect |
| PyPI | keras2onnx | — | indirect |
| PyPI | matplotlib | — | indirect |
| PyPI | matplotlib | 3.3.4 | indirect |
| PyPI | matplotlib | 3.4.2 | indirect |
| PyPI | multipledispatch | — | indirect |
| PyPI | mypy-protobuf | 2.8 | indirect |
| PyPI | nltk | — | indirect |
| PyPI | nvidia-modelopt | — | indirect |
| PyPI | onnx | — | indirect |
| PyPI | onnx | 1.12.0 | indirect |
| PyPI | onnx | 1.13.1 | indirect |
| PyPI | onnxconverter-common | 1.12.2 | indirect |
| PyPI | onnxruntime | — | indirect |
| PyPI | onnxruntime | 1.12.0 | indirect |
| PyPI | opencv-python | — | indirect |
| PyPI | opencv-python-headless | — | indirect |
| PyPI | packaging | — | indirect |
| PyPI | pandas | — | indirect |
| PyPI | pandas | 1.3.3 | indirect |
| PyPI | pandas | 1.5.1 | indirect |
| PyPI | prompt-toolkit | — | indirect |
| PyPI | protobuf | — | indirect |
| PyPI | protobuf | 3.19.4 | indirect |
| PyPI | protobuf | 3.19.6 | indirect |
| PyPI | protobuf | 3.20.2 | indirect |
| PyPI | psutil | — | indirect |
| PyPI | pulp | — | indirect |
| PyPI | pybind11 | — | indirect |
| PyPI | pydot | — | indirect |
| PyPI | pytest | — | indirect |
| PyPI | pytest-cov | — | indirect |
| PyPI | pytest-mock | — | indirect |
| PyPI | pytest-random-order | — | indirect |
| PyPI | pyyaml | — | indirect |
| PyPI | scikit-learn | — | indirect |
| PyPI | scipy | — | indirect |
| PyPI | sentencepiece | — | indirect |
| PyPI | sentencepiece | 0.1.96 | indirect |
| PyPI | setuptools | 67.5 | indirect |
| PyPI | synr | 0.4 | indirect |
| PyPI | tensorflow | — | indirect |
| PyPI | tensorflow | 1.15.0 | indirect |
| PyPI | tensorflow | 2.4.0 | indirect |
| PyPI | tensorflow | 2.7.2 | indirect |
| PyPI | tensorflow | 2.8.4 | indirect |
| PyPI | tensorflow-datasets | 4.7.0 | indirect |
| PyPI | tensorstore | — | indirect |
| PyPI | tf2onnx | — | indirect |
| PyPI | tiktoken | — | indirect |
| PyPI | tokenization | — | indirect |
| PyPI | torch | — | indirect |
| PyPI | torch | 1.12.1 | indirect |
| PyPI | torch | 1.13.1 | indirect |
| PyPI | torch | 1.9.1 | indirect |
| PyPI | torchvision | — | indirect |
| PyPI | torchvision | 0.10.1 | indirect |
| PyPI | torchvision | 0.13.1 | indirect |
| PyPI | tqdm | — | indirect |
| PyPI | transformers | — | indirect |
| PyPI | transformers | 4.40.0 | indirect |
| PyPI | triton | 2.1.0 | indirect |
| PyPI | typing-extensions | 3.7.4.3 | indirect |
| PyPI | virtualenv | 16.7.9 | indirect |
| PyPI | wandb | — | indirect |
| PyPI | wrapt | — | indirect |
| PyPI | wrapt | 1.14.1 | indirect |
| PyPI | zarr | — | indirect |
Dependency advisories 18
This repository publishes no package the index resolves, so its own dependency graph was assessed — 38 packages, which also include development and test pins that never ship: 18 carry known advisories, of which 1 are direct. 52 could not be assessed — no resolved version, an unsupported ecosystem, or beyond the reported package list.
| Package | Version | Relation | Severity | Advisories | Fixed in |
|---|---|---|---|---|---|
| black | 23.1.0 | indirect | critical | 4 | 26.3.1 |
| onnx | 1.12.0 | indirect | critical | 26 | 1.22.0 |
| onnx | 1.13.1 | indirect | critical | 24 | 1.22.0 |
| tensorflow | 1.15.0 | indirect | critical | 745 | 2.12.1 |
| tensorflow | 2.4.0 | indirect | critical | 760 | 2.12.1 |
| tensorflow | 2.7.2 | indirect | critical | 94 | 2.12.1 |
| tensorflow | 2.8.4 | indirect | critical | 46 | 2.12.1 |
| torch | 1.12.1 | indirect | critical | 30 | 2.13.0 |
| torch | 1.13.1 | indirect | critical | 28 | 2.13.0 |
| torch | 1.9.1 | indirect | critical | 30 | 2.13.0 |
| transformers | 4.40.0 | indirect | critical | 44 | 5.10.0 |
| virtualenv | 16.7.9 | indirect | critical | 4 | 20.36.1 |
| protobuf | 3.19.4 | indirect | high | 6 | 6.33.5 |
| protobuf | 3.19.6 | indirect | high | 4 | 6.33.5 |
| protobuf | 3.20.2 | indirect | high | 4 | 6.33.5 |
| sentencepiece | 0.1.96 | indirect | high | 2 | 0.2.1 |
| setuptools | 67.5 | indirect | high | 6 | 83.0.0 |
| numpy | 1.19.2 | direct | moderate | 2 | 1.22 |
An advisory means the version recorded in the dependency graph falls inside an advisory’s affected range. Reachability is not analysed, and the graph includes development and test pins — a finding may concern tooling rather than shipped software.
Raw JSON report machine-readable
Feedback
Spotted something off in this report, or have thoughts to share? Wrong measurements, missed tooling, ideas, questions — anything is welcome. Every message is read and gets a response.
Scores are signals, not warranties. They reflect publicly visible practices on GitHub — not a code audit, and not a security guarantee.
Missing data is excluded and weights renormalized, never scored as zero. Methodology is versioned and open: metrics v2.10.0, schema v0.34.0 — full methodology · metrics wiki.
How one result sits in the wider record: aggregate statistics.