Public recordGood
Software health reportschema 0.27.0 · metrics 1.13.0 · 2026-07-25 08:58 UTC
SWE-bench / SWE-smith
[NeurIPS 2025 D&B Spotlight] Scaling Data for SWE-agents
SWE-bench/SWE-smith holds a health index of 72 out of 100, placing it in the Good band. It scores highest on Engineering Quality (95/100) and lowest on Security (49/100). It was last updated 4 days ago. A single contributor accounts for most of its recent work.
72
overall / 100Software health index
Metrics are grouped into weighted categories on one standardized 1–100 scale. Overall starts as their weighted mean; when public evidence triggers the High-Risk Jurisdiction Policy, the rating is adjusted and receives an At risk ceiling of 49. AI Readiness sits outside the overall score.
72
Excellent85-100Exemplary; meets essentially all checked criteria
Good70-84Healthy; minor gaps
Moderate50-69Acceptable with notable gaps; review recommended
At risk30-49Significant weaknesses; adoption warrants caution
Critical1-29Severe problems (abandoned, single-maintainer, no hygiene)
Score profile
Each axis is a category. The shape matters more than the average — a healthy subject fills the whole shape, while a spike-and-crater profile means strength in one dimension is masking risk in another.
Ownership
SWE-benchOrganization
307 followers9 public repossince Jul 2023
This repository is backed by an organization — shared, accountable stewardship that can outlive any single maintainer.
Package ecosystems
| Registry | Package | Version | Downloads / mo | Versions | Last publish | Tags |
|---|---|---|---|---|---|---|
| PyPI | swesmith | 0.0.9 | 5,197,566 | 10 | 148 days ago | nlpbenchmarkcode |
Metrics by category
Vitality
Is the project alive — is code being written and are releases shipping?
64
Development activity
69Moderate
How it's scored
| 36/36 | Push recency — last push 4 days ago |
| 15.9/36 | Commit cadence — 23/52 weeks with commits |
| 17.3/18 | Commit volume — 83 commits in the last year |
| 0/10 | OpenSSF Scorecard: Maintained — 0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0 |
Inputs used
| commits_last_year | 83 |
| human_commit_share | 0.82 |
| days_since_last_push | 4 |
| active_weeks_last_year | 23 |
Release discipline
56Moderate
How it's scored
| 16.2/27 | Ships releases — 4 version tags (no GitHub releases) |
| 7.2/36 | Release recency — latest release 368 days ago |
| 27/27 | Release cadence — a release every ~26.9 days |
| 0/10 | OpenSSF Scorecard: Signed-Releases — no data |
Inputs used
| releases_count | 4 |
| latest_release_tag | v0.0.6 |
| releases_from_tags | yes |
| days_since_latest_release | 368 |
| mean_days_between_releases | 26.9 |
Excluded from scoring (no data or not applicable): OpenSSF Scorecard: Signed-Releases. Remaining weights renormalized.
Community & Adoption
Does the project have users, downloads, attention, and a welcoming setup for contributors?
82
Popularity & adoption
68Moderate
How it's scored
| 46.2/60 | Stars — 710 stars |
| 17.5/25 | Forks — 126 forks |
| 4.3/15 | Watchers — 7 watchers |
Inputs used
| forks | 126 |
| stars | 710 |
| watchers | 7 |
| growth_state | unverified |
| growth_factor_pct | 100 |
| growth_unverified_reason | no_history |
Community health
85Excellent
How it's scored
| 22.5/22.5 | README |
| 22.5/22.5 | License — recognized license (MIT) |
| 18/18 | CONTRIBUTING guide |
| 13.5/13.5 | Code of conduct |
| 0/7.2 | Issue template |
| 0/6.3 | PR template |
Inputs used
| has_readme | yes |
| has_license | yes |
| has_contributing | yes |
| has_issue_template | no |
| has_code_of_conduct | yes |
| has_pull_request_template | no |
Ecosystem adoption (downloads)
100Excellent
How it's scored
| 80/80 | Monthly downloads — 5,197,566 downloads/month across pypi |
| 0/20 | Registry dependents — not reported by this ecosystem |
Inputs used
| packages | swesmith |
| dependents | — |
| ecosystems | pypi |
| total_downloads | — |
| monthly_downloads | 5,197,566 |
Excluded from scoring (no data or not applicable): Registry dependents. Remaining weights renormalized.
Sustainability & Governance
Will the project survive its people — bus factor, responsiveness, who backs it, and package upkeep?
69
Maintainer resilience (bus factor)
43At risk
How it's scored
| 9/54 | Bus factor — 1 contributor(s) cover half of all commits |
| 10.9/22.5 | Commit distribution — top contributor authored 51% of commits |
| 13.5/13.5 | Contributor breadth — 18 contributors |
| 10/10 | OpenSSF Scorecard: Contributors — project has 26 contributing companies or organizations |
Inputs used
| bus_factor | 1 |
| contributors_sampled | 18 |
| top_contributor_share | 0.514 |
How it's scored
| 40/46.8 | Issue resolution — 86% of issues closed |
| 32.9/38.3 | PR acceptance — 123/143 decided PRs merged |
| 10.5/15 | OpenSSF Scorecard: Code-Review — Found 19/25 approved changesets -- score normalized to 7 |
Inputs used
| merged_prs | 123 |
| open_issues | 13 |
| closed_issues | 77 |
| issue_closed_ratio | 0.856 |
| closed_unmerged_prs | 20 |
Ownership & stewardship
61Moderate
How it's scored
| 30/30 | Ownership backing — organization-owned |
| 0/20 | Verified domain |
| 17.9/25 | Owner reach — 307 followers of SWE-bench |
| 13.3/25 | Track record — 9 public repos, account ~3 yr old |
Inputs used
| followers | 307 |
| owner_type | Organization |
| is_verified | — |
| owner_login | SWE-bench |
| public_repos | 9 |
| account_age_days | 1,106 |
Package maintenance
100Excellent
How it's scored
| 25/25 | Published & resolvable — 1 package(s) on pypi |
| 35/35 | Publish recency — latest publish 148 days ago |
| 20/20 | Version history — 10 published versions |
| 20/20 | Not deprecated — active, not deprecated or yanked |
Inputs used
| packages | swesmith |
| ecosystems | pypi |
| any_deprecated | no |
| min_days_since_publish | 148 |
Engineering Quality
Are baseline engineering and documentation practices in place?
95
Engineering practices
92Excellent
How it's scored
| 24/24 | CI workflows — 3 workflow(s) |
| 24/24 | Tests present |
| 16/16 | Linter config |
| 9.6/9.6 | Pre-commit hooks |
| 0/6.4 | .editorconfig |
| 18/20 | OpenSSF Scorecard: CI-Tests — 25 out of 26 merged PRs checked by a CI test -- score normalized to 9 |
Inputs used
| has_ci | yes |
| has_tests | yes |
| has_editorconfig | no |
| has_linter_config | yes |
| has_precommit_config | yes |
Documentation
100Excellent
How it's scored
| 30/30 | README |
| 25/25 | Documentation directory |
| 15/15 | Documentation / homepage site — https://swesmith.com/ |
| 10/10 | Repository description |
| 10/10 | Topics — 4 topics |
| 10/10 | Wiki |
Inputs used
| topics | agents, language-model, software-engineering, training |
| has_wiki | yes |
| homepage | https://swesmith.com/ |
| has_readme | yes |
| has_docs_dir | yes |
| has_description | yes |
Security
Are visible security and supply-chain practices strong, without unresolved high-risk jurisdiction exposure?
49
Security posture
49At risk
How it's scored
| 7.5/7.5 | Binary-Artifacts — no binaries found in the repo |
| 0/7.5 | Branch-Protection — branch protection not enabled on development/release branches |
| 2.2/2.5 | CI-Tests — 25 out of 26 merged PRs checked by a CI test -- score normalized to 9 |
| 0/2.5 | CII-Best-Practices — no effort to earn an OpenSSF best practices badge detected |
| 5.2/7.5 | Code-Review — Found 19/25 approved changesets -- score normalized to 7 |
| 2.5/2.5 | Contributors — project has 26 contributing companies or organizations |
| 10/10 | Dangerous-Workflow — no dangerous workflow patterns detected |
| 7.5/7.5 | Dependency-Update-Tool — update tool detected |
| 0/5 | Fuzzing — project is not fuzzed |
| 2.5/2.5 | License — license file detected |
| 0/7.5 | Maintained — 0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0 |
| 0/5 | Packaging — no data |
| 0/5 | Pinned-Dependencies — dependency not pinned by hash detected -- score normalized to 0 |
| 0/5 | SAST — SAST tool is not run on all commits -- score normalized to 0 |
| 0/5 | Security-Policy — security policy file not detected |
| 0/7.5 | Signed-Releases — no data |
| 0/7.5 | Token-Permissions — detected GitHub workflow tokens with excessive permissions |
| 7.5/7.5 | Vulnerabilities — 0 existing vulnerabilities detected |
Inputs used
| source | openssf_scorecard |
| checks_evaluated | 16 |
| scorecard_version | v5.5.0 |
| checks_inconclusive | 2 |
| scorecard_aggregate | 4.9 |
Excluded from scoring (no data or not applicable): packaging, signed_releases. Remaining weights renormalized.
AI Readiness
How well is the repo equipped to be developed and maintained with AI coding agents? An independent, experimental badge — weight 0.0, so it is surfaced on its own and does not affect the overall health score.
51
Agent context & guidance
50Moderate
How it's scored
| 18/45 | Agent instructions — AGENTS.md (stub) |
| 0/15 | Machine-readable docs (llms.txt) |
| 32.5/40 | Legible commit history — 50 of 82 human commits state their intent (structured subject or explanatory body) |
Inputs used
| has_llms_txt | no |
| legible_history_share | 0.61 |
| agent_instruction_files | AGENTS.md |
| agent_instruction_max_bytes | 194 |
Verify loop (build / test / typecheck)
51Moderate
How it's scored
| 0/18 | One-command bootstrap |
| 22/22 | Automated tests |
| 11/11 | Lint / format config |
| 0/11 | Static type checking |
| 0/10 | Reproducible environment |
| 10/10 | Demonstrated agent practice — 5 of the last 100 commits agent-authored or agent-credited |
| 8/8 | Automated maintenance — 6 of the last 100 commits are automated dependency updates |
| 0/10 | OpenSSF Scorecard: Pinned-Dependencies — dependency not pinned by hash detected -- score normalized to 0 |
Inputs used
| has_nix | no |
| has_tests | yes |
| lockfiles | — |
| has_dockerfile | no |
| typed_language | no |
| bootstrap_files | — |
| has_devcontainer | no |
| has_linter_config | yes |
| typecheck_configs | — |
| agent_commit_share | 0.05 |
| toolchain_manifests | — |
| dependency_bot_commit_share | 0.06 |
Code legibility for models
54Moderate
How it's scored
| 0/45 | Type-checkable code — Python without a type-check config |
| 53.8/55 | Manageable file sizes — 5/223 source files over 60KB |
Inputs used
| primary_language | Python |
| largest_source_bytes | 276,830 |
| source_files_sampled | 223 |
| oversized_source_files | 5 |
Key facts
710GitHub stars
18contributors
83commits, last 12 months
4days since last push
4releases
1bus factor
13open issues
PyPIpackage ecosystems
Data collection warnings
- Star history unavailable: GitHub GraphQL error: Resource not accessible by personal access token
More detail
Star and fork history 0 ★ / 126 ⇿
0Stars
126Forks
3Releases
When each star and fork was added, collected from GitHub and bucketed by day. Cumulative growth sits directly above the daily additions it is made of, so the two read against each other: steady organic accretion looks nothing like an abrupt, short-lived burst. Where that difference is measurable, it is reported as growth authenticity.
Major 0Minor 0Patch 3
Each point covers 2 days.
OpenSSF Scorecard 4.9 / 10
4.9aggregate
Independent, tool-agnostic security assessment from the open-source OpenSSF Scorecard. Each check rewards a security practice, not a specific vendor's tool. Checks Scorecard could not determine are marked n/a and excluded from the security score (never counted as zero).
| 10 | Binary-Artifacts | no binaries found in the repo |
| 0 | Branch-Protection | branch protection not enabled on development/release branches |
| 9 | CI-Tests | 25 out of 26 merged PRs checked by a CI test -- score normalized to 9 |
| 0 | CII-Best-Practices | no effort to earn an OpenSSF best practices badge detected |
| 7 | Code-Review | Found 19/25 approved changesets -- score normalized to 7 |
| 10 | Contributors | project has 26 contributing companies or organizations |
| 10 | Dangerous-Workflow | no dangerous workflow patterns detected |
| 10 | Dependency-Update-Tool | update tool detected |
| 0 | Fuzzing | project is not fuzzed |
| 10 | License | license file detected |
| 0 | Maintained | 0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0 |
| n/a | Packaging | packaging workflow not detected |
| 0 | Pinned-Dependencies | dependency not pinned by hash detected -- score normalized to 0 |
| 0 | SAST | SAST tool is not run on all commits -- score normalized to 0 |
| 0 | Security-Policy | security policy file not detected |
| n/a | Signed-Releases | no releases found |
| 0 | Token-Permissions | detected GitHub workflow tokens with excessive permissions |
| 10 | Vulnerabilities | 0 existing vulnerabilities detected |
All dependencies 0
Full resolved dependency set from the GitHub dependency graph: 0 direct and 0 indirect (transitive) packages. The transitive closure is complete when the repository commits a lockfile.
| Registry | Package | Version | Relation |
|---|
Dependency advisories not assessed
Advisory matching could not run for this report: No resolved dependencies to assess
Raw JSON report machine-readable
{
"data": {
"repo": {
"topics": [
"agents",
"language-model",
"software-engineering",
"training"
],
"is_fork": false,
"size_kb": 14518,
"has_wiki": true,
"homepage": "https://swesmith.com/",
"languages": {
"C": 19273,
"C#": 29209,
"Go": 37090,
"C++": 7281,
"PHP": 2418,
"Java": 4627,
"Ruby": 8030,
"Rust": 5645,
"Shell": 8390,
"Python": 2007366,
"JavaScript": 2249,
"TypeScript": 676
},
"pushed_at": "2026-07-20T19:41:52Z",
"created_at": "2025-05-01T06:06:30Z",
"owner_type": "Organization",
"updated_at": "2026-07-23T23:53:31Z",
"description": "[NeurIPS 2025 D&B Spotlight] Scaling Data for SWE-agents",
"is_archived": false,
"is_disabled": false,
"license_spdx": "MIT",
"default_branch": "main",
"license_spdx_raw": "MIT",
"primary_language": "Python",
"significant_languages": [
"Python"
]
},
"owner": {
"blog": "https://swebench.com/",
"name": "SWE-bench",
"type": "Organization",
"login": "SWE-bench",
"company": null,
"location": null,
"followers": 307,
"avatar_url": "https://avatars.githubusercontent.com/u/139597579?v=4",
"created_at": "2023-07-14T23:52:51Z",
"is_verified": null,
"public_repos": 9,
"account_age_days": 1106
},
"license": {
"state": "standard",
"spdx_id": "MIT",
"raw_spdx": "MIT",
"file_present": true,
"scorecard_found": true,
"profile_has_license": true
},
"activity": {
"releases": [
{
"tag": "v0.0.6",
"kind": "patch",
"published_at": "2025-07-22T00:50:27Z"
},
{
"tag": "v0.0.5",
"kind": "patch",
"published_at": "2025-07-09T19:42:17Z"
},
{
"tag": "v0.0.2",
"kind": "patch",
"published_at": "2025-05-07T08:11:56Z"
},
{
"tag": "v0.0.1",
"kind": "patch",
"published_at": "2025-05-02T10:59:41Z"
}
],
"recent_commits": [
{
"oid": "9b74ac08118a85c39c356802f7961893af73e07f",
"body": "* Add PHP language support with procedural bug generation",
"is_bot": false,
"headline": "Add PHP language support (#233)",
"author_name": "Arpandeep Khatua",
"author_login": "akhatua2",
"committed_at": "2026-03-21T23:39:59Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "6dbf9621bbc7542f0e77ee511a486d4ae61f3712",
"body": "* Add 20 Ruby procedural bug modifiers with property analysis\n\ntree-sitter-based modifiers across 5 modules: control_flow (if/else\ninvert, shuffle lines, guard clause invert), nil_introduction (safe\nnavigation, or-default, presence, bang method, or-equals, nil guard),\noperations (change, flip, swap \n[…]\nench_Multilingual eval_set repos (fluentd, fastlane, fpm). RSpec\nJSON and Minitest/test-unit log parsers with auto-detection. RSpec test\npath detection (_is_test_path override for spec/ and _spec.rb).",
"is_bot": false,
"headline": "Support Ruby procedural modifications (#231)",
"author_name": "Brian Cosgrove",
"author_login": "cosgroveb",
"committed_at": "2026-03-09T21:10:16Z",
"body_truncated": true,
"is_coding_agent": false
},
{
"oid": "9f2ba9402102937f624f050c8e3c0d7c532f1e01",
"body": "* Support --gather in bug_gen_modal.py\n\n* Update --gather to store to /logs/{language}/task_insts\n\n* Only write out json if task instances is not empty\n\n* Doubled modal sandbox time out to 20 minutes to account for repos that take longer to gather\n\n* feat: parallelize gather.py and fix thread safety\n[…]\nes in modal scripts\n\n* Remove unused import in bug gen modal\n\n* Remove unused modal helper scripts\n\n---------\n\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": false,
"headline": "Parallelize task instance gather and support --gather on Modal (#199)",
"author_name": "Kevin Xiang Li",
"author_login": "AlienKevin",
"committed_at": "2026-03-09T18:39:20Z",
"body_truncated": true,
"is_coding_agent": false
},
{
"oid": "65b31a127b0a0f59627764c9926462da6728af1d",
"body": "* cpp: inline draco excludes and apply file-level entity filtering\n\n* profiles: use threading lock for RepoProfile singleton state\n\nRepoProfile previously used multiprocessing.Lock for each registered profile instance. On this machine, profile registration can hit OS semaphore limits and fail with '\n[…]\nformation, see https://pre-commit.ci\n\n---------\n\nCo-authored-by: Priyank-Shethia3 <priyank.shethia3@gmail.com>\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": false,
"headline": "C++ modifier and bug_gen_modal updates (#234)",
"author_name": "Kevin Xiang Li",
"author_login": "AlienKevin",
"committed_at": "2026-03-09T18:12:28Z",
"body_truncated": true,
"is_coding_agent": false
},
{
"oid": "c8ed9ed4c17c13b739da6102ca400adb4fefac1c",
"body": "* Implement get_test_files() for Rust profile\n\nAdd test file resolution to RustProfile so SWE-smith can strip F2P/P2P\ntest files from Rust task instances, preventing agents from cheating by\nrunning gold tests. This mirrors the existing Go implementation.\n\n- Add _extract_test_fn_name() to handle all \n[…]\nn N of M -> Class\n- Display names: pkg.Class.CoercionEnabled: true -> Class\n\nMaps all .java filenames to paths (slightly over-inclusive vs\ntest-only files, which is acceptable per project convention).",
"is_bot": false,
"headline": "Implement get_test_files() for non-Python/Go profiles (#230)",
"author_name": "Arpandeep Khatua",
"author_login": "akhatua2",
"committed_at": "2026-03-04T17:45:14Z",
"body_truncated": true,
"is_coding_agent": false
},
{
"oid": "8676d588ab1a59544d8fe3169f19915b52b59e47",
"body": "* add ~145 cpp profiles\n\nMade-with: Cursor\n\n* commented out 11\n\n* [pre-commit.ci] auto fixes from pre-commit.com hooks\n\nfor more information, see https://pre-commit.ci\n\n---------\n\nCo-authored-by: John Yang <byjohnyang@gmail.com>\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": false,
"headline": "Cpp profiles (#228)",
"author_name": "Priyank-Shethia3",
"author_login": "Priyank-Shethia3",
"committed_at": "2026-03-04T06:35:26Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "40310b7c50c6c8f60c0ea20f93e3f69286f4af0c",
"body": "updates:\n- [github.com/astral-sh/ruff-pre-commit: v0.15.2 → v0.15.4](https://github.com/astral-sh/ruff-pre-commit/compare/v0.15.2...v0.15.4)\n\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": true,
"headline": "[pre-commit.ci] pre-commit autoupdate (#229)",
"author_name": "pre-commit-ci[bot]",
"author_login": "pre-commit-ci[bot]",
"committed_at": "2026-03-03T00:53:45Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "88886208f2d207a662fb3fde56360edbb0134a0e",
"body": null,
"is_bot": false,
"headline": "version bump",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2026-03-02T18:34:27Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "c91b1497295572479b8bf0b9253e5222f9e3605a",
"body": null,
"is_bot": false,
"headline": "Comment out azahar temporarily",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2026-02-25T09:21:48Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "7f204fa9bc58f36be1db14169547c8a71dd96498",
"body": null,
"is_bot": false,
"headline": "Add `--force` to create mirror to ensure ignored files pushed as well",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2026-02-24T19:33:22Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "4d9434661d422ee7ee748eccf4f381987327e32f",
"body": "updates:\n- [github.com/astral-sh/ruff-pre-commit: v0.15.1 → v0.15.2](https://github.com/astral-sh/ruff-pre-commit/compare/v0.15.1...v0.15.2)\n\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": true,
"headline": "[pre-commit.ci] pre-commit autoupdate (#225)",
"author_name": "pre-commit-ci[bot]",
"author_login": "pre-commit-ci[bot]",
"committed_at": "2026-02-24T17:21:00Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "ac96971d4a56041c622eea3e7535f2c2afeaaba9",
"body": null,
"is_bot": false,
"headline": "replace `testbed` w/ `ENV_NAME`",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2026-02-23T19:52:14Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "576e6056256660aef3188be102fbf8eaf40393a6",
"body": "* add ts profiles, parsers and unit tests\n\n* ruff\n\n* fixes\n\n* Adjustments: [Go] Remove commit checkout; [C++] Clone mirror repo\n\n* Comment out 6 repos\n\n* Remove `git checkout`\n\n* Remove checkout commit tests\n\n---------\n\nCo-authored-by: John Yang <byjohnyang@gmail.com>",
"is_bot": false,
"headline": "Ts profiles (#224)",
"author_name": "Priyank-Shethia3",
"author_login": "Priyank-Shethia3",
"committed_at": "2026-02-23T19:40:57Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "4b9fc1c539d95be8e68721af9b5a09ff06152238",
"body": "* feat(profiles): full private repository support\n\nAdd end-to-end private repository support across all profile types.\nVisibility-aware git authentication ensures private repos get private\nmirrors, SSH-based cloning, and BuildKit SSH forwarding during image\nbuilds. Eval containers receive SSH keys v\n[…]\non, see https://pre-commit.ci\n\n* Fix Ruff unused imports\n\n---------\n\nCo-authored-by: Rb <rubenwolff@gmail.com>\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": false,
"headline": "feat: full private repository support (#220)",
"author_name": "Muhammed Karamuk",
"author_login": "mkaramuk",
"committed_at": "2026-02-17T18:33:50Z",
"body_truncated": true,
"is_coding_agent": false
},
{
"oid": "1d5495af0e3b88382d5292e2476ddc255d8a2930",
"body": "updates:\n- [github.com/astral-sh/ruff-pre-commit: v0.15.0 → v0.15.1](https://github.com/astral-sh/ruff-pre-commit/compare/v0.15.0...v0.15.1)\n\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": true,
"headline": "[pre-commit.ci] pre-commit autoupdate (#222)",
"author_name": "pre-commit-ci[bot]",
"author_login": "pre-commit-ci[bot]",
"committed_at": "2026-02-17T00:53:03Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "b084b88535e2b9085fbf667d1bd85d7b2ce7b56c",
"body": "* add cpp profiles and parsers\n\n* modify profiles to inherit from cppprofiles\n\n* bug fixes\n\n* remove problematic repo, fix parsers, remove comments\n\n* add unit tests, fix parser bugs\n\n* ruff\n\n* Minor adjustment; delay submodule init to post commit checkout\n\n---------\n\nCo-authored-by: John Yang <byjohnyang@gmail.com>",
"is_bot": false,
"headline": "Cpp profiles (#221)",
"author_name": "Priyank-Shethia3",
"author_login": "Priyank-Shethia3",
"committed_at": "2026-02-14T19:44:23Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "07baf71a6bfbd786b1fb78be48822a14edaa0a62",
"body": "* add procedural bug generation for java\n\nCo-Authored-By: Claude Opus 4.5 <noreply@anthropic.com>\nCo-authored-by: priyank <priyank.shethia3@gmail.com>",
"is_bot": false,
"headline": "Add Java procedural modifiers (#213)",
"author_name": "Kevin Xiang Li",
"author_login": "AlienKevin",
"committed_at": "2026-02-14T08:10:07Z",
"body_truncated": false,
"is_coding_agent": true
},
{
"oid": "a7ee7c7d4b1438fc890a910eae2de86d946e9434",
"body": "`generate.py` now produces a combined output file (`{stem}__ig_llm.json`) with problem_statement included, consistent with the other issue gen scripts (`get_from_tests`, `get_from_pr`, `get_static`).\n\nCo-authored-by: Muhammed Karamuk <mkaramuk@proton.me>",
"is_bot": false,
"headline": "feat: merge LLM-generated problem statements into task instances (#215)",
"author_name": "Rb",
"author_login": "reisepass",
"committed_at": "2026-02-13T01:06:14Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "9a81039ebc4215db8a9212c9e1f2bfc039b77a2d",
"body": "Prevent cleanup from deleting wrong directory when try_install_py fails mid-install inside the cloned repo.\n\nCo-authored-by: reisepass <1474408+reisepass@users.noreply.github.com>",
"is_bot": false,
"headline": "fix: cd back to base dir before cleanup (#217)",
"author_name": "Muhammed Karamuk",
"author_login": "mkaramuk",
"committed_at": "2026-02-12T22:14:13Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "b5d4fba4c3a92b8f57d84a5becea5121319b4a9f",
"body": "jinja2 is used in issue generation but not listed in the generate or all dep groups.\n\nCo-authored-by: reisepass <1474408+reisepass@users.noreply.github.com>",
"is_bot": false,
"headline": "fix: add missing jinja2 dependency (#218)",
"author_name": "Muhammed Karamuk",
"author_login": "mkaramuk",
"committed_at": "2026-02-12T17:08:55Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "4a5c4da1ce6ce5729fa65e1e50419534ce91fbe9",
"body": "Add whereis-based conda lookup as a last resort for non-standard install paths.\n\nCo-authored-by: reisepass <1474408+reisepass@users.noreply.github.com>",
"is_bot": false,
"headline": "fix(install_repo): fallback to whereis for conda discovery (#219)",
"author_name": "Muhammed Karamuk",
"author_login": "mkaramuk",
"committed_at": "2026-02-12T17:08:20Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "bab2b72ae7c12d46af6b26ebf41dec5bfc01c483",
"body": null,
"is_bot": false,
"headline": "fix(issue_gen): update stale import in get_static.py (#214)",
"author_name": "Muhammed Karamuk",
"author_login": "mkaramuk",
"committed_at": "2026-02-11T22:01:16Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "220e23d0e621f8f5171d163fd612dfc05f403498",
"body": "- Increase PREGOLD_TIMEOUT 200→500s for slower test suites\n- Force profile.arch to x86_64 for consistent Docker images\n- Use async map.aio() for parallel generation\n- Add wrap_returned_exceptions=False for Modal API\n- Fix error counting: use r.get(\"error\") instead of \"error\" in r\n\nCo-authored-by: Claude Opus 4.5 <noreply@anthropic.com>",
"is_bot": false,
"headline": "Fix bug_gen_modal.py for robust validation (#211)",
"author_name": "Kevin Xiang Li",
"author_login": "AlienKevin",
"committed_at": "2026-02-10T07:50:39Z",
"body_truncated": false,
"is_coding_agent": true
},
{
"oid": "b838cdd81dd82de59269dc39f8097d7f6e6ddca7",
"body": "* [pre-commit.ci] pre-commit autoupdate\n\nupdates:\n- [github.com/astral-sh/ruff-pre-commit: v0.14.13 → v0.15.0](https://github.com/astral-sh/ruff-pre-commit/compare/v0.14.13...v0.15.0)\n\n* [pre-commit.ci] auto fixes from pre-commit.com hooks\n\nfor more information, see https://pre-commit.ci\n\n---------\n\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": true,
"headline": "[pre-commit.ci] pre-commit autoupdate (#203)",
"author_name": "pre-commit-ci[bot]",
"author_login": "pre-commit-ci[bot]",
"committed_at": "2026-02-10T07:48:19Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "80db6ccaa25aba562d29e2e006868459e027a6b0",
"body": "* Speed up procedural bug_gen patch generation 20-40x with difflib\n\nReplace git subprocess-based patch generation with Python's difflib for\na massive speed improvement in procedural bug generation.\n\nPerformance comparison (tested on libeigen/eigen repo, 599 patches):\n- Original git-based: ~2.3 it/s \n[…]\nore information, see https://pre-commit.ci\n\n---------\n\nCo-authored-by: Claude Opus 4.5 <noreply@anthropic.com>\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": false,
"headline": "Speed up procedural bug_gen patch generation 20-40x with difflib (#212)",
"author_name": "Kevin Xiang Li",
"author_login": "AlienKevin",
"committed_at": "2026-02-10T07:46:28Z",
"body_truncated": true,
"is_coding_agent": true
},
{
"oid": "67bd602a7a31597872d74d39413ee606de703b98",
"body": "* add java profiles w/ ~90% builds verified\n\n* refactor to abstract out parsers + remove repos with empty parsers. ~100% repos build verified\n\n* update profiels to use mirror and ENV_NAME\n\n* remove git checkout\n\n* final build test. removed timed out repos\n\n* add unit test for java parsers and profil\n[…]\ncomments, move import\n\n* Minor change\n\n---------\n\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>\nCo-authored-by: Ubuntu <ubuntu@ip-172-31-20-53.ec2.internal>",
"is_bot": false,
"headline": "New java repo profiles (#208)",
"author_name": "Priyank-Shethia3",
"author_login": "Priyank-Shethia3",
"committed_at": "2026-02-10T07:40:12Z",
"body_truncated": true,
"is_coding_agent": false
},
{
"oid": "9fa8af05393010d19c615dd38322b8ae68312a28",
"body": "Co-authored-by: zhehaoli <zhehaoli@stanford.edu>\nCo-authored-by: Claude Opus 4.5 <noreply@anthropic.com>",
"is_bot": false,
"headline": "Support C++ procedural modifications (#209)",
"author_name": "Kevin Xiang Li",
"author_login": "AlienKevin",
"committed_at": "2026-02-08T23:36:04Z",
"body_truncated": false,
"is_coding_agent": true
},
{
"oid": "1ad229b39e32c246c4e2b43b1019effd09463f12",
"body": "* Added the `typescript` adapter",
"is_bot": false,
"headline": "Typescript (#204)",
"author_name": "Arpandeep Khatua",
"author_login": "akhatua2",
"committed_at": "2026-02-03T20:22:03Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "3ebf9344a947d1352cbc2d27c915732dd4923cf2",
"body": "* rust modifiers: remove entity-level flip filter\n\n* rust modifiers: set likelihoods to 0.5",
"is_bot": false,
"headline": "Boost valid Rust tasks by 6x to over 5K (#206)",
"author_name": "Kevin Xiang Li",
"author_login": "AlienKevin",
"committed_at": "2026-02-03T03:39:40Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "e603f8eaf8ce7dc91250a85609ecbca6c99fb391",
"body": "updates:\n- [github.com/astral-sh/ruff-pre-commit: v0.14.11 → v0.14.13](https://github.com/astral-sh/ruff-pre-commit/compare/v0.14.11...v0.14.13)\n\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": true,
"headline": "[pre-commit.ci] pre-commit autoupdate (#200)",
"author_name": "pre-commit-ci[bot]",
"author_login": "pre-commit-ci[bot]",
"committed_at": "2026-01-26T05:30:55Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "a36a90c3d4398f7dd9cc7f0d479cdca883bceadc",
"body": "…\" in front of f\"{self.arch}\" (#201)\n\n* Revert Architecture enum back to plain str to fix extra \"architecture.\" in front of self.arch\n\n* [pre-commit.ci] auto fixes from pre-commit.com hooks\n\nfor more information, see https://pre-commit.ci\n\n---------\n\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": false,
"headline": "Revert Architecture enum back to plain str to fix extra \"architecture…",
"author_name": "Kevin Xiang Li",
"author_login": "AlienKevin",
"committed_at": "2026-01-26T05:30:41Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "b582d651bc2699f624eb244959631c0b2137e21c",
"body": null,
"is_bot": false,
"headline": "Release 0.0.8",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2026-01-16T01:27:08Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "91e662f4c269fe07aedbcc48afcb1d652a726462",
"body": "* Refactor Docker architecture handling\n\n- Introduce Architecture enum in constants.py\n- Update RepoProfile to use Architecture enum and make pltf a dynamic property\n- Add --arch CLI argument to create_images.py to support forcing specific architectures\n- Ensure docker build command uses the correct\n[…]\ne-commit.ci] auto fixes from pre-commit.com hooks\n\nfor more information, see https://pre-commit.ci\n\n---------\n\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": false,
"headline": "Improve Docker architecture handling (#198)",
"author_name": "Kevin Xiang Li",
"author_login": "AlienKevin",
"committed_at": "2026-01-15T22:01:38Z",
"body_truncated": true,
"is_coding_agent": false
},
{
"oid": "67e1db52682ef20b3bbf80335453a2c1f6f834ba",
"body": "updates:\n- [github.com/astral-sh/ruff-pre-commit: v0.14.10 → v0.14.11](https://github.com/astral-sh/ruff-pre-commit/compare/v0.14.10...v0.14.11)\n\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": true,
"headline": "[pre-commit.ci] pre-commit autoupdate (#197)",
"author_name": "pre-commit-ci[bot]",
"author_login": "pre-commit-ci[bot]",
"committed_at": "2026-01-15T21:58:44Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "1a919bc2bcd93dbe07efeacbce3c7557fe6bc3d7",
"body": "* Lazily import adapters to simplify validator image by avoiding the 10 tree-sitter dependencies\n\n* Explicitly specify exts for c, cpp, and csharp, like the rest of the languages\n\nThis prevents bug gen from picking up source files from unrelated languages.\n\n* Make exts a mandatory field in RepoProfi\n[…]\nions. Users were surprised with bugs generated for other languages. The default is now fixed to be empty and subclasses are required to override this default value with e.g. [\".py\"] for PythonProfile.",
"is_bot": false,
"headline": "Lazily import adapters to accelerate validator image builds (#196)",
"author_name": "Kevin Xiang Li",
"author_login": "AlienKevin",
"committed_at": "2026-01-09T01:29:16Z",
"body_truncated": true,
"is_coding_agent": false
},
{
"oid": "6ace4b4bf3157c143a788ba90faa41e49e8e22c0",
"body": "…ion on Modal (#192)\n\n* Add JavaScript procedural modifications for bug generation\n\n* [pre-commit.ci] auto fixes from pre-commit.com hooks\n\nfor more information, see https://pre-commit.ci\n\n* Remove unecessary script\n\n* Adding scripts from Kevin's branch\n\n* [pre-commit.ci] auto fixes from pre-commit.\n[…]\n1e8c4d.SUNet>\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>\nCo-authored-by: akhatua2 <akhatua@stanford.edu>\nCo-authored-by: John Yang <byjohnyang@gmail.com>",
"is_bot": false,
"headline": "Add JavaScript Procedural Modifiers; Scale Bug Generation and Validat…",
"author_name": "Kevin Xiang Li",
"author_login": "AlienKevin",
"committed_at": "2026-01-08T01:03:55Z",
"body_truncated": true,
"is_coding_agent": false
},
{
"oid": "fa9be530a50f6e0fbc54a3dd840176c4c9e91b14",
"body": null,
"is_bot": false,
"headline": "Fix mkdocs links",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-12-30T18:57:54Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "75c0016227dd297638d86b5a90cebe9c910c66bc",
"body": "* Add test to fail fast if docker image missing Python dep\n\n* wip\n\n* Add documentation\n\n---------\n\nCo-authored-by: Daniel Zayas <danielzayas@Daniels-MacBook-Pro.local>\nCo-authored-by: John Yang <byjohnyang@gmail.com>",
"is_bot": false,
"headline": "support Python version > 3.10; add test for missing Python deps (#185)",
"author_name": "Daniel Zayas",
"author_login": "danielzayas",
"committed_at": "2025-12-30T18:28:20Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "b63b630fd04c7a1642b33a0a0e83a2868048ca59",
"body": "* increase max file size for diff apply\n\n* Make parameters for attempting pr mirror recovery adjustable via cli\n\n---------\n\nCo-authored-by: Daniel Zayas <danielzayas@Daniels-MacBook-Pro.local>\nCo-authored-by: John Yang <byjohnyang@gmail.com>",
"is_bot": false,
"headline": "increase max file size for diff apply (#178)",
"author_name": "Daniel Zayas",
"author_login": "danielzayas",
"committed_at": "2025-12-24T01:19:34Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "3218efa6917062e7af5750c9bc44135072ea3df3",
"body": "* Extract common build_entity function\n\n- Remove duplication by extracting _build_entity helper functions in\n tree-sitter based language adapters to a shared function.\n- Preserve existing default indent size differences between the\n adapters, I've not thought about these defaults.\n\n* Make relative import absolute\n\n---------\n\nCo-authored-by: John Yang <byjohnyang@gmail.com>",
"is_bot": false,
"headline": "Extract common build_entity function (#157)",
"author_name": "Andrew Crump",
"author_login": "acrmp",
"committed_at": "2025-12-24T00:31:52Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "2fc7a4938e84ff6bb0486529fdf076338d3b157f",
"body": "* add 44 verified JS repos and 2 new JS parsers\n\n* ruff format\n\n* minor fixes\n\n* move registration loop to EOF for js\n\n* minor fixes\n\n* removed a repo taking too long to build during testing + minor fixes\n\n* add unit test for 2 newly added JS parsers\n\n* wip fixes\n\n* nit\n\n* wip\n\n* Update handsontable\n\n---------\n\nCo-authored-by: John Yang <byjohnyang@gmail.com>",
"is_bot": false,
"headline": "Add 44 verified JS repos and 2 new JS parsers (#188)",
"author_name": "Priyank-Shethia3",
"author_login": "Priyank-Shethia3",
"committed_at": "2025-12-23T22:05:44Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "51dee5a6fb599f7c0481d2c25b98970ef5363fa4",
"body": "updates:\n- [github.com/astral-sh/ruff-pre-commit: v0.14.9 → v0.14.10](https://github.com/astral-sh/ruff-pre-commit/compare/v0.14.9...v0.14.10)\n\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": true,
"headline": "[pre-commit.ci] pre-commit autoupdate (#189)",
"author_name": "pre-commit-ci[bot]",
"author_login": "pre-commit-ci[bot]",
"committed_at": "2025-12-22T23:43:32Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "50bd78d115672b3c95c2f50e2cad40b42d2bb325",
"body": "* Added 18 JS repos\n\n* Minor fixes\n\n* Replace hard-coded commits w/ ref\n\n* Make it easier to (force re-)build for specific repo(s)\n\n* Update docs\n\n---------\n\nCo-authored-by: John Yang <byjohnyang@gmail.com>",
"is_bot": false,
"headline": "Added 18 JS repos (#186)",
"author_name": "Kevin Xiang Li",
"author_login": "AlienKevin",
"committed_at": "2025-12-20T00:03:23Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "c72e742f31cf116d6fd95ee9155b5a01ee4389e8",
"body": "* refactor: Replace manual image existence tracking with dynamic property\n\nReplace the manual _cache_image_exists flag with a @cached_property that queries\nthe Docker daemon to check image existence dynamically. This eliminates\nthe need for manual state management. and ensures accurate image status\n\n[…]\nproperty directly, removing duplicate\n image inspection logic\n\nThis fixes issues where the manual flag could become stale, causing\nassertion errors in push_image() even when the image exists locally.",
"is_bot": false,
"headline": "fix push_image with dynamic image existence tracking (#184)",
"author_name": "Daniel Zayas",
"author_login": "danielzayas",
"committed_at": "2025-12-19T20:10:37Z",
"body_truncated": true,
"is_coding_agent": false
},
{
"oid": "75dc442b3be1ea1dd509c76e825d7b3661c3e325",
"body": null,
"is_bot": false,
"headline": "Update dockerhub org to swebench",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-12-18T18:33:30Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "7e7d6c9d8fde0e3a8f5efc00ffa494c4bb79a77d",
"body": "* Remove two duplicated Rust repo profiles\n\n* Keeping newer versions of repos\n\n---------\n\nCo-authored-by: John Yang <byjohnyang@gmail.com>",
"is_bot": false,
"headline": "Remove two duplicated Rust repo profiles (#187)",
"author_name": "Kevin Xiang Li",
"author_login": "AlienKevin",
"committed_at": "2025-12-18T18:22:02Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "27e27098a9478dc64b57dc91050ca574b0810ad8",
"body": null,
"is_bot": false,
"headline": "Fix broken pytest",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-12-17T23:35:30Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "322ad08abec4a77b0c68a2e09c74d91f77a2a6f8",
"body": null,
"is_bot": false,
"headline": "Merge branch 'main' of github.com:SWE-bench/SWE-smith",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-12-17T23:29:47Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "46ede50813ede22ac706e316281fd1347d74572b",
"body": null,
"is_bot": false,
"headline": "Add Rust repos, replace `testbed` with `ENV_NAME`",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-12-17T23:29:40Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "e31e95f51ff13a998391142c4ea3f739f4e2847d",
"body": "updates:\n- [github.com/astral-sh/ruff-pre-commit: v0.12.12 → v0.14.9](https://github.com/astral-sh/ruff-pre-commit/compare/v0.12.12...v0.14.9)\n\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": true,
"headline": "[pre-commit.ci] pre-commit autoupdate (#161)",
"author_name": "pre-commit-ci[bot]",
"author_login": "pre-commit-ci[bot]",
"committed_at": "2025-12-17T06:28:05Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "e8b8499c54a7d1d398b62ac6b69a0f75fe2ec89d",
"body": "Bumps [actions/cache](https://github.com/actions/cache) from 4 to 5.\n- [Release notes](https://github.com/actions/cache/releases)\n- [Changelog](https://github.com/actions/cache/blob/main/RELEASES.md)\n- [Commits](https://github.com/actions/cache/compare/v4...v5)\n\n---\nupdated-dependencies:\n- dependenc\n[…]\nirect:production\n update-type: version-update:semver-major\n...\n\nSigned-off-by: dependabot[bot] <support@github.com>\nCo-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>",
"is_bot": true,
"headline": "Bump actions/cache from 4 to 5 (#182)",
"author_name": "dependabot[bot]",
"author_login": "dependabot[bot]",
"committed_at": "2025-12-16T22:29:58Z",
"body_truncated": true,
"is_coding_agent": false
},
{
"oid": "c9263d563e9482f476723f1a9529d168283a2cf5",
"body": "Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 5.5.1 to 5.5.2.\n- [Release notes](https://github.com/codecov/codecov-action/releases)\n- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/codecov/codecov-action/c\n[…]\nirect:production\n update-type: version-update:semver-patch\n...\n\nSigned-off-by: dependabot[bot] <support@github.com>\nCo-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>",
"is_bot": true,
"headline": "Bump codecov/codecov-action from 5.5.1 to 5.5.2 (#183)",
"author_name": "dependabot[bot]",
"author_login": "dependabot[bot]",
"committed_at": "2025-12-16T22:29:52Z",
"body_truncated": true,
"is_coding_agent": false
},
{
"oid": "fb862a205d1efa371452d51db7088ee9d50b374a",
"body": "… (#180)\n\n* Fix container name conflict in get_container() for parallel execution\n\n* Style: fix formatting checks",
"is_bot": false,
"headline": "Fix container name conflict in get_container() for parallel execution…",
"author_name": "Alan",
"author_login": "Simplewyl2000",
"committed_at": "2025-12-16T22:14:16Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "364b561f30537dea0ea099f47fefb036ad3798d3",
"body": null,
"is_bot": false,
"headline": "Update repo exts",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-12-15T06:18:41Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "179560be4acd6527b317885028b95920fe026334",
"body": "Adds comprehensive landing page to /guides/ with:\n- Intro summarizing the complete workflow\n- Prerequisites section (Docker, Ubuntu requirements)\n- Grid cards navigation for all 6 tutorials\n- Mermaid workflow diagram showing recommended path\n- Concise descriptions for each tutorial section\n\nFixes #169",
"is_bot": false,
"headline": "Docs: add Tutorials landing page with overview and navigation (#172)",
"author_name": "Cesar Garcia",
"author_login": "Chesars",
"committed_at": "2025-12-10T23:42:48Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "a9ae089bc2e6f595e61e1a0ec0e1fee6d9f75f44",
"body": "Bumps [actions/checkout](https://github.com/actions/checkout) from 5 to 6.\n- [Release notes](https://github.com/actions/checkout/releases)\n- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/actions/checkout/compare/v5...v6)\n\n---\nupdated-dependenc\n[…]\nirect:production\n update-type: version-update:semver-major\n...\n\nSigned-off-by: dependabot[bot] <support@github.com>\nCo-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>",
"is_bot": true,
"headline": "Bump actions/checkout from 5 to 6 (#175)",
"author_name": "dependabot[bot]",
"author_login": "dependabot[bot]",
"committed_at": "2025-12-10T23:42:02Z",
"body_truncated": true,
"is_coding_agent": false
},
{
"oid": "5bde13b9f5f8e638d010c5cf37ff8c25029006df",
"body": "* Add Rust procedural generation support for operations\n\n- Implement RustProceduralModifier base class\n- Add 5 operation modifiers for Rust:\n - OperationChangeModifier: Change operators within same category\n - OperationFlipOperatorModifier: Flip operators to their opposites\n - OperationSwapOperan\n[…]\nheck happy\n\n---------\n\nCo-authored-by: Devin AI <158243242+devin-ai-integration[bot]@users.noreply.github.com>\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": false,
"headline": "Add Rust procedural bug generation modifiers (#167)",
"author_name": "Kevin Xiang Li",
"author_login": "AlienKevin",
"committed_at": "2025-11-20T20:34:15Z",
"body_truncated": true,
"is_coding_agent": true
},
{
"oid": "7329a01b56ce1e64873b9d60a6bcb1a9375a0f3d",
"body": null,
"is_bot": false,
"headline": "Update README with NeurIPS 2025 information",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-11-12T17:23:58Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "475843c38a840be4202e5bf94544117926a40436",
"body": "Updated citation format for the SWE-smith paper.",
"is_bot": false,
"headline": "Modify citation entry for SWE-smith paper",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-09-23T17:55:59Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "aa7f64ced123ed3e42ff230593c32f0b379a7426",
"body": null,
"is_bot": false,
"headline": "Release 0.0.7",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-09-15T16:52:27Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "7d9d849baaf82320a83f9621aab4c022097dba84",
"body": "Bumps [actions/setup-python](https://github.com/actions/setup-python) from 5 to 6.\n- [Release notes](https://github.com/actions/setup-python/releases)\n- [Commits](https://github.com/actions/setup-python/compare/v5...v6)\n\n---\nupdated-dependencies:\n- dependency-name: actions/setup-python\n dependency-\n[…]\nirect:production\n update-type: version-update:semver-major\n...\n\nSigned-off-by: dependabot[bot] <support@github.com>\nCo-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>",
"is_bot": true,
"headline": "Bump actions/setup-python from 5 to 6 (#158)",
"author_name": "dependabot[bot]",
"author_login": "dependabot[bot]",
"committed_at": "2025-09-12T21:07:42Z",
"body_truncated": true,
"is_coding_agent": false
},
{
"oid": "e6efef385391a6afa7bc8fa2dd1dc279c264099d",
"body": "- tree-sitter-go v0.25.0 introduces a new statement_list intermediate\n AST node.\n- This breaks the Go ControlShuffleLinesModifier because it expects to\n find the function statements directly underneath the function body.\n- Update the modifier so that it handles the statement_list when\n present.",
"is_bot": false,
"headline": "Fix Go line shuffling procedural modification (#153)",
"author_name": "Andrew Crump",
"author_login": "acrmp",
"committed_at": "2025-09-12T21:02:20Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "e6946b6d1bb78731de689270df6bd042fb425427",
"body": "- The issue_gen.get_from_tests command was updated in cabec5da to read\n the test command timeout from the repo profile\n- However the caller was not updated to pass the repo profile through\n\nFixes:\n\n```\nTypeError: run_command_in_container() missing 1 required positional argument: 'rp'\n```",
"is_bot": false,
"headline": "Fix get_from_tests run_command_in_container call (#156)",
"author_name": "Andrew Crump",
"author_login": "acrmp",
"committed_at": "2025-09-12T21:00:18Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "b0a654251c7a3ca118d86da9ebc93d5a79318463",
"body": "Bumps [codecov/codecov-action](https://github.com/codecov/codecov-action) from 5.4.3 to 5.5.1.\n- [Release notes](https://github.com/codecov/codecov-action/releases)\n- [Changelog](https://github.com/codecov/codecov-action/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/codecov/codecov-action/c\n[…]\nirect:production\n update-type: version-update:semver-minor\n...\n\nSigned-off-by: dependabot[bot] <support@github.com>\nCo-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>",
"is_bot": true,
"headline": "Bump codecov/codecov-action from 5.4.3 to 5.5.1 (#159)",
"author_name": "dependabot[bot]",
"author_login": "dependabot[bot]",
"committed_at": "2025-09-12T20:58:52Z",
"body_truncated": true,
"is_coding_agent": false
},
{
"oid": "aff47e7b20353e36ca69031a9b228be876f651a1",
"body": "Bumps [actions/checkout](https://github.com/actions/checkout) from 4 to 5.\n- [Release notes](https://github.com/actions/checkout/releases)\n- [Changelog](https://github.com/actions/checkout/blob/main/CHANGELOG.md)\n- [Commits](https://github.com/actions/checkout/compare/v4...v5)\n\n---\nupdated-dependenc\n[…]\nirect:production\n update-type: version-update:semver-major\n...\n\nSigned-off-by: dependabot[bot] <support@github.com>\nCo-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>",
"is_bot": true,
"headline": "Bump actions/checkout from 4 to 5 (#149)",
"author_name": "dependabot[bot]",
"author_login": "dependabot[bot]",
"committed_at": "2025-09-12T20:58:31Z",
"body_truncated": true,
"is_coding_agent": false
},
{
"oid": "a8a55bf21d7595fae71d79ba88f39ad616163d16",
"body": "updates:\n- [github.com/astral-sh/ruff-pre-commit: v0.12.8 → v0.12.12](https://github.com/astral-sh/ruff-pre-commit/compare/v0.12.8...v0.12.12)\n\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": true,
"headline": "[pre-commit.ci] pre-commit autoupdate (#145)",
"author_name": "pre-commit-ci[bot]",
"author_login": "pre-commit-ci[bot]",
"committed_at": "2025-09-12T20:58:21Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "471c4d38c2857d4166ea1030c2392bc8cef869e6",
"body": "- Attempts to use gpt-5 models with certain commands would fail\n previously due to temperature not being supported\n- Set litellm.drop_params to drop parameters not supported by models\n\nFixes:\n\nlitellm.exceptions.UnsupportedParamsError: litellm.UnsupportedParamsError: gpt-5 models don't support temperature=0. Only temperature=1 is supported. To drop unsupported params set `litellm.drop_params = True`",
"is_bot": false,
"headline": "Drop unsupported model params (#155)",
"author_name": "Andrew Crump",
"author_login": "acrmp",
"committed_at": "2025-09-12T20:58:03Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "22bd6f3fe04078a5107cc50f6f2e84f4c970e6e8",
"body": "Update javascript.py",
"is_bot": false,
"headline": "Remove incorrect comment",
"author_name": "tmr",
"author_login": "tmr1",
"committed_at": "2025-09-01T21:14:22Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "edfd6d77491d571e9939ef0515f67ccbdc8f5268",
"body": "Remove incorrect comment",
"is_bot": false,
"headline": "Update javascript.py",
"author_name": "tmr",
"author_login": "tmr1",
"committed_at": "2025-09-01T10:24:29Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "030f87a8152df66474962c04cd592430fa9561e9",
"body": "The `--type` arguments were removed from procedural modification and LLM\nrewriting in commit 1a7e7141.",
"is_bot": false,
"headline": "Remove outdated --type arguments from doc examples (#142)",
"author_name": "Andrew Crump",
"author_login": "acrmp",
"committed_at": "2025-08-14T21:45:59Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "63a3d0c216a7f18031eee6646b3651a72698d6bb",
"body": null,
"is_bot": false,
"headline": "Update README",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-08-14T21:45:17Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "5d7a761e3440980cc4ba264c890efd07f7106368",
"body": null,
"is_bot": false,
"headline": "Revert LICENSE back to MIT",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-08-14T21:44:45Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "af2615a610c76e826dbe1c68262693178fc940bd",
"body": null,
"is_bot": false,
"headline": "Doc: Fix citation name (#139)",
"author_name": "Kilian Lieret",
"author_login": "klieret",
"committed_at": "2025-08-13T17:55:12Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "1d0b85e31248705d30bdd4ead8b483ba4eace248",
"body": null,
"is_bot": false,
"headline": "Doc: Also add mini bubble to frontpage (#138)",
"author_name": "Kilian Lieret",
"author_login": "klieret",
"committed_at": "2025-08-13T17:40:52Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "99f70a17e6dd7e2d0529116c1137d60ee8061122",
"body": "updates:\n- [github.com/astral-sh/ruff-pre-commit: v0.12.5 → v0.12.8](https://github.com/astral-sh/ruff-pre-commit/compare/v0.12.5...v0.12.8)\n\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": true,
"headline": "[pre-commit.ci] pre-commit autoupdate (#137)",
"author_name": "pre-commit-ci[bot]",
"author_login": "pre-commit-ci[bot]",
"committed_at": "2025-08-13T17:40:45Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "5b9cc1f41b4c27f2cfd64dbc5a4e85a4f043c06a",
"body": null,
"is_bot": false,
"headline": "Minor fix",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-08-03T22:52:44Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "15098ce93c6594d6fdece8a399a0c07499b26473",
"body": "* Add C++ test fixture\n\nFrom: https://github.com/libcpr/cpr/blob/e10e86f4327aff234b0dda73622557d19ff281c6/cpr/util.cpp\n\n* Support bug generation for C++\n\nSupports *.cpp files but does not attempt to handle function\ndefinitions within headers.\n\n* Remove C++ from CONTRIBUTING desired languages\n\n* Add `swesmith/profiles/cpp.py`\n\n---------\n\nCo-authored-by: John Yang <byjohnyang@gmail.com>",
"is_bot": false,
"headline": "Support C++ bug generation (#132)",
"author_name": "Andrew Crump",
"author_login": "acrmp",
"committed_at": "2025-08-03T21:42:33Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "dac860833b51b656ff2782392cff5a84de31f515",
"body": "* added Rust and JS repos\n\n* [pre-commit.ci] auto fixes from pre-commit.com hooks\n\nfor more information, see https://pre-commit.ci\n\n* Add `eval_sets` field to specify if repos are part of test sets\n\n* Address feedback\n\n* Update links\n\n---------\n\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>\nCo-authored-by: John Yang <byjohnyang@gmail.com>",
"is_bot": false,
"headline": "Added Rust and JS repos (#133)",
"author_name": "Richard Zhuang",
"author_login": "richardzhuang0412",
"committed_at": "2025-08-03T20:32:52Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "c135509dd0415112d178036ef49b079e54aa4181",
"body": "* Add `eval_sets` field to specify if repos are part of test sets\n\n* Address feedback\n\n* Fix dead links\n\n* More dead link fixes",
"is_bot": false,
"headline": "Add `eval_sets` field to specify if repos are part of test sets (#136)",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-08-03T19:17:12Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "f02363ae4802cd8db4f8c0fa234521c8c13c1e1d",
"body": "updates:\n- [github.com/astral-sh/ruff-pre-commit: v0.12.4 → v0.12.5](https://github.com/astral-sh/ruff-pre-commit/compare/v0.12.4...v0.12.5)\n\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": true,
"headline": "[pre-commit.ci] pre-commit autoupdate (#130)",
"author_name": "pre-commit-ci[bot]",
"author_login": "pre-commit-ci[bot]",
"committed_at": "2025-08-02T21:02:15Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "6af7ddc3488b10620d06f0383d189008a0f02be9",
"body": null,
"is_bot": false,
"headline": "Doc: Link mini (#131)",
"author_name": "Kilian Lieret",
"author_login": "klieret",
"committed_at": "2025-08-02T21:02:05Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "f60ddeee16bc143804cfe3fcaa1a94ad6a4b6eb2",
"body": null,
"is_bot": false,
"headline": "Remove `dependabot.y*ml` files during mirror clone",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-07-29T06:00:57Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "9e481c2d8142a49a45865b692a750a0df682d2ea",
"body": null,
"is_bot": false,
"headline": "Remove last batch of timeout Go repos",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-07-29T05:34:59Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "c9fc1963dfb26829f745e1ea8fb6b6519436499d",
"body": null,
"is_bot": false,
"headline": "Remove non-working Go repos",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-07-24T23:03:35Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "64e955dc0403465c644b8d1e2f28f6f4cbba0a59",
"body": null,
"is_bot": false,
"headline": "Add operations mods for Go; Add CommonPMs",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-07-24T20:55:45Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "b154665b356d02d0554b5d37a64e93bd04e9eb6f",
"body": null,
"is_bot": false,
"headline": "Fix golang shuffling; More informative pbars for harnesses",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-07-24T05:35:46Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "1c3e249d1d12a1b202e424141c94e7e94783b1a6",
"body": null,
"is_bot": false,
"headline": "Add more PM funcs + tests for go",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-07-24T00:19:01Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "16333ef5a1d6e78aa1b8d50682cbaa33273917e1",
"body": null,
"is_bot": false,
"headline": "Add `analyze_properties` to Golang; Init golang procedural mod",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-07-23T23:01:18Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "7681546b11e7a11740cd5678cafd42ea742418c5",
"body": null,
"is_bot": false,
"headline": "Make procedural mod. `generate` lang. agnostic",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-07-23T20:00:45Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "3a48544d40252893f1e7e39d0e6e45b50c5335ea",
"body": "- Previously we included the leading \"test \" prefix from the test output\n in the name of each test. Remove the prefix as it is unnecessary.\n- Backfill log parsing tests.",
"is_bot": false,
"headline": "Remove \"test \" prefix from Rust test names (#126)",
"author_name": "Andrew Crump",
"author_login": "acrmp",
"committed_at": "2025-07-23T18:07:08Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "4c8ce5ca1c4c3de28cb0192dbc08e035446bb088",
"body": null,
"is_bot": false,
"headline": "Check exit code when building docker image (#125)",
"author_name": "Andrew Crump",
"author_login": "acrmp",
"committed_at": "2025-07-23T18:05:43Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "6f4899a65bebe59d59fb5c0f5991e82fdfafb306",
"body": "* Add Rust profiles\n\n* Bump tokio profile timeout",
"is_bot": false,
"headline": "Add Rust profiles (#124)",
"author_name": "Andrew Crump",
"author_login": "acrmp",
"committed_at": "2025-07-23T17:41:11Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "057f0478b6918bfcd89a51ceeec7229c60bb1028",
"body": null,
"is_bot": false,
"headline": "v0.0.6",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-07-22T00:50:27Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "108b9272ed939575f770bd8c4eb4a6bbe59cc922",
"body": "…le` field; Implement `get_container` method",
"is_bot": false,
"headline": "Rename `global_registry` -> `registry`; Move `GhApi` to be `RepoProfi…",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-07-22T00:46:34Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "b39592ecb04b6b8d93229b6c75bface86e08980d",
"body": null,
"is_bot": false,
"headline": "Add links to CONTRIBUTING doc (#123)",
"author_name": "Andrew Crump",
"author_login": "acrmp",
"committed_at": "2025-07-21T22:41:10Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "b5869a622171adccb0fa71df4a75e28048d5e165",
"body": null,
"is_bot": false,
"headline": "Update license",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-07-21T22:20:22Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "334c65702544f2f39244dcf7856ec1bc68c5b5e8",
"body": "updates:\n- [github.com/astral-sh/ruff-pre-commit: v0.12.3 → v0.12.4](https://github.com/astral-sh/ruff-pre-commit/compare/v0.12.3...v0.12.4)\n\nCo-authored-by: pre-commit-ci[bot] <66853113+pre-commit-ci[bot]@users.noreply.github.com>",
"is_bot": true,
"headline": "[pre-commit.ci] pre-commit autoupdate (#122)",
"author_name": "pre-commit-ci[bot]",
"author_login": "pre-commit-ci[bot]",
"committed_at": "2025-07-21T19:07:35Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "f9b4cffe9c9429265630fda5d2a85e0ef5affe45",
"body": null,
"is_bot": false,
"headline": "Init `CONTRIBUTING.md`",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-07-21T19:05:06Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "1345cf488368a77c147d2a20c2848659e247beb8",
"body": "--resolved_only does not support currently.",
"is_bot": false,
"headline": "Update collect_trajs.py (#121)",
"author_name": "Di Lu",
"author_login": "ludybupt",
"committed_at": "2025-07-21T18:06:49Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "fe3595d01048e44099bfaeae766029ee13ba0dbb",
"body": null,
"is_bot": false,
"headline": "Merge branch 'fix-tree-sitter-query-api-usage'",
"author_name": "John Yang",
"author_login": "john-b-yang",
"committed_at": "2025-07-21T06:28:33Z",
"body_truncated": false,
"is_coding_agent": false
}
],
"releases_count": 4,
"commits_last_year": 83,
"latest_release_at": "2025-07-22T00:50:27Z",
"latest_release_tag": "v0.0.6",
"releases_from_tags": true,
"days_since_last_push": 4,
"active_weeks_last_year": 23,
"days_since_latest_release": 368,
"mean_days_between_releases": 26.9
},
"community": {
"has_readme": true,
"has_license": true,
"has_description": true,
"has_contributing": true,
"health_percentage": 75,
"has_issue_template": false,
"has_code_of_conduct": true,
"has_pull_request_template": false
},
"ecosystem": {
"packages": [
{
"name": "swesmith",
"exists": true,
"license": null,
"keywords": [
"nlp",
"benchmark",
"code",
"License :: OSI Approved :: MIT License",
"Operating System :: OS Independent",
"Programming Language :: Python :: 3 :: Only",
"Programming Language :: Python :: 3.10",
"Programming Language :: Python :: 3.11"
],
"ecosystem": "pypi",
"matches_repo": true,
"registry_url": "https://pypi.org/project/swesmith/",
"is_deprecated": false,
"latest_version": "0.0.9",
"repository_url": "https://github.com/SWE-bench/SWE-smith",
"versions_count": 10,
"total_downloads": null,
"dependents_count": null,
"deprecation_note": null,
"maintainers_count": null,
"monthly_downloads": 5197566,
"first_published_at": "2025-04-29T21:44:11.537835Z",
"latest_published_at": "2026-02-27T01:06:13.455087Z",
"latest_version_yanked": null,
"days_since_latest_publish": 148
}
]
},
"popularity": {
"forks": 126,
"stars": 710,
"watchers": 7,
"fork_history": {
"days": [
{
"date": "2025-05-07",
"count": 4
},
{
"date": "2025-05-09",
"count": 1
},
{
"date": "2025-05-12",
"count": 1
},
{
"date": "2025-05-19",
"count": 1
},
{
"date": "2025-05-21",
"count": 1
},
{
"date": "2025-05-22",
"count": 1
},
{
"date": "2025-05-23",
"count": 1
},
{
"date": "2025-05-27",
"count": 1
},
{
"date": "2025-05-28",
"count": 1
},
{
"date": "2025-05-29",
"count": 1
},
{
"date": "2025-06-04",
"count": 2
},
{
"date": "2025-06-05",
"count": 1
},
{
"date": "2025-06-08",
"count": 1
},
{
"date": "2025-06-09",
"count": 1
},
{
"date": "2025-06-12",
"count": 1
},
{
"date": "2025-06-13",
"count": 3
},
{
"date": "2025-06-17",
"count": 1
},
{
"date": "2025-06-18",
"count": 1
},
{
"date": "2025-06-27",
"count": 1
},
{
"date": "2025-06-30",
"count": 1
},
{
"date": "2025-07-01",
"count": 1
},
{
"date": "2025-07-02",
"count": 1
},
{
"date": "2025-07-05",
"count": 1
},
{
"date": "2025-07-07",
"count": 1
},
{
"date": "2025-07-08",
"count": 2
},
{
"date": "2025-07-09",
"count": 1
},
{
"date": "2025-07-10",
"count": 1
},
{
"date": "2025-07-11",
"count": 1
},
{
"date": "2025-07-12",
"count": 1
},
{
"date": "2025-07-14",
"count": 1
},
{
"date": "2025-07-15",
"count": 1
},
{
"date": "2025-07-18",
"count": 1
},
{
"date": "2025-07-21",
"count": 1
},
{
"date": "2025-07-24",
"count": 1
},
{
"date": "2025-07-27",
"count": 1
},
{
"date": "2025-07-30",
"count": 1
},
{
"date": "2025-08-08",
"count": 1
},
{
"date": "2025-08-11",
"count": 1
},
{
"date": "2025-08-12",
"count": 1
},
{
"date": "2025-08-13",
"count": 1
},
{
"date": "2025-08-14",
"count": 1
},
{
"date": "2025-08-15",
"count": 1
},
{
"date": "2025-08-21",
"count": 1
},
{
"date": "2025-08-28",
"count": 1
},
{
"date": "2025-09-01",
"count": 1
},
{
"date": "2025-09-02",
"count": 1
},
{
"date": "2025-09-10",
"count": 1
},
{
"date": "2025-09-12",
"count": 1
},
{
"date": "2025-09-20",
"count": 1
},
{
"date": "2025-10-11",
"count": 1
},
{
"date": "2025-10-16",
"count": 1
},
{
"date": "2025-10-18",
"count": 1
},
{
"date": "2025-10-20",
"count": 1
},
{
"date": "2025-10-21",
"count": 1
},
{
"date": "2025-10-22",
"count": 2
},
{
"date": "2025-10-23",
"count": 1
},
{
"date": "2025-10-27",
"count": 1
},
{
"date": "2025-10-28",
"count": 2
},
{
"date": "2025-11-02",
"count": 1
},
{
"date": "2025-11-03",
"count": 1
},
{
"date": "2025-11-13",
"count": 1
},
{
"date": "2025-11-20",
"count": 1
},
{
"date": "2025-11-21",
"count": 1
},
{
"date": "2025-11-22",
"count": 1
},
{
"date": "2025-11-23",
"count": 1
},
{
"date": "2025-11-25",
"count": 2
},
{
"date": "2025-11-27",
"count": 1
},
{
"date": "2025-11-29",
"count": 1
},
{
"date": "2025-12-09",
"count": 1
},
{
"date": "2025-12-11",
"count": 1
},
{
"date": "2025-12-13",
"count": 2
},
{
"date": "2025-12-17",
"count": 2
},
{
"date": "2026-01-15",
"count": 1
},
{
"date": "2026-01-16",
"count": 1
},
{
"date": "2026-01-19",
"count": 1
},
{
"date": "2026-01-26",
"count": 1
},
{
"date": "2026-01-27",
"count": 1
},
{
"date": "2026-01-29",
"count": 2
},
{
"date": "2026-01-30",
"count": 2
},
{
"date": "2026-02-07",
"count": 1
},
{
"date": "2026-02-08",
"count": 3
},
{
"date": "2026-02-09",
"count": 1
},
{
"date": "2026-02-10",
"count": 2
},
{
"date": "2026-02-17",
"count": 1
},
{
"date": "2026-02-20",
"count": 1
},
{
"date": "2026-02-21",
"count": 1
},
{
"date": "2026-02-25",
"count": 1
},
{
"date": "2026-03-02",
"count": 1
},
{
"date": "2026-03-03",
"count": 1
},
{
"date": "2026-03-08",
"count": 1
},
{
"date": "2026-03-17",
"count": 1
},
{
"date": "2026-03-26",
"count": 1
},
{
"date": "2026-04-07",
"count": 1
},
{
"date": "2026-04-14",
"count": 1
},
{
"date": "2026-04-16",
"count": 1
},
{
"date": "2026-04-21",
"count": 1
},
{
"date": "2026-05-13",
"count": 1
},
{
"date": "2026-05-19",
"count": 1
},
{
"date": "2026-05-22",
"count": 1
},
{
"date": "2026-05-31",
"count": 1
},
{
"date": "2026-06-05",
"count": 1
},
{
"date": "2026-06-23",
"count": 1
},
{
"date": "2026-06-25",
"count": 1
},
{
"date": "2026-06-29",
"count": 1
},
{
"date": "2026-07-01",
"count": 1
},
{
"date": "2026-07-02",
"count": 1
},
{
"date": "2026-07-11",
"count": 1
}
],
"complete": true,
"collected": 124,
"total_forks": 126
},
"star_history": null,
"open_issues_and_prs": 22
},
"ai_readiness": {
"has_nix": false,
"example_dirs": [],
"has_llms_txt": false,
"has_dockerfile": false,
"has_mcp_signal": false,
"bootstrap_files": [],
"api_schema_files": [],
"has_devcontainer": false,
"typecheck_configs": [],
"toolchain_manifests": [],
"largest_source_bytes": 276830,
"source_files_sampled": 223,
"oversized_source_files": 5,
"agent_instruction_files": [
"AGENTS.md"
],
"agent_instruction_max_bytes": 194
},
"dependencies": {
"manifests": [
"pyproject.toml"
],
"advisories": {
"error": "No resolved dependencies to assess",
"scope": "repository_graph",
"source": null,
"findings": [],
"collected": false,
"malicious": [],
"truncated": false,
"by_severity": {},
"advisory_count": 0,
"affected_count": 0,
"assessed_count": 0,
"malicious_count": 0,
"assessed_package": null,
"unassessed_count": 0,
"direct_affected_count": 0
},
"ecosystems": [
"pypi"
],
"dependencies": [],
"all_dependencies": {
"error": null,
"source": "github-sbom",
"packages": [],
"collected": true,
"truncated": false,
"total_count": 0,
"direct_count": 0,
"indirect_count": 0
}
},
"maintainership": {
"issues": {
"open_prs": 9,
"merged_prs": 123,
"open_issues": 13,
"closed_ratio": 0.856,
"closed_issues": 77,
"closed_unmerged_prs": 20
},
"bus_factor": 1,
"bot_contributors": 2,
"top_contributors": [
{
"type": "User",
"login": "john-b-yang",
"commits": 94,
"avatar_url": "https://avatars.githubusercontent.com/u/10174390?v=4"
},
{
"type": "User",
"login": "acrmp",
"commits": 27,
"avatar_url": "https://avatars.githubusercontent.com/u/444264?v=4"
},
{
"type": "User",
"login": "klieret",
"commits": 15,
"avatar_url": "https://avatars.githubusercontent.com/u/13602468?v=4"
},
{
"type": "User",
"login": "AlienKevin",
"commits": 14,
"avatar_url": "https://avatars.githubusercontent.com/u/22850071?v=4"
},
{
"type": "User",
"login": "mkaramuk",
"commits": 5,
"avatar_url": "https://avatars.githubusercontent.com/u/39464118?v=4"
},
{
"type": "User",
"login": "Priyank-Shethia3",
"commits": 5,
"avatar_url": "https://avatars.githubusercontent.com/u/106236880?v=4"
},
{
"type": "User",
"login": "tmr1",
"commits": 4,
"avatar_url": "https://avatars.githubusercontent.com/u/29013762?v=4"
},
{
"type": "User",
"login": "akhatua2",
"commits": 3,
"avatar_url": "https://avatars.githubusercontent.com/u/54747935?v=4"
},
{
"type": "User",
"login": "danielzayas",
"commits": 3,
"avatar_url": "https://avatars.githubusercontent.com/u/31169718?v=4"
},
{
"type": "User",
"login": "MarcCote",
"commits": 3,
"avatar_url": "https://avatars.githubusercontent.com/u/660004?v=4"
}
],
"contributors_sampled": 18,
"top_contributor_share": 0.514
},
"quality_signals": {
"has_ci": true,
"has_tests": true,
"ci_workflows": [
"build-docs.yaml",
"check-links.yaml",
"pytest.yaml"
],
"has_docs_dir": true,
"linter_configs": [],
"has_editorconfig": false,
"has_linter_config": true,
"has_precommit_config": true
},
"security_signals": {
"lockfiles": [],
"scorecard": {
"checks": [
{
"name": "Binary-Artifacts",
"score": 10,
"reason": "no binaries found in the repo",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#binary-artifacts"
},
{
"name": "Branch-Protection",
"score": 0,
"reason": "branch protection not enabled on development/release branches",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#branch-protection"
},
{
"name": "CI-Tests",
"score": 9,
"reason": "25 out of 26 merged PRs checked by a CI test -- score normalized to 9",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#ci-tests"
},
{
"name": "CII-Best-Practices",
"score": 0,
"reason": "no effort to earn an OpenSSF best practices badge detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#cii-best-practices"
},
{
"name": "Code-Review",
"score": 7,
"reason": "Found 19/25 approved changesets -- score normalized to 7",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#code-review"
},
{
"name": "Contributors",
"score": 10,
"reason": "project has 26 contributing companies or organizations",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#contributors"
},
{
"name": "Dangerous-Workflow",
"score": 10,
"reason": "no dangerous workflow patterns detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#dangerous-workflow"
},
{
"name": "Dependency-Update-Tool",
"score": 10,
"reason": "update tool detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#dependency-update-tool"
},
{
"name": "Fuzzing",
"score": 0,
"reason": "project is not fuzzed",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#fuzzing"
},
{
"name": "License",
"score": 10,
"reason": "license file detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#license"
},
{
"name": "Maintained",
"score": 0,
"reason": "0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#maintained"
},
{
"name": "Packaging",
"score": null,
"reason": "packaging workflow not detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#packaging"
},
{
"name": "Pinned-Dependencies",
"score": 0,
"reason": "dependency not pinned by hash detected -- score normalized to 0",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#pinned-dependencies"
},
{
"name": "SAST",
"score": 0,
"reason": "SAST tool is not run on all commits -- score normalized to 0",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#sast"
},
{
"name": "Security-Policy",
"score": 0,
"reason": "security policy file not detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#security-policy"
},
{
"name": "Signed-Releases",
"score": null,
"reason": "no releases found",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#signed-releases"
},
{
"name": "Token-Permissions",
"score": 0,
"reason": "detected GitHub workflow tokens with excessive permissions",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#token-permissions"
},
{
"name": "Vulnerabilities",
"score": 10,
"reason": "0 existing vulnerabilities detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#vulnerabilities"
}
],
"commit": "9b74ac08118a85c39c356802f7961893af73e07f",
"ran_at": "2026-07-25T08:58:09Z",
"aggregate_score": 4.9,
"scorecard_version": "v5.5.0"
},
"has_codeql_workflow": false,
"has_security_policy": false,
"has_dependabot_config": true
},
"contribution_flow": {
"collected": true,
"ci_last_run_at": "2026-07-01T02:01:09Z",
"oldest_open_prs": [
{
"number": 177,
"created_at": "2025-11-29T04:23:41Z",
"last_comment_at": "2026-01-06T19:53:58Z",
"last_comment_author": "danielzayas"
},
{
"number": 202,
"created_at": "2026-01-24T14:51:30Z",
"last_comment_at": "2026-01-29T17:58:17Z",
"last_comment_author": "john-b-yang"
},
{
"number": 227,
"created_at": "2026-02-27T01:13:20Z",
"last_comment_at": null,
"last_comment_author": null
},
{
"number": 235,
"created_at": "2026-03-09T19:52:44Z",
"last_comment_at": "2026-03-09T20:44:52Z",
"last_comment_author": "codecov"
},
{
"number": 240,
"created_at": "2026-06-08T11:04:32Z",
"last_comment_at": null,
"last_comment_author": null
},
{
"number": 241,
"created_at": "2026-06-29T14:28:04Z",
"last_comment_at": null,
"last_comment_author": null
},
{
"number": 242,
"created_at": "2026-07-11T17:47:32Z",
"last_comment_at": null,
"last_comment_author": null
},
{
"number": 243,
"created_at": "2026-07-12T17:12:27Z",
"last_comment_at": null,
"last_comment_author": null
},
{
"number": 244,
"created_at": "2026-07-12T17:45:58Z",
"last_comment_at": null,
"last_comment_author": null
}
],
"last_merged_pr_at": "2026-03-21T23:39:59Z",
"ci_last_conclusion": "SUCCESS",
"oldest_open_issues": [
{
"number": 64,
"created_at": "2025-06-23T18:59:20Z",
"last_comment_at": null,
"last_comment_author": null
},
{
"number": 98,
"created_at": "2025-07-08T04:42:07Z",
"last_comment_at": "2025-07-23T14:31:43Z",
"last_comment_author": "Martovark"
},
{
"number": 115,
"created_at": "2025-07-17T14:21:20Z",
"last_comment_at": "2025-09-12T21:07:18Z",
"last_comment_author": "john-b-yang"
},
{
"number": 127,
"created_at": "2025-07-25T07:49:51Z",
"last_comment_at": "2025-07-31T18:48:08Z",
"last_comment_author": "arktrail"
},
{
"number": 146,
"created_at": "2025-08-22T09:47:08Z",
"last_comment_at": "2025-09-02T22:11:09Z",
"last_comment_author": "acrmp"
},
{
"number": 147,
"created_at": "2025-08-23T21:40:59Z",
"last_comment_at": "2025-09-12T21:04:11Z",
"last_comment_author": "john-b-yang"
},
{
"number": 190,
"created_at": "2025-12-26T17:57:32Z",
"last_comment_at": "2025-12-28T06:54:12Z",
"last_comment_author": "AlienKevin"
},
{
"number": 193,
"created_at": "2026-01-03T13:20:23Z",
"last_comment_at": null,
"last_comment_author": null
},
{
"number": 194,
"created_at": "2026-01-06T19:53:02Z",
"last_comment_at": null,
"last_comment_author": null
},
{
"number": 195,
"created_at": "2026-01-06T20:10:47Z",
"last_comment_at": null,
"last_comment_author": null
},
{
"number": 226,
"created_at": "2026-02-25T08:07:19Z",
"last_comment_at": null,
"last_comment_author": null
},
{
"number": 232,
"created_at": "2026-03-08T17:32:27Z",
"last_comment_at": null,
"last_comment_author": null
},
{
"number": 238,
"created_at": "2026-05-09T00:45:41Z",
"last_comment_at": null,
"last_comment_author": null
}
]
}
},
"config": {
"disabled_metrics": [],
"disabled_categories": [],
"disabled_components": {}
},
"source": {
"url": "https://github.com/SWE-bench/SWE-smith",
"host": "github.com",
"name": "SWE-smith",
"owner": "SWE-bench"
},
"metrics": {
"overall": {
"key": "overall",
"band": "good",
"name": "Overall health",
"note": null,
"notes": [],
"value": 72,
"inputs": {
"security": 49,
"vitality": 64,
"community": 82,
"governance": 69,
"engineering": 95
},
"components": []
},
"categories": [
{
"key": "vitality",
"band": "moderate",
"name": "Vitality",
"value": 64,
"weight": 0.22,
"metrics": [
{
"key": "development_activity",
"band": "moderate",
"name": "Development activity",
"note": null,
"notes": [],
"value": 69,
"inputs": {
"commits_last_year": 83,
"human_commit_share": 0.82,
"days_since_last_push": 4,
"active_weeks_last_year": 23
},
"components": [
{
"key": "push_recency",
"name": "Push recency",
"detail": "last push 4 days ago",
"points": 36,
"status": "met",
"details": [
{
"code": "push_recency",
"params": {
"days": 4
}
}
],
"max_points": 36
},
{
"key": "commit_cadence",
"name": "Commit cadence",
"detail": "23/52 weeks with commits",
"points": 15.9,
"status": "partial",
"details": [
{
"code": "commit_cadence_weeks",
"params": {
"weeks": 23
}
}
],
"max_points": 36
},
{
"key": "commit_volume",
"name": "Commit volume",
"detail": "83 commits in the last year",
"points": 17.3,
"status": "partial",
"details": [
{
"code": "commits_last_year",
"params": {
"count": 83
}
}
],
"max_points": 18
},
{
"key": "openssf_scorecard_maintained",
"name": "OpenSSF Scorecard: Maintained",
"detail": "0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0",
"points": 0,
"status": "missed",
"details": [],
"max_points": 10
}
]
},
{
"key": "release_discipline",
"band": "moderate",
"name": "Release discipline",
"note": "Excluded from scoring (no data or not applicable): OpenSSF Scorecard: Signed-Releases. Remaining weights renormalized.",
"notes": [
{
"code": "excluded_no_data",
"params": {
"components": [
"openssf_scorecard_signed_releases"
]
}
},
{
"code": "weights_renormalized",
"params": {}
}
],
"value": 56,
"inputs": {
"releases_count": 4,
"latest_release_tag": "v0.0.6",
"releases_from_tags": true,
"days_since_latest_release": 368,
"mean_days_between_releases": 26.9
},
"components": [
{
"key": "ships_releases",
"name": "Ships releases",
"detail": "4 version tags (no GitHub releases)",
"points": 16.2,
"status": "partial",
"details": [
{
"code": "version_tags_no_releases",
"params": {
"count": 4
}
}
],
"max_points": 27
},
{
"key": "release_recency",
"name": "Release recency",
"detail": "latest release 368 days ago",
"points": 7.2,
"status": "partial",
"details": [
{
"code": "release_recency",
"params": {
"days": 368
}
}
],
"max_points": 36
},
{
"key": "release_cadence",
"name": "Release cadence",
"detail": "a release every ~26.9 days",
"points": 27,
"status": "met",
"details": [
{
"code": "release_cadence",
"params": {
"gap": 26.9
}
}
],
"max_points": 27
},
{
"key": "openssf_scorecard_signed_releases",
"name": "OpenSSF Scorecard: Signed-Releases",
"detail": "no releases found",
"points": 0,
"status": "excluded",
"details": [
{
"code": "no_data",
"params": {}
}
],
"max_points": 10
}
]
},
{
"key": "abandonment",
"band": "excellent",
"name": "Abandonment",
"note": null,
"notes": [],
"value": 100,
"inputs": {
"cap": null,
"state": "maintained",
"guards": [],
"signals": [],
"red_flag": false,
"multiplier_pct": 100,
"declared_reason": null,
"unverified_reason": null,
"unanswered_open_prs": null,
"unanswered_open_issues": null,
"days_since_last_merged_pr": null,
"days_since_last_human_commit": 125,
"days_since_last_human_commit_is_floor": false
},
"components": [
{
"key": "project_is_still_maintained",
"name": "Project is still maintained",
"detail": "last human commit 125 days ago",
"points": 100,
"status": "met",
"details": [
{
"code": "abandonment_maintained",
"params": {
"days": 125
}
}
],
"max_points": 100
}
]
}
],
"description": "Is the project alive — is code being written and are releases shipping?"
},
{
"key": "community",
"band": "good",
"name": "Community & Adoption",
"value": 82,
"weight": 0.18,
"metrics": [
{
"key": "popularity",
"band": "moderate",
"name": "Popularity & adoption",
"note": null,
"notes": [],
"value": 68,
"inputs": {
"forks": 126,
"stars": 710,
"watchers": 7,
"growth_state": "unverified",
"growth_factor_pct": 100,
"growth_unverified_reason": "no_history"
},
"components": [
{
"key": "stars",
"name": "Stars",
"detail": "710 stars",
"points": 46.2,
"status": "partial",
"details": [
{
"code": "stars",
"params": {
"count": 710
}
}
],
"max_points": 60
},
{
"key": "forks",
"name": "Forks",
"detail": "126 forks",
"points": 17.5,
"status": "partial",
"details": [
{
"code": "forks",
"params": {
"count": 126
}
}
],
"max_points": 25
},
{
"key": "watchers",
"name": "Watchers",
"detail": "7 watchers",
"points": 4.3,
"status": "partial",
"details": [
{
"code": "watchers",
"params": {
"count": 7
}
}
],
"max_points": 15
}
]
},
{
"key": "community_health",
"band": "excellent",
"name": "Community health",
"note": null,
"notes": [],
"value": 85,
"inputs": {
"has_readme": true,
"has_license": true,
"has_contributing": true,
"has_issue_template": false,
"has_code_of_conduct": true,
"has_pull_request_template": false
},
"components": [
{
"key": "readme",
"name": "README",
"detail": null,
"points": 22.5,
"status": "met",
"details": [],
"max_points": 22.5
},
{
"key": "license",
"name": "License",
"detail": "recognized license (MIT)",
"points": 22.5,
"status": "met",
"details": [
{
"code": "license_standard",
"params": {}
},
{
"code": "license_spdx",
"params": {
"spdx": "MIT"
}
}
],
"max_points": 22.5
},
{
"key": "contributing_guide",
"name": "CONTRIBUTING guide",
"detail": null,
"points": 18,
"status": "met",
"details": [],
"max_points": 18
},
{
"key": "code_of_conduct",
"name": "Code of conduct",
"detail": null,
"points": 13.5,
"status": "met",
"details": [],
"max_points": 13.5
},
{
"key": "issue_template",
"name": "Issue template",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 7.2
},
{
"key": "pr_template",
"name": "PR template",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 6.3
}
]
},
{
"key": "ecosystem_adoption",
"band": "excellent",
"name": "Ecosystem adoption (downloads)",
"note": "Excluded from scoring (no data or not applicable): Registry dependents. Remaining weights renormalized.",
"notes": [
{
"code": "excluded_no_data",
"params": {
"components": [
"registry_dependents"
]
}
},
{
"code": "weights_renormalized",
"params": {}
}
],
"value": 100,
"inputs": {
"packages": [
"swesmith"
],
"dependents": null,
"ecosystems": "pypi",
"total_downloads": null,
"monthly_downloads": 5197566
},
"components": [
{
"key": "monthly_downloads",
"name": "Monthly downloads",
"detail": "5,197,566 downloads/month across pypi",
"points": 80,
"status": "met",
"details": [
{
"code": "downloads_monthly",
"params": {
"count": 5197566,
"ecosystems": "pypi"
}
}
],
"max_points": 80
},
{
"key": "registry_dependents",
"name": "Registry dependents",
"detail": "not reported by this ecosystem",
"points": 0,
"status": "excluded",
"details": [
{
"code": "not_reported_by_this_ecosystem",
"params": {}
}
],
"max_points": 20
}
]
}
],
"description": "Does the project have users, downloads, attention, and a welcoming setup for contributors?"
},
{
"key": "governance",
"band": "moderate",
"name": "Sustainability & Governance",
"value": 69,
"weight": 0.24,
"metrics": [
{
"key": "maintainer_resilience",
"band": "at_risk",
"name": "Maintainer resilience (bus factor)",
"note": null,
"notes": [],
"value": 43,
"inputs": {
"bus_factor": 1,
"contributors_sampled": 18,
"top_contributor_share": 0.514
},
"components": [
{
"key": "bus_factor",
"name": "Bus factor",
"detail": "1 contributor(s) cover half of all commits",
"points": 9,
"status": "partial",
"details": [
{
"code": "bus_factor",
"params": {
"count": 1
}
}
],
"max_points": 54
},
{
"key": "commit_distribution",
"name": "Commit distribution",
"detail": "top contributor authored 51% of commits",
"points": 10.9,
"status": "partial",
"details": [
{
"code": "top_contributor_share",
"params": {
"share": 51
}
}
],
"max_points": 22.5
},
{
"key": "contributor_breadth",
"name": "Contributor breadth",
"detail": "18 contributors",
"points": 13.5,
"status": "met",
"details": [
{
"code": "contributors_sampled",
"params": {
"count": 18
}
}
],
"max_points": 13.5
},
{
"key": "openssf_scorecard_contributors",
"name": "OpenSSF Scorecard: Contributors",
"detail": "project has 26 contributing companies or organizations",
"points": 10,
"status": "met",
"details": [],
"max_points": 10
}
]
},
{
"key": "responsiveness",
"band": "good",
"name": "Issue & PR responsiveness",
"note": null,
"notes": [],
"value": 83,
"inputs": {
"merged_prs": 123,
"open_issues": 13,
"closed_issues": 77,
"issue_closed_ratio": 0.856,
"closed_unmerged_prs": 20
},
"components": [
{
"key": "issue_resolution",
"name": "Issue resolution",
"detail": "86% of issues closed",
"points": 40,
"status": "partial",
"details": [
{
"code": "issues_closed_share",
"params": {
"share": 86
}
}
],
"max_points": 46.75
},
{
"key": "pr_acceptance",
"name": "PR acceptance",
"detail": "123/143 decided PRs merged",
"points": 32.9,
"status": "partial",
"details": [
{
"code": "decided_prs_merged",
"params": {
"merged": 123,
"decided": 143
}
}
],
"max_points": 38.25
},
{
"key": "openssf_scorecard_code_review",
"name": "OpenSSF Scorecard: Code-Review",
"detail": "Found 19/25 approved changesets -- score normalized to 7",
"points": 10.5,
"status": "partial",
"details": [],
"max_points": 15
}
]
},
{
"key": "stewardship",
"band": "moderate",
"name": "Ownership & stewardship",
"note": null,
"notes": [],
"value": 61,
"inputs": {
"followers": 307,
"owner_type": "Organization",
"is_verified": null,
"owner_login": "SWE-bench",
"public_repos": 9,
"account_age_days": 1106
},
"components": [
{
"key": "ownership_backing",
"name": "Ownership backing",
"detail": "organization-owned",
"points": 30,
"status": "met",
"details": [
{
"code": "owner_organization",
"params": {}
}
],
"max_points": 30
},
{
"key": "verified_domain",
"name": "Verified domain",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 20
},
{
"key": "owner_reach",
"name": "Owner reach",
"detail": "307 followers of SWE-bench",
"points": 17.9,
"status": "partial",
"details": [
{
"code": "owner_followers",
"params": {
"count": 307,
"login": "SWE-bench"
}
}
],
"max_points": 25
},
{
"key": "track_record",
"name": "Track record",
"detail": "9 public repos, account ~3 yr old",
"points": 13.3,
"status": "partial",
"details": [
{
"code": "public_repos",
"params": {
"count": 9
}
},
{
"code": "account_age_years",
"params": {
"years": 3
}
}
],
"max_points": 25
}
]
},
{
"key": "package_maintenance",
"band": "excellent",
"name": "Package maintenance",
"note": null,
"notes": [],
"value": 100,
"inputs": {
"packages": [
"swesmith"
],
"ecosystems": "pypi",
"any_deprecated": false,
"min_days_since_publish": 148
},
"components": [
{
"key": "published_resolvable",
"name": "Published & resolvable",
"detail": "1 package(s) on pypi",
"points": 25,
"status": "met",
"details": [
{
"code": "packages_published",
"params": {
"count": 1,
"ecosystems": "pypi"
}
}
],
"max_points": 25
},
{
"key": "publish_recency",
"name": "Publish recency",
"detail": "latest publish 148 days ago",
"points": 35,
"status": "met",
"details": [
{
"code": "publish_recency",
"params": {
"days": 148
}
}
],
"max_points": 35
},
{
"key": "version_history",
"name": "Version history",
"detail": "10 published versions",
"points": 20,
"status": "met",
"details": [
{
"code": "published_versions",
"params": {
"count": 10
}
}
],
"max_points": 20
},
{
"key": "not_deprecated",
"name": "Not deprecated",
"detail": "active, not deprecated or yanked",
"points": 20,
"status": "met",
"details": [
{
"code": "package_not_deprecated",
"params": {}
}
],
"max_points": 20
}
]
}
],
"description": "Will the project survive its people — bus factor, responsiveness, who backs it, and package upkeep?"
},
{
"key": "engineering",
"band": "excellent",
"name": "Engineering Quality",
"value": 95,
"weight": 0.2,
"metrics": [
{
"key": "engineering_practices",
"band": "excellent",
"name": "Engineering practices",
"note": null,
"notes": [],
"value": 92,
"inputs": {
"has_ci": true,
"has_tests": true,
"has_editorconfig": false,
"has_linter_config": true,
"has_precommit_config": true
},
"components": [
{
"key": "ci_workflows",
"name": "CI workflows",
"detail": "3 workflow(s)",
"points": 24,
"status": "met",
"details": [
{
"code": "ci_workflows",
"params": {
"count": 3
}
}
],
"max_points": 24
},
{
"key": "tests_present",
"name": "Tests present",
"detail": null,
"points": 24,
"status": "met",
"details": [],
"max_points": 24
},
{
"key": "linter_config",
"name": "Linter config",
"detail": null,
"points": 16,
"status": "met",
"details": [],
"max_points": 16
},
{
"key": "pre_commit_hooks",
"name": "Pre-commit hooks",
"detail": null,
"points": 9.6,
"status": "met",
"details": [],
"max_points": 9.6
},
{
"key": "editorconfig",
"name": ".editorconfig",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 6.4
},
{
"key": "openssf_scorecard_ci_tests",
"name": "OpenSSF Scorecard: CI-Tests",
"detail": "25 out of 26 merged PRs checked by a CI test -- score normalized to 9",
"points": 18,
"status": "partial",
"details": [],
"max_points": 20
}
]
},
{
"key": "documentation",
"band": "excellent",
"name": "Documentation",
"note": null,
"notes": [],
"value": 100,
"inputs": {
"topics": [
"agents",
"language-model",
"software-engineering",
"training"
],
"has_wiki": true,
"homepage": "https://swesmith.com/",
"has_readme": true,
"has_docs_dir": true,
"has_description": true
},
"components": [
{
"key": "readme",
"name": "README",
"detail": null,
"points": 30,
"status": "met",
"details": [],
"max_points": 30
},
{
"key": "documentation_directory",
"name": "Documentation directory",
"detail": null,
"points": 25,
"status": "met",
"details": [],
"max_points": 25
},
{
"key": "documentation_homepage_site",
"name": "Documentation / homepage site",
"detail": "https://swesmith.com/",
"points": 15,
"status": "met",
"details": [],
"max_points": 15
},
{
"key": "repository_description",
"name": "Repository description",
"detail": null,
"points": 10,
"status": "met",
"details": [],
"max_points": 10
},
{
"key": "topics",
"name": "Topics",
"detail": "4 topics",
"points": 10,
"status": "met",
"details": [
{
"code": "topics_count",
"params": {
"count": 4
}
}
],
"max_points": 10
},
{
"key": "wiki",
"name": "Wiki",
"detail": null,
"points": 10,
"status": "met",
"details": [],
"max_points": 10
}
]
}
],
"description": "Are baseline engineering and documentation practices in place?"
},
{
"key": "security",
"band": "at_risk",
"name": "Security",
"value": 49,
"weight": 0.16,
"metrics": [
{
"key": "security_posture",
"band": "at_risk",
"name": "Security posture",
"note": "Excluded from scoring (no data or not applicable): Packaging, Signed-Releases. Remaining weights renormalized.",
"notes": [
{
"code": "excluded_no_data",
"params": {
"components": [
"packaging",
"signed_releases"
]
}
},
{
"code": "weights_renormalized",
"params": {}
}
],
"value": 49,
"inputs": {
"source": "openssf_scorecard",
"checks_evaluated": 16,
"scorecard_version": "v5.5.0",
"checks_inconclusive": 2,
"scorecard_aggregate": 4.9
},
"components": [
{
"key": "binary_artifacts",
"name": "Binary-Artifacts",
"detail": "no binaries found in the repo",
"points": 7.5,
"status": "met",
"details": [],
"max_points": 7.5
},
{
"key": "branch_protection",
"name": "Branch-Protection",
"detail": "branch protection not enabled on development/release branches",
"points": 0,
"status": "missed",
"details": [],
"max_points": 7.5
},
{
"key": "ci_tests",
"name": "CI-Tests",
"detail": "25 out of 26 merged PRs checked by a CI test -- score normalized to 9",
"points": 2.2,
"status": "partial",
"details": [],
"max_points": 2.5
},
{
"key": "cii_best_practices",
"name": "CII-Best-Practices",
"detail": "no effort to earn an OpenSSF best practices badge detected",
"points": 0,
"status": "missed",
"details": [],
"max_points": 2.5
},
{
"key": "code_review",
"name": "Code-Review",
"detail": "Found 19/25 approved changesets -- score normalized to 7",
"points": 5.2,
"status": "partial",
"details": [],
"max_points": 7.5
},
{
"key": "contributors",
"name": "Contributors",
"detail": "project has 26 contributing companies or organizations",
"points": 2.5,
"status": "met",
"details": [],
"max_points": 2.5
},
{
"key": "dangerous_workflow",
"name": "Dangerous-Workflow",
"detail": "no dangerous workflow patterns detected",
"points": 10,
"status": "met",
"details": [],
"max_points": 10
},
{
"key": "dependency_update_tool",
"name": "Dependency-Update-Tool",
"detail": "update tool detected",
"points": 7.5,
"status": "met",
"details": [],
"max_points": 7.5
},
{
"key": "fuzzing",
"name": "Fuzzing",
"detail": "project is not fuzzed",
"points": 0,
"status": "missed",
"details": [],
"max_points": 5
},
{
"key": "license",
"name": "License",
"detail": "license file detected",
"points": 2.5,
"status": "met",
"details": [],
"max_points": 2.5
},
{
"key": "maintained",
"name": "Maintained",
"detail": "0 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 0",
"points": 0,
"status": "missed",
"details": [],
"max_points": 7.5
},
{
"key": "packaging",
"name": "Packaging",
"detail": "packaging workflow not detected",
"points": 0,
"status": "excluded",
"details": [
{
"code": "no_data",
"params": {}
}
],
"max_points": 5
},
{
"key": "pinned_dependencies",
"name": "Pinned-Dependencies",
"detail": "dependency not pinned by hash detected -- score normalized to 0",
"points": 0,
"status": "missed",
"details": [],
"max_points": 5
},
{
"key": "sast",
"name": "SAST",
"detail": "SAST tool is not run on all commits -- score normalized to 0",
"points": 0,
"status": "missed",
"details": [],
"max_points": 5
},
{
"key": "security_policy",
"name": "Security-Policy",
"detail": "security policy file not detected",
"points": 0,
"status": "missed",
"details": [],
"max_points": 5
},
{
"key": "signed_releases",
"name": "Signed-Releases",
"detail": "no releases found",
"points": 0,
"status": "excluded",
"details": [
{
"code": "no_data",
"params": {}
}
],
"max_points": 7.5
},
{
"key": "token_permissions",
"name": "Token-Permissions",
"detail": "detected GitHub workflow tokens with excessive permissions",
"points": 0,
"status": "missed",
"details": [],
"max_points": 7.5
},
{
"key": "vulnerabilities",
"name": "Vulnerabilities",
"detail": "0 existing vulnerabilities detected",
"points": 7.5,
"status": "met",
"details": [],
"max_points": 7.5
}
]
},
{
"key": "high_risk_jurisdiction_exposure",
"band": "excellent",
"name": "High-Risk Jurisdiction Exposure",
"note": "Only high-confidence self-published location evidence affects this multiplier. Ambiguous matches are review-only; country evidence is not proof of nationality, citizenship, legal registration, malicious intent, or sanctions status.",
"notes": [
{
"code": "jurisdiction_evidence_limits",
"params": {}
}
],
"value": 100,
"inputs": {
"meaning": "self-published location evidence; not nationality or citizenship",
"red_flag": false,
"exposures": [],
"policy_countries": [
"Russia",
"Iran",
"North Korea"
],
"review_only_matches": 0,
"assessed_self_published_locations": 14
},
"components": [
{
"key": "policy_exposure_multiplier",
"name": "Policy exposure multiplier",
"detail": "no confirmed policy-scope location match",
"points": 100,
"status": "met",
"details": [
{
"code": "jurisdiction_no_match",
"params": {}
}
],
"max_points": 100
}
]
}
],
"description": "Are visible security and supply-chain practices strong, with no malicious dependency and no unresolved high-risk jurisdiction exposure?"
},
{
"key": "ai_readiness",
"band": "moderate",
"name": "AI Readiness",
"value": 51,
"weight": 0,
"metrics": [
{
"key": "ai_agent_context",
"band": "moderate",
"name": "Agent context & guidance",
"note": null,
"notes": [],
"value": 50,
"inputs": {
"has_llms_txt": false,
"legible_history_share": 0.61,
"agent_instruction_files": [
"AGENTS.md"
],
"agent_instruction_max_bytes": 194
},
"components": [
{
"key": "agent_instructions",
"name": "Agent instructions",
"detail": "AGENTS.md (stub)",
"points": 18,
"status": "partial",
"details": [
{
"code": "file_list",
"params": {
"files": "AGENTS.md"
}
},
{
"code": "agent_instructions_stub",
"params": {}
}
],
"max_points": 45
},
{
"key": "machine_readable_docs_llms_txt",
"name": "Machine-readable docs (llms.txt)",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 15
},
{
"key": "legible_commit_history",
"name": "Legible commit history",
"detail": "50 of 82 human commits state their intent (structured subject or explanatory body)",
"points": 32.5,
"status": "partial",
"details": [
{
"code": "legible_history",
"params": {
"legible": 50,
"sampled": 82
}
}
],
"max_points": 40
}
]
},
{
"key": "ai_verify_loop",
"band": "moderate",
"name": "Verify loop (build / test / typecheck)",
"note": null,
"notes": [],
"value": 51,
"inputs": {
"has_nix": false,
"has_tests": true,
"lockfiles": [],
"has_dockerfile": false,
"typed_language": false,
"bootstrap_files": [],
"has_devcontainer": false,
"has_linter_config": true,
"typecheck_configs": [],
"agent_commit_share": 0.05,
"toolchain_manifests": [],
"dependency_bot_commit_share": 0.06
},
"components": [
{
"key": "one_command_bootstrap",
"name": "One-command bootstrap",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 18
},
{
"key": "automated_tests",
"name": "Automated tests",
"detail": null,
"points": 22,
"status": "met",
"details": [],
"max_points": 22
},
{
"key": "lint_format_config",
"name": "Lint / format config",
"detail": null,
"points": 11,
"status": "met",
"details": [],
"max_points": 11
},
{
"key": "static_type_checking",
"name": "Static type checking",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 11
},
{
"key": "reproducible_environment",
"name": "Reproducible environment",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 10
},
{
"key": "demonstrated_agent_practice",
"name": "Demonstrated agent practice",
"detail": "5 of the last 100 commits agent-authored or agent-credited",
"points": 10,
"status": "met",
"details": [
{
"code": "agent_authored_commits",
"params": {
"count": 5,
"sampled": 100
}
}
],
"max_points": 10
},
{
"key": "automated_maintenance",
"name": "Automated maintenance",
"detail": "6 of the last 100 commits are automated dependency updates",
"points": 8,
"status": "met",
"details": [
{
"code": "dependency_bot_commits",
"params": {
"count": 6,
"sampled": 100
}
}
],
"max_points": 8
},
{
"key": "openssf_scorecard_pinned_dependencies",
"name": "OpenSSF Scorecard: Pinned-Dependencies",
"detail": "dependency not pinned by hash detected -- score normalized to 0",
"points": 0,
"status": "missed",
"details": [],
"max_points": 10
}
]
},
{
"key": "ai_code_legibility",
"band": "moderate",
"name": "Code legibility for models",
"note": null,
"notes": [],
"value": 54,
"inputs": {
"primary_language": "Python",
"largest_source_bytes": 276830,
"source_files_sampled": 223,
"oversized_source_files": 5
},
"components": [
{
"key": "type_checkable_code",
"name": "Type-checkable code",
"detail": "Python without a type-check config",
"points": 0,
"status": "missed",
"details": [
{
"code": "no_typecheck_config_language",
"params": {
"language": "Python"
}
}
],
"max_points": 45
},
{
"key": "manageable_file_sizes",
"name": "Manageable file sizes",
"detail": "5/223 source files over 60KB",
"points": 53.8,
"status": "partial",
"details": [
{
"code": "oversized_source_files",
"params": {
"kb": 60,
"sampled": 223,
"oversized": 5
}
}
],
"max_points": 55
}
]
}
],
"description": "How well is the repo equipped to be developed and maintained with AI coding agents? An independent, experimental badge — weight 0.0, so it is surfaced on its own and does not affect the overall health score."
}
],
"metrics_version": "1.13.0"
},
"warnings": [
"Star history unavailable: GitHub GraphQL error: Resource not accessible by personal access token"
],
"report_type": "repository",
"generated_at": "2026-07-25T08:58:33.400396Z",
"schema_version": "0.27.0",
"badge_url": "https://raw.githubusercontent.com/inspect-software/badges/main/v1/s/SWE-bench/SWE-smith.svg",
"full_name": "SWE-bench/SWE-smith",
"license_state": "standard",
"license_spdx": "MIT"
}Scores are signals, not warranties. They reflect publicly visible practices on GitHub — not a code audit, and not a security guarantee.
Missing data is excluded and weights renormalized, never scored as zero. Methodology is versioned and open: metrics v1.13.0, schema v0.27.0 — full methodology · metrics wiki.
How one result sits in the wider record: aggregate statistics — PyPI.