Public recordModerate
Software health reportschema 0.26.0 · metrics 1.13.0 · 2026-07-22 04:00 UTC
tiktoken-go / tokenizer
Pure Go implementation of OpenAI's tiktoken tokenizer
tiktoken-go/tokenizer holds a health index of 54 out of 100, placing it in the Moderate band. It scores highest on Vitality (64/100) and lowest on Security (35/100). It was last updated 4 days ago. A single contributor accounts for most of its recent work.
54
overall / 100Software health index
Metrics are grouped into weighted categories on one standardized 1–100 scale. Overall starts as their weighted mean; when public evidence triggers the High-Risk Jurisdiction Policy, the rating is adjusted and receives an At risk ceiling of 49. AI Readiness sits outside the overall score.
54
Excellent85-100Exemplary; meets essentially all checked criteria
Good70-84Healthy; minor gaps
Moderate50-69Acceptable with notable gaps; review recommended
At risk30-49Significant weaknesses; adoption warrants caution
Critical1-29Severe problems (abandoned, single-maintainer, no hygiene)
Score profile
Each axis is a category. The shape matters more than the average — a healthy subject fills the whole shape, while a spike-and-crater profile means strength in one dimension is masking risk in another.
Ownership
tiktoken-goOrganization
5 followers1 public reposince Apr 2023
This repository is backed by an organization — shared, accountable stewardship that can outlive any single maintainer.
Package ecosystems
| Registry | Package | Version | Downloads / mo | Versions | Last publish |
|---|---|---|---|---|---|
| Go | github.com/tiktoken-go/tokenizer | v0.8.1 | - | 16 | 4 days ago |
Metrics by category
Vitality
Is the project alive — is code being written and are releases shipping?
64
Development activity
46At risk
How it's scored
| 36/36 | Push recency — last push 4 days ago |
| 2.1/36 | Commit cadence — 3/52 weeks with commits |
| 7/18 | Commit volume — 5 commits in the last year |
| 1/10 | OpenSSF Scorecard: Maintained — 2 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 1 |
Inputs used
| commits_last_year | 5 |
| human_commit_share | 1 |
| days_since_last_push | 4 |
| active_weeks_last_year | 3 |
Release discipline
92Excellent
How it's scored
| 27/27 | Ships releases — 13 releases published |
| 36/36 | Release recency — latest release 4 days ago |
| 19.8/27 | Release cadence — a release every ~66.9 days |
| 0/10 | OpenSSF Scorecard: Signed-Releases — no data |
Inputs used
| releases_count | 13 |
| latest_release_tag | v0.8.1 |
| releases_from_tags | no |
| days_since_latest_release | 4 |
| mean_days_between_releases | 66.9 |
Excluded from scoring (no data or not applicable): OpenSSF Scorecard: Signed-Releases. Remaining weights renormalized.
Community & Adoption
Does the project have users, downloads, attention, and a welcoming setup for contributors?
55
Popularity & adoption
59Moderate
How it's scored
| 42.9/60 | Stars — 445 stars |
| 12.4/25 | Forks — 32 forks |
| 3.9/15 | Watchers — 6 watchers |
Inputs used
| forks | 32 |
| stars | 445 |
| watchers | 6 |
| growth_state | organic |
| growth_factor_pct | 100 |
Community health
50Moderate
How it's scored
| 22.5/22.5 | README |
| 22.5/22.5 | License — recognized license (MIT) |
| 0/18 | CONTRIBUTING guide |
| 0/13.5 | Code of conduct |
| 0/7.2 | Issue template |
| 0/6.3 | PR template |
Inputs used
| has_readme | yes |
| has_license | yes |
| has_contributing | no |
| has_issue_template | no |
| has_code_of_conduct | no |
| has_pull_request_template | no |
Sustainability & Governance
Will the project survive its people — bus factor, responsiveness, who backs it, and package upkeep?
60
Maintainer resilience (bus factor)
29Critical
How it's scored
| 9/54 | Bus factor — 1 contributor(s) cover half of all commits |
| 7.7/22.5 | Commit distribution — top contributor authored 66% of commits |
| 12.2/13.5 | Contributor breadth — 9 contributors |
| 0/10 | OpenSSF Scorecard: Contributors — project has 0 contributing companies or organizations -- score normalized to 0 |
Inputs used
| bus_factor | 1 |
| contributors_sampled | 9 |
| top_contributor_share | 0.658 |
How it's scored
| 39.6/46.8 | Issue resolution — 85% of issues closed |
| 34.8/38.3 | PR acceptance — 10/11 decided PRs merged |
| 7.5/15 | OpenSSF Scorecard: Code-Review — Found 8/16 approved changesets -- score normalized to 5 |
Inputs used
| merged_prs | 10 |
| open_issues | 2 |
| closed_issues | 11 |
| issue_closed_ratio | 0.846 |
| closed_unmerged_prs | 1 |
Ownership & stewardship
44At risk
How it's scored
| 30/30 | Ownership backing — organization-owned |
| 0/20 | Verified domain |
| 5.6/25 | Owner reach — 5 followers of tiktoken-go |
| 8.8/25 | Track record — 1 public repos, account ~3 yr old |
Inputs used
| followers | 5 |
| owner_type | Organization |
| is_verified | — |
| owner_login | tiktoken-go |
| public_repos | 1 |
| account_age_days | 1,202 |
Package maintenance
100Excellent
How it's scored
| 25/25 | Published & resolvable — 1 package(s) on go |
| 35/35 | Publish recency — latest publish 4 days ago |
| 20/20 | Version history — 16 published versions |
| 20/20 | Not deprecated — active, not deprecated or yanked |
Inputs used
| packages | github.com/tiktoken-go/tokenizer |
| ecosystems | go |
| any_deprecated | no |
| min_days_since_publish | 4 |
Engineering Quality
Are baseline engineering and documentation practices in place?
50
Engineering practices
50Moderate
How it's scored
| 24/24 | CI workflows — 1 workflow(s) |
| 24/24 | Tests present |
| 0/16 | Linter config |
| 0/9.6 | Pre-commit hooks |
| 0/6.4 | .editorconfig |
| 2/20 | OpenSSF Scorecard: CI-Tests — 1 out of 8 merged PRs checked by a CI test -- score normalized to 1 |
Inputs used
| has_ci | yes |
| has_tests | yes |
| has_editorconfig | no |
| has_linter_config | no |
| has_precommit_config | no |
Documentation
50Moderate
How it's scored
| 30/30 | README |
| 0/25 | Documentation directory |
| 0/15 | Documentation / homepage site |
| 10/10 | Repository description |
| 0/10 | Topics |
| 10/10 | Wiki |
Inputs used
| topics | — |
| has_wiki | yes |
| homepage | — |
| has_readme | yes |
| has_docs_dir | no |
| has_description | yes |
Security
Are visible security and supply-chain practices strong, without unresolved high-risk jurisdiction exposure?
35
Security posture
35At risk
How it's scored
| 7.5/7.5 | Binary-Artifacts — no binaries found in the repo |
| 0/7.5 | Branch-Protection — branch protection not enabled on development/release branches |
| 0.2/2.5 | CI-Tests — 1 out of 8 merged PRs checked by a CI test -- score normalized to 1 |
| 0/2.5 | CII-Best-Practices — no effort to earn an OpenSSF best practices badge detected |
| 3.8/7.5 | Code-Review — Found 8/16 approved changesets -- score normalized to 5 |
| 0/2.5 | Contributors — project has 0 contributing companies or organizations -- score normalized to 0 |
| 10/10 | Dangerous-Workflow — no dangerous workflow patterns detected |
| 0/7.5 | Dependency-Update-Tool — no update tool detected |
| 0/5 | Fuzzing — project is not fuzzed |
| 2.5/2.5 | License — license file detected |
| 0.8/7.5 | Maintained — 2 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 1 |
| 0/5 | Packaging — no data |
| 0/5 | Pinned-Dependencies — dependency not pinned by hash detected -- score normalized to 0 |
| 0/5 | SAST — SAST tool is not run on all commits -- score normalized to 0 |
| 0/5 | Security-Policy — security policy file not detected |
| 0/7.5 | Signed-Releases — no data |
| 0/7.5 | Token-Permissions — detected GitHub workflow tokens with excessive permissions |
| 7.5/7.5 | Vulnerabilities — 0 existing vulnerabilities detected |
Inputs used
| source | openssf_scorecard |
| checks_evaluated | 16 |
| scorecard_version | v5.5.0 |
| checks_inconclusive | 2 |
| scorecard_aggregate | 3.5 |
Excluded from scoring (no data or not applicable): packaging, signed_releases. Remaining weights renormalized.
AI Readiness
How well is the repo equipped to be developed and maintained with AI coding agents? An independent, experimental badge — weight 0.0, so it is surfaced on its own and does not affect the overall health score.
51
Agent context & guidance
27Critical
How it's scored
| 0/45 | Agent instructions — no CLAUDE.md / AGENTS.md / editor rules |
| 0/15 | Machine-readable docs (llms.txt) |
| 26.7/40 | Legible commit history — 22 of 44 human commits state their intent (structured subject or explanatory body) |
Inputs used
| has_llms_txt | no |
| legible_history_share | 0.5 |
| agent_instruction_files | — |
| agent_instruction_max_bytes | — |
Verify loop (build / test / typecheck)
56Moderate
How it's scored
| 12.6/18 | One-command bootstrap — go.mod (toolchain convention, no task runner) |
| 22/22 | Automated tests |
| 0/11 | Lint / format config |
| 11/11 | Static type checking — Go (statically typed) |
| 10/10 | Reproducible environment — lockfile |
| 0/10 | Demonstrated agent practice — no agent-authored commits among the last 44 |
| 0/8 | Automated maintenance — no automated dependency updates observed |
| 0/10 | OpenSSF Scorecard: Pinned-Dependencies — dependency not pinned by hash detected -- score normalized to 0 |
Inputs used
| has_nix | no |
| has_tests | yes |
| lockfiles | go.sum |
| has_dockerfile | no |
| typed_language | yes |
| bootstrap_files | — |
| has_devcontainer | no |
| has_linter_config | no |
| typecheck_configs | — |
| agent_commit_share | 0 |
| toolchain_manifests | go.mod |
| dependency_bot_commit_share | 0 |
Code legibility for models
87Excellent
How it's scored
| 45/45 | Type-checkable code — Go (statically typed) |
| 42.1/55 | Manageable file sizes — 4/17 source files over 60KB |
Inputs used
| primary_language | Go |
| largest_source_bytes | 7,852,594 |
| source_files_sampled | 17 |
| oversized_source_files | 4 |
Key facts
445GitHub stars
9contributors
5commits, last 12 months
4days since last push
13releases
1bus factor
2open issues
Gopackage ecosystems
Data collection warnings
- GitHub dependency-graph SBOM unavailable (404); the dependency graph may be disabled for this repository
More detail
Star and fork history 445 ★ / 32 ⇿
445Stars
32Forks
13Releases
When each star and fork was added, collected from GitHub and bucketed by day. Cumulative growth sits directly above the daily additions it is made of, so the two read against each other: steady organic accretion looks nothing like an abrupt, short-lived burst. Where that difference is measurable, it is reported as growth authenticity.
Major 0Minor 6Patch 7
Each point covers 4 days.
OpenSSF Scorecard 3.5 / 10
3.5aggregate
Independent, tool-agnostic security assessment from the open-source OpenSSF Scorecard. Each check rewards a security practice, not a specific vendor's tool. Checks Scorecard could not determine are marked n/a and excluded from the security score (never counted as zero).
| 10 | Binary-Artifacts | no binaries found in the repo |
| 0 | Branch-Protection | branch protection not enabled on development/release branches |
| 1 | CI-Tests | 1 out of 8 merged PRs checked by a CI test -- score normalized to 1 |
| 0 | CII-Best-Practices | no effort to earn an OpenSSF best practices badge detected |
| 5 | Code-Review | Found 8/16 approved changesets -- score normalized to 5 |
| 0 | Contributors | project has 0 contributing companies or organizations -- score normalized to 0 |
| 10 | Dangerous-Workflow | no dangerous workflow patterns detected |
| 0 | Dependency-Update-Tool | no update tool detected |
| 0 | Fuzzing | project is not fuzzed |
| 10 | License | license file detected |
| 1 | Maintained | 2 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 1 |
| n/a | Packaging | packaging workflow not detected |
| 0 | Pinned-Dependencies | dependency not pinned by hash detected -- score normalized to 0 |
| 0 | SAST | SAST tool is not run on all commits -- score normalized to 0 |
| 0 | Security-Policy | security policy file not detected |
| n/a | Signed-Releases | no releases found |
| 0 | Token-Permissions | detected GitHub workflow tokens with excessive permissions |
| 10 | Vulnerabilities | 0 existing vulnerabilities detected |
Direct dependencies 1
| Registry | Package | Version constraint | Manifest |
|---|---|---|---|
| Go | github.com/dlclark/regexp2/v2 | v2.5.1 | go.mod |
All dependencies not collected
The resolved dependency set could not be collected for this report: GitHub dependency-graph SBOM unavailable (404); the dependency graph may be disabled for this repository
Raw JSON report machine-readable
{
"data": {
"repo": {
"topics": [],
"is_fork": false,
"size_kb": 4481,
"has_wiki": true,
"homepage": null,
"languages": {
"Go": 57424
},
"pushed_at": "2026-07-17T16:07:27Z",
"created_at": "2023-04-06T16:56:55Z",
"owner_type": "Organization",
"updated_at": "2026-07-21T17:35:27Z",
"description": "Pure Go implementation of OpenAI's tiktoken tokenizer",
"is_archived": false,
"is_disabled": false,
"license_spdx": "MIT",
"default_branch": "main",
"license_spdx_raw": "MIT",
"primary_language": "Go",
"significant_languages": [
"Go"
]
},
"owner": {
"blog": null,
"name": null,
"type": "Organization",
"login": "tiktoken-go",
"company": null,
"location": null,
"followers": 5,
"avatar_url": "https://avatars.githubusercontent.com/u/130084571?v=4",
"created_at": "2023-04-06T16:54:55Z",
"is_verified": null,
"public_repos": 1,
"account_age_days": 1202
},
"license": {
"state": "standard",
"spdx_id": "MIT",
"raw_spdx": "MIT",
"file_present": true,
"scorecard_found": true,
"profile_has_license": true
},
"activity": {
"releases": [
{
"tag": "v0.8.1",
"kind": "patch",
"published_at": "2026-07-17T16:07:27Z"
},
{
"tag": "v0.7.0",
"kind": "minor",
"published_at": "2025-08-08T21:09:51Z"
},
{
"tag": "v0.6.2",
"kind": "patch",
"published_at": "2025-05-09T22:25:24Z"
},
{
"tag": "v0.6.1",
"kind": "patch",
"published_at": "2025-03-27T14:38:34Z"
},
{
"tag": "v0.6.0",
"kind": "minor",
"published_at": "2025-03-07T14:52:12Z"
},
{
"tag": "v0.5.1",
"kind": "patch",
"published_at": "2025-02-25T15:57:06Z"
},
{
"tag": "v0.5.0",
"kind": "minor",
"published_at": "2025-02-17T20:46:57Z"
},
{
"tag": "v0.4.0",
"kind": "minor",
"published_at": "2025-02-12T23:00:11Z"
},
{
"tag": "v0.3.0",
"kind": "minor",
"published_at": "2025-01-28T18:45:20Z"
},
{
"tag": "v0.2.1",
"kind": "patch",
"published_at": "2024-11-22T22:16:28Z"
},
{
"tag": "v0.2.0",
"kind": "minor",
"published_at": "2024-09-13T15:32:23Z"
},
{
"tag": "v0.1.1",
"kind": "patch",
"published_at": "2024-05-10T19:49:15Z"
},
{
"tag": "v0.0.8",
"kind": "patch",
"published_at": "2023-04-09T21:48:24Z"
}
],
"recent_commits": [
{
"oid": "ca39f5c7bff9edfc9d70012c5dfdf23b44f7971e",
"body": null,
"is_bot": false,
"headline": "🐛 Bugfix with generated regular expressions",
"author_name": "bluescreen10",
"author_login": "bluescreen10",
"committed_at": "2026-07-17T16:05:43Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "a7606a16d9e9235d97b6ad0e25f3237cbdb8b3cf",
"body": "With this commit we bumped to v2 of the regular\nexpression library.",
"is_bot": false,
"headline": "Use pre-compiled regexp for more performace",
"author_name": "bluescreen10",
"author_login": "bluescreen10",
"committed_at": "2026-05-25T19:03:43Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "629192d03fc5026d8d04030a93f4aee3b3458f65",
"body": "Added GPT-5 models",
"is_bot": false,
"headline": "Merge pull request #21 from ando-masaki/gpt-5-fix",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2025-08-08T21:09:07Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "3ac0d0b657027c27041af52613b31edd4f6b3420",
"body": "Update tokenizer.go",
"is_bot": false,
"headline": "Merge pull request #2 from ando-masaki/ando-masaki-patch-1",
"author_name": "ando-masaki",
"author_login": "ando-masaki",
"committed_at": "2025-08-08T02:53:48Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "5cc8c3ce694c2fc3b3f04e475b07e565a16edf57",
"body": null,
"is_bot": false,
"headline": "Update tokenizer.go",
"author_name": "ando-masaki",
"author_login": "ando-masaki",
"committed_at": "2025-08-08T02:52:01Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "07878231645aadb13fdea5ce1cd3603177a6ce42",
"body": "Add o3, o4-min, gpt-4.1",
"is_bot": false,
"headline": "Merge pull request #18 from ando-masaki/add-o3-and-o4-mini-fix",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2025-06-05T18:34:31Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "4e6fabdf3c93d28645d98f81403f21fdea81f863",
"body": "Add o3, o4-min, gpt-4.1",
"is_bot": false,
"headline": "Update tokenizer.go",
"author_name": "ando-masaki",
"author_login": "ando-masaki",
"committed_at": "2025-06-05T06:16:33Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "d82759aa66c5ab2fbc972525d7775141b0099065",
"body": "This reverts commit a26622ed5c27d9dd14a06ac61bb5e20a7cddd2ac.",
"is_bot": false,
"headline": "Revert \"small performance improvement\"",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2025-05-09T22:24:16Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "a26622ed5c27d9dd14a06ac61bb5e20a7cddd2ac",
"body": null,
"is_bot": false,
"headline": "small performance improvement",
"author_name": "Mariano Wahlmann",
"author_login": null,
"committed_at": "2025-03-27T14:28:02Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "e56ab3c0656c9d36f5a7483b5949f55ec8e280ac",
"body": null,
"is_bot": false,
"headline": "Remove defer, also small improvement using strings for bpe",
"author_name": "Mariano Wahlmann",
"author_login": null,
"committed_at": "2025-03-27T14:18:22Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "bc8ba39dc0bbee16968778c1a6e792e79b94f486",
"body": null,
"is_bot": false,
"headline": "🚀 feat: added support for O3-mini model",
"author_name": "Mariano Wahlmann",
"author_login": null,
"committed_at": "2025-03-07T14:50:38Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "e4624aa318a9fce453eebc18536c2f96b33e35c5",
"body": null,
"is_bot": false,
"headline": "🐛 problem with regexp generated regexp code dependencies",
"author_name": "Mariano Wahlmann",
"author_login": null,
"committed_at": "2025-02-25T15:55:43Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "8ac6fdfcb9cb83eb69eb62709090abf5a6fc6080",
"body": "feat: memory-efficient token counting functionality",
"is_bot": false,
"headline": "Merge pull request #13 from amalucelli/count",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2025-02-17T20:46:05Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "d47b68072e97bb7663b46f802b7a1835127b059a",
"body": "This uses iterators for the common logic and the function Count and Encode would do different things",
"is_bot": false,
"headline": "Refactored code to remove duplication",
"author_name": "Mariano Wahlmann",
"author_login": null,
"committed_at": "2025-02-13T17:50:18Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "cd27b29fe9bcefbfde7b860b81776f27132b15a7",
"body": null,
"is_bot": false,
"headline": "feat: add count method for tokens",
"author_name": "Alexandre Guimarães Malucelli",
"author_login": "amalucelli",
"committed_at": "2025-02-13T17:50:16Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "03e0e5f2a364bc0069e17f15107ad16ddd443355",
"body": "pre-compile regexp2 with regexp2cg",
"is_bot": false,
"headline": "Merge pull request #11 from terwey/regexp2cg",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2025-02-12T22:59:04Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "2e19c1421ebbcdd3d959ebd04fc8e44b70e64696",
"body": null,
"is_bot": false,
"headline": "Added a go generate command for the compiled regular expression",
"author_name": "Mariano Wahlmann",
"author_login": null,
"committed_at": "2025-02-12T22:57:06Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "025023d0144d79b96726319c4375e21a8b80837e",
"body": "Add support of chatgpt-4o-latest model",
"is_bot": false,
"headline": "Merge pull request #12 from DenisPalnitsky/chatgpt-4o-support",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2025-01-28T18:43:40Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "fdcd81a21a133e40754a66a8346439cfa3b87350",
"body": null,
"is_bot": false,
"headline": "support of chatgpt-4o-latest",
"author_name": "Denis Palnitsky",
"author_login": "DenisPalnitsky",
"committed_at": "2025-01-24T18:29:54Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "45878105be78c8394b2cbd428bbbe2049640029d",
"body": null,
"is_bot": false,
"headline": "pre-compile regexp2 with regexp2cg",
"author_name": "Yorick Terweijden",
"author_login": "terwey",
"committed_at": "2024-12-04T20:56:37Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "2ad5811ad0b7d7c869c93cc33f4f161390464fe8",
"body": "Added o1-preview and o1-mini",
"is_bot": false,
"headline": "Merge pull request #10 from ando-masaki/o1-preview-and-mini-fix",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2024-11-22T22:12:55Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "60153be10d9a8246e1623eb6cf2e6e269e34bba3",
"body": null,
"is_bot": false,
"headline": "Simplified model selection",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2024-11-22T22:10:42Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "1ca7752d6d32ae47d743f933ac9c8783dd4a26c6",
"body": null,
"is_bot": false,
"headline": "Update tokenizer.go",
"author_name": "ando-masaki",
"author_login": "ando-masaki",
"committed_at": "2024-11-19T03:42:24Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "60becaa41f4a53a291b43420c9c88dac330880b5",
"body": "Update tokenizer.go",
"is_bot": false,
"headline": "Merge pull request #8 from GOOD21/main",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2024-11-07T14:32:20Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "ad6a208b2232926024e7f642fd617ff8e4ca9d71",
"body": null,
"is_bot": false,
"headline": "Update tokenizer.go",
"author_name": "GOOD21",
"author_login": "GOOD21",
"committed_at": "2024-11-07T10:09:50Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "b74b6b8b3a6705a0d7cb58ef9df45f6aba8404a4",
"body": "Fix panic caused by model name length",
"is_bot": false,
"headline": "Update tokenizer.go",
"author_name": "GOOD21",
"author_login": "GOOD21",
"committed_at": "2024-11-07T09:48:04Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "53141e47ff8b62f2b1f97a11f024f3cb515d09b7",
"body": "This commit should add support for all OpenAI\ncurrently available models.",
"is_bot": false,
"headline": "Added support for GPT4o and o200k_base models",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2024-09-13T15:28:04Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "7f65fe7d582fc7772cd74271d20dfad52fb8611a",
"body": "Generate embedded vocabulary maps lazily",
"is_bot": false,
"headline": "Merge pull request #6 from espadolini/lazy-load",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2024-05-10T19:46:47Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "93a91392ef75e411fc080030b02c448d09c8250e",
"body": null,
"is_bot": false,
"headline": "Format the code after codegen",
"author_name": "Edoardo Spadolini",
"author_login": "espadolini",
"committed_at": "2024-05-10T18:27:19Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "944cac3e81265a66bea9a725714c1698565e346c",
"body": null,
"is_bot": false,
"headline": "Generate embedded vocabulary maps lazily",
"author_name": "Edoardo Spadolini",
"author_login": "espadolini",
"committed_at": "2024-05-09T09:40:58Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "70d9b595feda6b98d6339ee4bab5f7a0c7c997f0",
"body": "Update alternatives",
"is_bot": false,
"headline": "Merge pull request #3 from philippgille/patch-1",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-19T13:07:29Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "4f793dc73f725f0c821ada3bcc30a84659850a5a",
"body": null,
"is_bot": false,
"headline": "Update alternatives",
"author_name": "Philipp Gillé",
"author_login": "philippgille",
"committed_at": "2023-04-18T20:32:25Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "94e23ef2a7f6a2832126c7ace1001258220d3cd5",
"body": "Go's regexp doesn't allow for look-around so had to re-write the\nregexp but that had a bug with multiple spaces. Switched to regexp2\nthat supports the native tiktoken regexp",
"is_bot": false,
"headline": "🐛 fixed bug with spaces",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-11T13:19:07Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "876e66b44d595542c8a0251b0a5bbadfdbb9b205",
"body": "before it was not possible to install the cli",
"is_bot": false,
"headline": "🐛 moved cli into its own folder",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-11T02:28:56Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "4868b0b5dc763f5886b8956dd4a9c9749aa7ae7e",
"body": "Renamed variables struct to reflect what they do a little better.",
"is_bot": false,
"headline": "🪄 clean up the bpe algorithm",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-09T21:43:58Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "12a3097aa3fad93104c64a7e774d974879e42927",
"body": "📘 fixed usage example in readme",
"is_bot": false,
"headline": "Merge pull request #1 from MarvinJWendt/patch-1",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-08T15:00:03Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "2886eb0986a0e4fe3220a8430ef4ccbe1a0560b5",
"body": "- Indents in the import block were not consistent\r\n- `()` was missing in the main function\r\n- `token` was unused",
"is_bot": false,
"headline": "🛂 fixed usage example in readme",
"author_name": "Marvin Wendt",
"author_login": "MarvinJWendt",
"committed_at": "2023-04-08T02:07:20Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "859c61e7dc23560fefb62403e163a1afd80814cf",
"body": "After comfirming the expected behavior changed tests to match OpenAI's\nbehavior. It should now work as expected.",
"is_bot": false,
"headline": "🛂 fixed tests",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-07T15:06:52Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "1b538b580912c6e4e1c8e8ba72e82aece232bd9f",
"body": "this would be helpful for other people to test and use the tokenizer.\nEven from other languages because this can be distributed as a single\nbinary.",
"is_bot": false,
"headline": "🌟 added a cli to use the tokenizer via the command Lines",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-06T21:56:04Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "79aa80d63f09a0d74b360df077d204f7755bc714",
"body": "The internal generate command was using log.Fatal instead of the\ncorrect log.Fatalf.",
"is_bot": false,
"headline": "🐛 fixed an issue in the internal generate command",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-06T20:58:15Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "ea2b0eea6b85ee333cdf235e71442c8196c742ad",
"body": "Right now emits the same output as OpenAI's tiktoken but I'm not sure\nthat's the right output.",
"is_bot": false,
"headline": "🤫 temporarily skipping failing test until I confirm the expected output",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-06T20:10:08Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "2e69c6a4db9547039ca536c281a73c350a20b57b",
"body": "This will make it more obvious the status of the build when reading the landing page",
"is_bot": false,
"headline": "📗 adding test results badge to README.md",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-06T20:02:57Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "a4eb7ed16c9dbe62ab99a614157759f64fd99dac",
"body": "This should help verify future releases. The test are not passing as of now because I'm not sure tiktoken's (original python library) output is correct.",
"is_bot": false,
"headline": "🛂 add github actions",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-06T19:58:32Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "72848e8b530ee3bcb5f464232e39d905e06109a4",
"body": "This implementations should work for most of OpenAI's model. However\nWe are not yet dealing with the special tokens and/or gpt2 encoding.",
"is_bot": false,
"headline": "🚢 Initial Release",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-06T19:53:05Z",
"body_truncated": false,
"is_coding_agent": false
}
],
"releases_count": 13,
"commits_last_year": 5,
"latest_release_at": "2026-07-17T16:07:27Z",
"latest_release_tag": "v0.8.1",
"releases_from_tags": false,
"days_since_last_push": 4,
"active_weeks_last_year": 3,
"days_since_latest_release": 4,
"mean_days_between_releases": 66.9
},
"community": {
"has_readme": true,
"has_license": true,
"has_description": true,
"has_contributing": false,
"health_percentage": 50,
"has_issue_template": false,
"has_code_of_conduct": false,
"has_pull_request_template": false
},
"ecosystem": {
"packages": [
{
"name": "github.com/tiktoken-go/tokenizer",
"exists": true,
"license": null,
"keywords": [],
"ecosystem": "go",
"matches_repo": true,
"registry_url": "https://pkg.go.dev/github.com/tiktoken-go/tokenizer",
"is_deprecated": false,
"latest_version": "v0.8.1",
"repository_url": "https://github.com/tiktoken-go/tokenizer",
"versions_count": 16,
"total_downloads": null,
"dependents_count": null,
"deprecation_note": null,
"maintainers_count": null,
"monthly_downloads": null,
"first_published_at": null,
"latest_published_at": "2026-07-17T16:05:43Z",
"latest_version_yanked": null,
"days_since_latest_publish": 4
}
]
},
"popularity": {
"forks": 32,
"stars": 445,
"watchers": 6,
"fork_history": {
"days": [
{
"date": "2023-04-07",
"count": 2
},
{
"date": "2023-04-09",
"count": 1
},
{
"date": "2023-04-11",
"count": 1
},
{
"date": "2023-04-18",
"count": 1
},
{
"date": "2023-05-05",
"count": 1
},
{
"date": "2023-05-24",
"count": 1
},
{
"date": "2023-06-28",
"count": 1
},
{
"date": "2023-11-09",
"count": 1
},
{
"date": "2023-12-18",
"count": 1
},
{
"date": "2024-01-17",
"count": 1
},
{
"date": "2024-04-15",
"count": 1
},
{
"date": "2024-06-06",
"count": 1
},
{
"date": "2024-06-09",
"count": 1
},
{
"date": "2024-06-13",
"count": 1
},
{
"date": "2024-06-15",
"count": 1
},
{
"date": "2024-11-06",
"count": 1
},
{
"date": "2024-11-08",
"count": 1
},
{
"date": "2024-11-19",
"count": 1
},
{
"date": "2024-12-04",
"count": 1
},
{
"date": "2025-01-10",
"count": 1
},
{
"date": "2025-01-24",
"count": 1
},
{
"date": "2025-02-10",
"count": 1
},
{
"date": "2025-06-10",
"count": 1
},
{
"date": "2025-07-10",
"count": 1
},
{
"date": "2025-08-03",
"count": 1
},
{
"date": "2025-09-04",
"count": 1
},
{
"date": "2025-09-12",
"count": 1
},
{
"date": "2026-02-02",
"count": 1
},
{
"date": "2026-02-11",
"count": 1
},
{
"date": "2026-06-23",
"count": 1
},
{
"date": "2026-07-16",
"count": 1
}
],
"complete": true,
"collected": 32,
"total_forks": 32
},
"star_history": {
"days": [
{
"date": "2023-04-07",
"count": 23
},
{
"date": "2023-04-08",
"count": 18
},
{
"date": "2023-04-09",
"count": 7
},
{
"date": "2023-04-10",
"count": 6
},
{
"date": "2023-04-11",
"count": 28
},
{
"date": "2023-04-12",
"count": 20
},
{
"date": "2023-04-13",
"count": 6
},
{
"date": "2023-04-14",
"count": 1
},
{
"date": "2023-04-15",
"count": 2
},
{
"date": "2023-04-16",
"count": 4
},
{
"date": "2023-04-17",
"count": 9
},
{
"date": "2023-04-18",
"count": 4
},
{
"date": "2023-04-19",
"count": 2
},
{
"date": "2023-04-20",
"count": 1
},
{
"date": "2023-04-22",
"count": 2
},
{
"date": "2023-04-23",
"count": 1
},
{
"date": "2023-04-24",
"count": 2
},
{
"date": "2023-04-25",
"count": 1
},
{
"date": "2023-04-26",
"count": 2
},
{
"date": "2023-04-27",
"count": 2
},
{
"date": "2023-04-29",
"count": 2
},
{
"date": "2023-05-03",
"count": 2
},
{
"date": "2023-05-04",
"count": 2
},
{
"date": "2023-05-05",
"count": 1
},
{
"date": "2023-05-06",
"count": 1
},
{
"date": "2023-05-10",
"count": 1
},
{
"date": "2023-05-11",
"count": 1
},
{
"date": "2023-05-12",
"count": 2
},
{
"date": "2023-05-14",
"count": 1
},
{
"date": "2023-05-16",
"count": 1
},
{
"date": "2023-05-17",
"count": 2
},
{
"date": "2023-05-18",
"count": 1
},
{
"date": "2023-05-29",
"count": 1
},
{
"date": "2023-05-30",
"count": 2
},
{
"date": "2023-06-01",
"count": 2
},
{
"date": "2023-06-02",
"count": 1
},
{
"date": "2023-06-03",
"count": 1
},
{
"date": "2023-06-05",
"count": 1
},
{
"date": "2023-06-07",
"count": 1
},
{
"date": "2023-06-08",
"count": 1
},
{
"date": "2023-06-09",
"count": 2
},
{
"date": "2023-06-11",
"count": 1
},
{
"date": "2023-06-13",
"count": 1
},
{
"date": "2023-06-17",
"count": 1
},
{
"date": "2023-06-28",
"count": 1
},
{
"date": "2023-06-29",
"count": 1
},
{
"date": "2023-07-04",
"count": 1
},
{
"date": "2023-07-06",
"count": 1
},
{
"date": "2023-07-09",
"count": 1
},
{
"date": "2023-07-10",
"count": 2
},
{
"date": "2023-07-13",
"count": 2
},
{
"date": "2023-07-22",
"count": 2
},
{
"date": "2023-07-29",
"count": 1
},
{
"date": "2023-08-03",
"count": 1
},
{
"date": "2023-08-08",
"count": 1
},
{
"date": "2023-08-14",
"count": 1
},
{
"date": "2023-08-21",
"count": 1
},
{
"date": "2023-08-23",
"count": 2
},
{
"date": "2023-08-29",
"count": 1
},
{
"date": "2023-09-04",
"count": 1
},
{
"date": "2023-09-15",
"count": 1
},
{
"date": "2023-09-21",
"count": 1
},
{
"date": "2023-10-05",
"count": 1
},
{
"date": "2023-10-12",
"count": 1
},
{
"date": "2023-10-13",
"count": 1
},
{
"date": "2023-10-14",
"count": 1
},
{
"date": "2023-10-27",
"count": 1
},
{
"date": "2023-11-03",
"count": 1
},
{
"date": "2023-11-07",
"count": 1
},
{
"date": "2023-11-10",
"count": 1
},
{
"date": "2023-11-14",
"count": 2
},
{
"date": "2023-11-22",
"count": 1
},
{
"date": "2023-11-28",
"count": 1
},
{
"date": "2023-11-29",
"count": 1
},
{
"date": "2023-12-01",
"count": 1
},
{
"date": "2023-12-04",
"count": 1
},
{
"date": "2023-12-16",
"count": 1
},
{
"date": "2023-12-18",
"count": 1
},
{
"date": "2023-12-31",
"count": 1
},
{
"date": "2024-01-10",
"count": 1
},
{
"date": "2024-01-31",
"count": 1
},
{
"date": "2024-02-05",
"count": 1
},
{
"date": "2024-02-17",
"count": 1
},
{
"date": "2024-02-21",
"count": 1
},
{
"date": "2024-03-01",
"count": 1
},
{
"date": "2024-03-05",
"count": 1
},
{
"date": "2024-03-07",
"count": 1
},
{
"date": "2024-04-11",
"count": 1
},
{
"date": "2024-04-16",
"count": 1
},
{
"date": "2024-04-18",
"count": 1
},
{
"date": "2024-04-26",
"count": 1
},
{
"date": "2024-05-04",
"count": 1
},
{
"date": "2024-05-12",
"count": 1
},
{
"date": "2024-05-20",
"count": 1
},
{
"date": "2024-05-27",
"count": 1
},
{
"date": "2024-05-29",
"count": 1
},
{
"date": "2024-06-06",
"count": 1
},
{
"date": "2024-06-11",
"count": 7
},
{
"date": "2024-06-12",
"count": 6
},
{
"date": "2024-06-13",
"count": 9
},
{
"date": "2024-06-14",
"count": 2
},
{
"date": "2024-06-15",
"count": 2
},
{
"date": "2024-06-16",
"count": 1
},
{
"date": "2024-06-17",
"count": 2
},
{
"date": "2024-06-19",
"count": 2
},
{
"date": "2024-06-27",
"count": 1
},
{
"date": "2024-07-09",
"count": 1
},
{
"date": "2024-07-10",
"count": 1
},
{
"date": "2024-07-11",
"count": 1
},
{
"date": "2024-07-16",
"count": 1
},
{
"date": "2024-07-21",
"count": 1
},
{
"date": "2024-07-22",
"count": 1
},
{
"date": "2024-08-12",
"count": 1
},
{
"date": "2024-08-14",
"count": 1
},
{
"date": "2024-08-15",
"count": 1
},
{
"date": "2024-08-21",
"count": 1
},
{
"date": "2024-08-24",
"count": 1
},
{
"date": "2024-08-28",
"count": 1
},
{
"date": "2024-09-01",
"count": 1
},
{
"date": "2024-09-06",
"count": 1
},
{
"date": "2024-09-11",
"count": 1
},
{
"date": "2024-09-13",
"count": 1
},
{
"date": "2024-09-25",
"count": 1
},
{
"date": "2024-09-30",
"count": 1
},
{
"date": "2024-10-03",
"count": 1
},
{
"date": "2024-10-12",
"count": 1
},
{
"date": "2024-10-30",
"count": 1
},
{
"date": "2024-11-05",
"count": 3
},
{
"date": "2024-11-10",
"count": 1
},
{
"date": "2024-11-16",
"count": 2
},
{
"date": "2024-11-20",
"count": 1
},
{
"date": "2024-11-23",
"count": 1
},
{
"date": "2024-11-25",
"count": 3
},
{
"date": "2024-11-27",
"count": 1
},
{
"date": "2024-11-29",
"count": 1
},
{
"date": "2024-12-02",
"count": 1
},
{
"date": "2024-12-08",
"count": 1
},
{
"date": "2024-12-18",
"count": 1
},
{
"date": "2024-12-19",
"count": 2
},
{
"date": "2024-12-30",
"count": 1
},
{
"date": "2025-01-12",
"count": 1
},
{
"date": "2025-01-24",
"count": 1
},
{
"date": "2025-01-27",
"count": 1
},
{
"date": "2025-01-29",
"count": 2
},
{
"date": "2025-01-30",
"count": 2
},
{
"date": "2025-01-31",
"count": 1
},
{
"date": "2025-02-04",
"count": 1
},
{
"date": "2025-02-09",
"count": 1
},
{
"date": "2025-02-11",
"count": 2
},
{
"date": "2025-02-16",
"count": 1
},
{
"date": "2025-02-17",
"count": 1
},
{
"date": "2025-02-18",
"count": 1
},
{
"date": "2025-02-19",
"count": 2
},
{
"date": "2025-02-23",
"count": 1
},
{
"date": "2025-03-01",
"count": 1
},
{
"date": "2025-03-03",
"count": 2
},
{
"date": "2025-03-07",
"count": 3
},
{
"date": "2025-03-11",
"count": 2
},
{
"date": "2025-03-15",
"count": 1
},
{
"date": "2025-03-17",
"count": 2
},
{
"date": "2025-03-19",
"count": 1
},
{
"date": "2025-03-20",
"count": 2
},
{
"date": "2025-03-21",
"count": 1
},
{
"date": "2025-03-22",
"count": 1
},
{
"date": "2025-03-27",
"count": 1
},
{
"date": "2025-03-28",
"count": 1
},
{
"date": "2025-04-03",
"count": 1
},
{
"date": "2025-04-06",
"count": 1
},
{
"date": "2025-04-09",
"count": 1
},
{
"date": "2025-04-10",
"count": 1
},
{
"date": "2025-04-11",
"count": 2
},
{
"date": "2025-04-17",
"count": 1
},
{
"date": "2025-04-19",
"count": 1
},
{
"date": "2025-04-24",
"count": 1
},
{
"date": "2025-05-04",
"count": 1
},
{
"date": "2025-05-15",
"count": 1
},
{
"date": "2025-05-21",
"count": 1
},
{
"date": "2025-05-27",
"count": 1
},
{
"date": "2025-05-30",
"count": 2
},
{
"date": "2025-06-09",
"count": 1
},
{
"date": "2025-06-10",
"count": 2
},
{
"date": "2025-06-17",
"count": 2
},
{
"date": "2025-06-18",
"count": 1
},
{
"date": "2025-06-19",
"count": 1
},
{
"date": "2025-06-25",
"count": 2
},
{
"date": "2025-06-27",
"count": 1
},
{
"date": "2025-06-28",
"count": 1
},
{
"date": "2025-06-29",
"count": 1
},
{
"date": "2025-07-01",
"count": 1
},
{
"date": "2025-07-02",
"count": 1
},
{
"date": "2025-07-08",
"count": 2
},
{
"date": "2025-07-10",
"count": 1
},
{
"date": "2025-07-13",
"count": 1
},
{
"date": "2025-07-24",
"count": 1
},
{
"date": "2025-07-27",
"count": 1
},
{
"date": "2025-08-01",
"count": 2
},
{
"date": "2025-08-03",
"count": 1
},
{
"date": "2025-08-15",
"count": 1
},
{
"date": "2025-08-19",
"count": 2
},
{
"date": "2025-08-21",
"count": 1
},
{
"date": "2025-08-26",
"count": 1
},
{
"date": "2025-09-03",
"count": 1
},
{
"date": "2025-09-09",
"count": 2
},
{
"date": "2025-09-22",
"count": 1
},
{
"date": "2025-09-23",
"count": 1
},
{
"date": "2025-10-14",
"count": 2
},
{
"date": "2025-10-19",
"count": 1
},
{
"date": "2025-10-27",
"count": 1
},
{
"date": "2025-10-28",
"count": 1
},
{
"date": "2025-10-29",
"count": 1
},
{
"date": "2025-11-03",
"count": 1
},
{
"date": "2025-11-04",
"count": 1
},
{
"date": "2025-11-06",
"count": 2
},
{
"date": "2025-11-12",
"count": 1
},
{
"date": "2025-11-14",
"count": 1
},
{
"date": "2025-11-26",
"count": 1
},
{
"date": "2025-11-27",
"count": 3
},
{
"date": "2025-12-02",
"count": 3
},
{
"date": "2025-12-04",
"count": 1
},
{
"date": "2025-12-29",
"count": 2
},
{
"date": "2026-01-05",
"count": 1
},
{
"date": "2026-01-12",
"count": 1
},
{
"date": "2026-01-19",
"count": 2
},
{
"date": "2026-01-31",
"count": 1
},
{
"date": "2026-02-06",
"count": 1
},
{
"date": "2026-02-09",
"count": 2
},
{
"date": "2026-02-12",
"count": 1
},
{
"date": "2026-02-15",
"count": 2
},
{
"date": "2026-02-24",
"count": 1
},
{
"date": "2026-03-11",
"count": 1
},
{
"date": "2026-03-15",
"count": 1
},
{
"date": "2026-03-18",
"count": 1
},
{
"date": "2026-03-27",
"count": 1
},
{
"date": "2026-04-01",
"count": 1
},
{
"date": "2026-04-07",
"count": 1
},
{
"date": "2026-04-22",
"count": 1
},
{
"date": "2026-04-26",
"count": 1
},
{
"date": "2026-04-30",
"count": 1
},
{
"date": "2026-05-06",
"count": 1
},
{
"date": "2026-05-11",
"count": 1
},
{
"date": "2026-05-17",
"count": 1
},
{
"date": "2026-05-28",
"count": 1
},
{
"date": "2026-05-29",
"count": 1
},
{
"date": "2026-06-07",
"count": 1
},
{
"date": "2026-06-08",
"count": 1
},
{
"date": "2026-06-09",
"count": 1
},
{
"date": "2026-06-26",
"count": 1
},
{
"date": "2026-06-27",
"count": 1
},
{
"date": "2026-06-30",
"count": 1
},
{
"date": "2026-07-01",
"count": 1
},
{
"date": "2026-07-05",
"count": 1
},
{
"date": "2026-07-15",
"count": 1
},
{
"date": "2026-07-21",
"count": 1
}
],
"complete": true,
"collected": 445,
"total_stars": 445
},
"open_issues_and_prs": 3
},
"ai_readiness": {
"has_nix": false,
"example_dirs": [],
"has_llms_txt": false,
"has_dockerfile": false,
"has_mcp_signal": false,
"bootstrap_files": [],
"api_schema_files": [],
"has_devcontainer": false,
"typecheck_configs": [],
"toolchain_manifests": [
"go.mod"
],
"largest_source_bytes": 7852594,
"source_files_sampled": 17,
"oversized_source_files": 4,
"agent_instruction_files": [],
"agent_instruction_max_bytes": null
},
"dependencies": {
"manifests": [
"go.mod"
],
"advisories": {
"error": null,
"scope": null,
"source": null,
"findings": [],
"collected": false,
"malicious": [],
"truncated": false,
"by_severity": {},
"advisory_count": 0,
"affected_count": 0,
"assessed_count": 0,
"malicious_count": 0,
"assessed_package": null,
"unassessed_count": 0,
"direct_affected_count": 0
},
"ecosystems": [
"go"
],
"dependencies": [
{
"name": "github.com/dlclark/regexp2/v2",
"manifest": "go.mod",
"ecosystem": "go",
"version_constraint": "v2.5.1"
}
],
"all_dependencies": {
"error": "GitHub dependency-graph SBOM unavailable (404); the dependency graph may be disabled for this repository",
"source": null,
"packages": [],
"collected": false,
"truncated": false,
"total_count": null,
"direct_count": null,
"indirect_count": null
}
},
"maintainership": {
"issues": {
"open_prs": 1,
"merged_prs": 10,
"open_issues": 2,
"closed_ratio": 0.846,
"closed_issues": 11,
"closed_unmerged_prs": 1
},
"bus_factor": 1,
"bot_contributors": 0,
"top_contributors": [
{
"type": "User",
"login": "bluescreen10",
"commits": 25,
"avatar_url": "https://avatars.githubusercontent.com/u/86094?v=4"
},
{
"type": "User",
"login": "ando-masaki",
"commits": 4,
"avatar_url": "https://avatars.githubusercontent.com/u/4375029?v=4"
},
{
"type": "User",
"login": "espadolini",
"commits": 2,
"avatar_url": "https://avatars.githubusercontent.com/u/886075?v=4"
},
{
"type": "User",
"login": "GOOD21",
"commits": 2,
"avatar_url": "https://avatars.githubusercontent.com/u/3470273?v=4"
},
{
"type": "User",
"login": "amalucelli",
"commits": 1,
"avatar_url": "https://avatars.githubusercontent.com/u/9258376?v=4"
},
{
"type": "User",
"login": "DenisPalnitsky",
"commits": 1,
"avatar_url": "https://avatars.githubusercontent.com/u/154290?v=4"
},
{
"type": "User",
"login": "MarvinJWendt",
"commits": 1,
"avatar_url": "https://avatars.githubusercontent.com/u/31022056?v=4"
},
{
"type": "User",
"login": "philippgille",
"commits": 1,
"avatar_url": "https://avatars.githubusercontent.com/u/170670?v=4"
},
{
"type": "User",
"login": "terwey",
"commits": 1,
"avatar_url": "https://avatars.githubusercontent.com/u/1596124?v=4"
}
],
"contributors_sampled": 9,
"top_contributor_share": 0.658
},
"quality_signals": {
"has_ci": true,
"has_tests": true,
"ci_workflows": [
"go.yml"
],
"has_docs_dir": false,
"linter_configs": [],
"has_editorconfig": false,
"has_linter_config": false,
"has_precommit_config": false
},
"security_signals": {
"lockfiles": [
"go.sum"
],
"scorecard": {
"checks": [
{
"name": "Binary-Artifacts",
"score": 10,
"reason": "no binaries found in the repo",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#binary-artifacts"
},
{
"name": "Branch-Protection",
"score": 0,
"reason": "branch protection not enabled on development/release branches",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#branch-protection"
},
{
"name": "CI-Tests",
"score": 1,
"reason": "1 out of 8 merged PRs checked by a CI test -- score normalized to 1",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#ci-tests"
},
{
"name": "CII-Best-Practices",
"score": 0,
"reason": "no effort to earn an OpenSSF best practices badge detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#cii-best-practices"
},
{
"name": "Code-Review",
"score": 5,
"reason": "Found 8/16 approved changesets -- score normalized to 5",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#code-review"
},
{
"name": "Contributors",
"score": 0,
"reason": "project has 0 contributing companies or organizations -- score normalized to 0",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#contributors"
},
{
"name": "Dangerous-Workflow",
"score": 10,
"reason": "no dangerous workflow patterns detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#dangerous-workflow"
},
{
"name": "Dependency-Update-Tool",
"score": 0,
"reason": "no update tool detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#dependency-update-tool"
},
{
"name": "Fuzzing",
"score": 0,
"reason": "project is not fuzzed",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#fuzzing"
},
{
"name": "License",
"score": 10,
"reason": "license file detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#license"
},
{
"name": "Maintained",
"score": 1,
"reason": "2 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 1",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#maintained"
},
{
"name": "Packaging",
"score": null,
"reason": "packaging workflow not detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#packaging"
},
{
"name": "Pinned-Dependencies",
"score": 0,
"reason": "dependency not pinned by hash detected -- score normalized to 0",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#pinned-dependencies"
},
{
"name": "SAST",
"score": 0,
"reason": "SAST tool is not run on all commits -- score normalized to 0",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#sast"
},
{
"name": "Security-Policy",
"score": 0,
"reason": "security policy file not detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#security-policy"
},
{
"name": "Signed-Releases",
"score": null,
"reason": "no releases found",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#signed-releases"
},
{
"name": "Token-Permissions",
"score": 0,
"reason": "detected GitHub workflow tokens with excessive permissions",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#token-permissions"
},
{
"name": "Vulnerabilities",
"score": 10,
"reason": "0 existing vulnerabilities detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#vulnerabilities"
}
],
"commit": "ca39f5c7bff9edfc9d70012c5dfdf23b44f7971e",
"ran_at": "2026-07-22T03:59:48Z",
"aggregate_score": 3.5,
"scorecard_version": "v5.5.0"
},
"has_codeql_workflow": false,
"has_security_policy": false,
"has_dependabot_config": false
},
"contribution_flow": {
"collected": true,
"ci_last_run_at": "2026-07-17T16:06:22Z",
"oldest_open_prs": [
{
"number": 25,
"created_at": "2026-07-16T03:56:27Z",
"last_comment_at": "2026-07-17T15:44:04Z",
"last_comment_author": "bluescreen10"
}
],
"last_merged_pr_at": "2025-08-08T21:09:07Z",
"ci_last_conclusion": "SUCCESS",
"oldest_open_issues": [
{
"number": 22,
"created_at": "2025-12-04T08:48:17Z",
"last_comment_at": "2025-12-05T03:54:50Z",
"last_comment_author": "ppipada"
},
{
"number": 23,
"created_at": "2026-02-19T11:17:31Z",
"last_comment_at": null,
"last_comment_author": null
}
]
}
},
"config": {
"disabled_metrics": [],
"disabled_categories": [],
"disabled_components": {}
},
"source": {
"url": "https://github.com/tiktoken-go/tokenizer",
"host": "github.com",
"name": "tokenizer",
"owner": "tiktoken-go"
},
"metrics": {
"overall": {
"key": "overall",
"band": "moderate",
"name": "Overall health",
"note": null,
"notes": [],
"value": 54,
"inputs": {
"security": 35,
"vitality": 64,
"community": 55,
"governance": 60,
"engineering": 50
},
"components": []
},
"categories": [
{
"key": "vitality",
"band": "moderate",
"name": "Vitality",
"value": 64,
"weight": 0.22,
"metrics": [
{
"key": "development_activity",
"band": "at_risk",
"name": "Development activity",
"note": null,
"notes": [],
"value": 46,
"inputs": {
"commits_last_year": 5,
"human_commit_share": 1,
"days_since_last_push": 4,
"active_weeks_last_year": 3
},
"components": [
{
"key": "push_recency",
"name": "Push recency",
"detail": "last push 4 days ago",
"points": 36,
"status": "met",
"details": [
{
"code": "push_recency",
"params": {
"days": 4
}
}
],
"max_points": 36
},
{
"key": "commit_cadence",
"name": "Commit cadence",
"detail": "3/52 weeks with commits",
"points": 2.1,
"status": "partial",
"details": [
{
"code": "commit_cadence_weeks",
"params": {
"weeks": 3
}
}
],
"max_points": 36
},
{
"key": "commit_volume",
"name": "Commit volume",
"detail": "5 commits in the last year",
"points": 7,
"status": "partial",
"details": [
{
"code": "commits_last_year",
"params": {
"count": 5
}
}
],
"max_points": 18
},
{
"key": "openssf_scorecard_maintained",
"name": "OpenSSF Scorecard: Maintained",
"detail": "2 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 1",
"points": 1,
"status": "partial",
"details": [],
"max_points": 10
}
]
},
{
"key": "release_discipline",
"band": "excellent",
"name": "Release discipline",
"note": "Excluded from scoring (no data or not applicable): OpenSSF Scorecard: Signed-Releases. Remaining weights renormalized.",
"notes": [
{
"code": "excluded_no_data",
"params": {
"components": [
"openssf_scorecard_signed_releases"
]
}
},
{
"code": "weights_renormalized",
"params": {}
}
],
"value": 92,
"inputs": {
"releases_count": 13,
"latest_release_tag": "v0.8.1",
"releases_from_tags": false,
"days_since_latest_release": 4,
"mean_days_between_releases": 66.9
},
"components": [
{
"key": "ships_releases",
"name": "Ships releases",
"detail": "13 releases published",
"points": 27,
"status": "met",
"details": [
{
"code": "releases_published",
"params": {
"count": 13
}
}
],
"max_points": 27
},
{
"key": "release_recency",
"name": "Release recency",
"detail": "latest release 4 days ago",
"points": 36,
"status": "met",
"details": [
{
"code": "release_recency",
"params": {
"days": 4
}
}
],
"max_points": 36
},
{
"key": "release_cadence",
"name": "Release cadence",
"detail": "a release every ~66.9 days",
"points": 19.8,
"status": "partial",
"details": [
{
"code": "release_cadence",
"params": {
"gap": 66.9
}
}
],
"max_points": 27
},
{
"key": "openssf_scorecard_signed_releases",
"name": "OpenSSF Scorecard: Signed-Releases",
"detail": "no releases found",
"points": 0,
"status": "excluded",
"details": [
{
"code": "no_data",
"params": {}
}
],
"max_points": 10
}
]
},
{
"key": "abandonment",
"band": "excellent",
"name": "Abandonment",
"note": null,
"notes": [],
"value": 100,
"inputs": {
"cap": null,
"state": "maintained",
"guards": [],
"signals": [],
"red_flag": false,
"multiplier_pct": 100,
"declared_reason": null,
"unverified_reason": null,
"unanswered_open_prs": null,
"unanswered_open_issues": null,
"days_since_last_merged_pr": null,
"days_since_last_human_commit": 4,
"days_since_last_human_commit_is_floor": false
},
"components": [
{
"key": "project_is_still_maintained",
"name": "Project is still maintained",
"detail": "last human commit 4 days ago",
"points": 100,
"status": "met",
"details": [
{
"code": "abandonment_maintained",
"params": {
"days": 4
}
}
],
"max_points": 100
}
]
}
],
"description": "Is the project alive — is code being written and are releases shipping?"
},
{
"key": "community",
"band": "moderate",
"name": "Community & Adoption",
"value": 55,
"weight": 0.18,
"metrics": [
{
"key": "popularity",
"band": "moderate",
"name": "Popularity & adoption",
"note": null,
"notes": [],
"value": 59,
"inputs": {
"forks": 32,
"stars": 445,
"watchers": 6,
"growth_state": "organic",
"growth_factor_pct": 100
},
"components": [
{
"key": "stars",
"name": "Stars",
"detail": "445 stars",
"points": 42.9,
"status": "partial",
"details": [
{
"code": "stars",
"params": {
"count": 445
}
}
],
"max_points": 60
},
{
"key": "forks",
"name": "Forks",
"detail": "32 forks",
"points": 12.4,
"status": "partial",
"details": [
{
"code": "forks",
"params": {
"count": 32
}
}
],
"max_points": 25
},
{
"key": "watchers",
"name": "Watchers",
"detail": "6 watchers",
"points": 3.9,
"status": "partial",
"details": [
{
"code": "watchers",
"params": {
"count": 6
}
}
],
"max_points": 15
}
]
},
{
"key": "community_health",
"band": "moderate",
"name": "Community health",
"note": null,
"notes": [],
"value": 50,
"inputs": {
"has_readme": true,
"has_license": true,
"has_contributing": false,
"has_issue_template": false,
"has_code_of_conduct": false,
"has_pull_request_template": false
},
"components": [
{
"key": "readme",
"name": "README",
"detail": null,
"points": 22.5,
"status": "met",
"details": [],
"max_points": 22.5
},
{
"key": "license",
"name": "License",
"detail": "recognized license (MIT)",
"points": 22.5,
"status": "met",
"details": [
{
"code": "license_standard",
"params": {}
},
{
"code": "license_spdx",
"params": {
"spdx": "MIT"
}
}
],
"max_points": 22.5
},
{
"key": "contributing_guide",
"name": "CONTRIBUTING guide",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 18
},
{
"key": "code_of_conduct",
"name": "Code of conduct",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 13.5
},
{
"key": "issue_template",
"name": "Issue template",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 7.2
},
{
"key": "pr_template",
"name": "PR template",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 6.3
}
]
}
],
"description": "Does the project have users, downloads, attention, and a welcoming setup for contributors?"
},
{
"key": "governance",
"band": "moderate",
"name": "Sustainability & Governance",
"value": 60,
"weight": 0.24,
"metrics": [
{
"key": "maintainer_resilience",
"band": "critical",
"name": "Maintainer resilience (bus factor)",
"note": null,
"notes": [],
"value": 29,
"inputs": {
"bus_factor": 1,
"contributors_sampled": 9,
"top_contributor_share": 0.658
},
"components": [
{
"key": "bus_factor",
"name": "Bus factor",
"detail": "1 contributor(s) cover half of all commits",
"points": 9,
"status": "partial",
"details": [
{
"code": "bus_factor",
"params": {
"count": 1
}
}
],
"max_points": 54
},
{
"key": "commit_distribution",
"name": "Commit distribution",
"detail": "top contributor authored 66% of commits",
"points": 7.7,
"status": "partial",
"details": [
{
"code": "top_contributor_share",
"params": {
"share": 66
}
}
],
"max_points": 22.5
},
{
"key": "contributor_breadth",
"name": "Contributor breadth",
"detail": "9 contributors",
"points": 12.2,
"status": "partial",
"details": [
{
"code": "contributors_sampled",
"params": {
"count": 9
}
}
],
"max_points": 13.5
},
{
"key": "openssf_scorecard_contributors",
"name": "OpenSSF Scorecard: Contributors",
"detail": "project has 0 contributing companies or organizations -- score normalized to 0",
"points": 0,
"status": "missed",
"details": [],
"max_points": 10
}
]
},
{
"key": "responsiveness",
"band": "good",
"name": "Issue & PR responsiveness",
"note": null,
"notes": [],
"value": 82,
"inputs": {
"merged_prs": 10,
"open_issues": 2,
"closed_issues": 11,
"issue_closed_ratio": 0.846,
"closed_unmerged_prs": 1
},
"components": [
{
"key": "issue_resolution",
"name": "Issue resolution",
"detail": "85% of issues closed",
"points": 39.6,
"status": "partial",
"details": [
{
"code": "issues_closed_share",
"params": {
"share": 85
}
}
],
"max_points": 46.75
},
{
"key": "pr_acceptance",
"name": "PR acceptance",
"detail": "10/11 decided PRs merged",
"points": 34.8,
"status": "partial",
"details": [
{
"code": "decided_prs_merged",
"params": {
"merged": 10,
"decided": 11
}
}
],
"max_points": 38.25
},
{
"key": "openssf_scorecard_code_review",
"name": "OpenSSF Scorecard: Code-Review",
"detail": "Found 8/16 approved changesets -- score normalized to 5",
"points": 7.5,
"status": "partial",
"details": [],
"max_points": 15
}
]
},
{
"key": "stewardship",
"band": "at_risk",
"name": "Ownership & stewardship",
"note": null,
"notes": [],
"value": 44,
"inputs": {
"followers": 5,
"owner_type": "Organization",
"is_verified": null,
"owner_login": "tiktoken-go",
"public_repos": 1,
"account_age_days": 1202
},
"components": [
{
"key": "ownership_backing",
"name": "Ownership backing",
"detail": "organization-owned",
"points": 30,
"status": "met",
"details": [
{
"code": "owner_organization",
"params": {}
}
],
"max_points": 30
},
{
"key": "verified_domain",
"name": "Verified domain",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 20
},
{
"key": "owner_reach",
"name": "Owner reach",
"detail": "5 followers of tiktoken-go",
"points": 5.6,
"status": "partial",
"details": [
{
"code": "owner_followers",
"params": {
"count": 5,
"login": "tiktoken-go"
}
}
],
"max_points": 25
},
{
"key": "track_record",
"name": "Track record",
"detail": "1 public repos, account ~3 yr old",
"points": 8.8,
"status": "partial",
"details": [
{
"code": "public_repos",
"params": {
"count": 1
}
},
{
"code": "account_age_years",
"params": {
"years": 3
}
}
],
"max_points": 25
}
]
},
{
"key": "package_maintenance",
"band": "excellent",
"name": "Package maintenance",
"note": null,
"notes": [],
"value": 100,
"inputs": {
"packages": [
"github.com/tiktoken-go/tokenizer"
],
"ecosystems": "go",
"any_deprecated": false,
"min_days_since_publish": 4
},
"components": [
{
"key": "published_resolvable",
"name": "Published & resolvable",
"detail": "1 package(s) on go",
"points": 25,
"status": "met",
"details": [
{
"code": "packages_published",
"params": {
"count": 1,
"ecosystems": "go"
}
}
],
"max_points": 25
},
{
"key": "publish_recency",
"name": "Publish recency",
"detail": "latest publish 4 days ago",
"points": 35,
"status": "met",
"details": [
{
"code": "publish_recency",
"params": {
"days": 4
}
}
],
"max_points": 35
},
{
"key": "version_history",
"name": "Version history",
"detail": "16 published versions",
"points": 20,
"status": "met",
"details": [
{
"code": "published_versions",
"params": {
"count": 16
}
}
],
"max_points": 20
},
{
"key": "not_deprecated",
"name": "Not deprecated",
"detail": "active, not deprecated or yanked",
"points": 20,
"status": "met",
"details": [
{
"code": "package_not_deprecated",
"params": {}
}
],
"max_points": 20
}
]
}
],
"description": "Will the project survive its people — bus factor, responsiveness, who backs it, and package upkeep?"
},
{
"key": "engineering",
"band": "moderate",
"name": "Engineering Quality",
"value": 50,
"weight": 0.2,
"metrics": [
{
"key": "engineering_practices",
"band": "moderate",
"name": "Engineering practices",
"note": null,
"notes": [],
"value": 50,
"inputs": {
"has_ci": true,
"has_tests": true,
"has_editorconfig": false,
"has_linter_config": false,
"has_precommit_config": false
},
"components": [
{
"key": "ci_workflows",
"name": "CI workflows",
"detail": "1 workflow(s)",
"points": 24,
"status": "met",
"details": [
{
"code": "ci_workflows",
"params": {
"count": 1
}
}
],
"max_points": 24
},
{
"key": "tests_present",
"name": "Tests present",
"detail": null,
"points": 24,
"status": "met",
"details": [],
"max_points": 24
},
{
"key": "linter_config",
"name": "Linter config",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 16
},
{
"key": "pre_commit_hooks",
"name": "Pre-commit hooks",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 9.6
},
{
"key": "editorconfig",
"name": ".editorconfig",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 6.4
},
{
"key": "openssf_scorecard_ci_tests",
"name": "OpenSSF Scorecard: CI-Tests",
"detail": "1 out of 8 merged PRs checked by a CI test -- score normalized to 1",
"points": 2,
"status": "partial",
"details": [],
"max_points": 20
}
]
},
{
"key": "documentation",
"band": "moderate",
"name": "Documentation",
"note": null,
"notes": [],
"value": 50,
"inputs": {
"topics": [],
"has_wiki": true,
"homepage": null,
"has_readme": true,
"has_docs_dir": false,
"has_description": true
},
"components": [
{
"key": "readme",
"name": "README",
"detail": null,
"points": 30,
"status": "met",
"details": [],
"max_points": 30
},
{
"key": "documentation_directory",
"name": "Documentation directory",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 25
},
{
"key": "documentation_homepage_site",
"name": "Documentation / homepage site",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 15
},
{
"key": "repository_description",
"name": "Repository description",
"detail": null,
"points": 10,
"status": "met",
"details": [],
"max_points": 10
},
{
"key": "topics",
"name": "Topics",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 10
},
{
"key": "wiki",
"name": "Wiki",
"detail": null,
"points": 10,
"status": "met",
"details": [],
"max_points": 10
}
]
}
],
"description": "Are baseline engineering and documentation practices in place?"
},
{
"key": "security",
"band": "at_risk",
"name": "Security",
"value": 35,
"weight": 0.16,
"metrics": [
{
"key": "security_posture",
"band": "at_risk",
"name": "Security posture",
"note": "Excluded from scoring (no data or not applicable): Packaging, Signed-Releases. Remaining weights renormalized.",
"notes": [
{
"code": "excluded_no_data",
"params": {
"components": [
"packaging",
"signed_releases"
]
}
},
{
"code": "weights_renormalized",
"params": {}
}
],
"value": 35,
"inputs": {
"source": "openssf_scorecard",
"checks_evaluated": 16,
"scorecard_version": "v5.5.0",
"checks_inconclusive": 2,
"scorecard_aggregate": 3.5
},
"components": [
{
"key": "binary_artifacts",
"name": "Binary-Artifacts",
"detail": "no binaries found in the repo",
"points": 7.5,
"status": "met",
"details": [],
"max_points": 7.5
},
{
"key": "branch_protection",
"name": "Branch-Protection",
"detail": "branch protection not enabled on development/release branches",
"points": 0,
"status": "missed",
"details": [],
"max_points": 7.5
},
{
"key": "ci_tests",
"name": "CI-Tests",
"detail": "1 out of 8 merged PRs checked by a CI test -- score normalized to 1",
"points": 0.2,
"status": "partial",
"details": [],
"max_points": 2.5
},
{
"key": "cii_best_practices",
"name": "CII-Best-Practices",
"detail": "no effort to earn an OpenSSF best practices badge detected",
"points": 0,
"status": "missed",
"details": [],
"max_points": 2.5
},
{
"key": "code_review",
"name": "Code-Review",
"detail": "Found 8/16 approved changesets -- score normalized to 5",
"points": 3.8,
"status": "partial",
"details": [],
"max_points": 7.5
},
{
"key": "contributors",
"name": "Contributors",
"detail": "project has 0 contributing companies or organizations -- score normalized to 0",
"points": 0,
"status": "missed",
"details": [],
"max_points": 2.5
},
{
"key": "dangerous_workflow",
"name": "Dangerous-Workflow",
"detail": "no dangerous workflow patterns detected",
"points": 10,
"status": "met",
"details": [],
"max_points": 10
},
{
"key": "dependency_update_tool",
"name": "Dependency-Update-Tool",
"detail": "no update tool detected",
"points": 0,
"status": "missed",
"details": [],
"max_points": 7.5
},
{
"key": "fuzzing",
"name": "Fuzzing",
"detail": "project is not fuzzed",
"points": 0,
"status": "missed",
"details": [],
"max_points": 5
},
{
"key": "license",
"name": "License",
"detail": "license file detected",
"points": 2.5,
"status": "met",
"details": [],
"max_points": 2.5
},
{
"key": "maintained",
"name": "Maintained",
"detail": "2 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 1",
"points": 0.8,
"status": "partial",
"details": [],
"max_points": 7.5
},
{
"key": "packaging",
"name": "Packaging",
"detail": "packaging workflow not detected",
"points": 0,
"status": "excluded",
"details": [
{
"code": "no_data",
"params": {}
}
],
"max_points": 5
},
{
"key": "pinned_dependencies",
"name": "Pinned-Dependencies",
"detail": "dependency not pinned by hash detected -- score normalized to 0",
"points": 0,
"status": "missed",
"details": [],
"max_points": 5
},
{
"key": "sast",
"name": "SAST",
"detail": "SAST tool is not run on all commits -- score normalized to 0",
"points": 0,
"status": "missed",
"details": [],
"max_points": 5
},
{
"key": "security_policy",
"name": "Security-Policy",
"detail": "security policy file not detected",
"points": 0,
"status": "missed",
"details": [],
"max_points": 5
},
{
"key": "signed_releases",
"name": "Signed-Releases",
"detail": "no releases found",
"points": 0,
"status": "excluded",
"details": [
{
"code": "no_data",
"params": {}
}
],
"max_points": 7.5
},
{
"key": "token_permissions",
"name": "Token-Permissions",
"detail": "detected GitHub workflow tokens with excessive permissions",
"points": 0,
"status": "missed",
"details": [],
"max_points": 7.5
},
{
"key": "vulnerabilities",
"name": "Vulnerabilities",
"detail": "0 existing vulnerabilities detected",
"points": 7.5,
"status": "met",
"details": [],
"max_points": 7.5
}
]
},
{
"key": "high_risk_jurisdiction_exposure",
"band": "excellent",
"name": "High-Risk Jurisdiction Exposure",
"note": "Only high-confidence self-published location evidence affects this multiplier. Ambiguous matches are review-only; country evidence is not proof of nationality, citizenship, legal registration, malicious intent, or sanctions status.",
"notes": [
{
"code": "jurisdiction_evidence_limits",
"params": {}
}
],
"value": 100,
"inputs": {
"meaning": "self-published location evidence; not nationality or citizenship",
"red_flag": false,
"exposures": [],
"policy_countries": [
"Russia",
"Iran",
"North Korea"
],
"review_only_matches": 0,
"assessed_self_published_locations": 13
},
"components": [
{
"key": "policy_exposure_multiplier",
"name": "Policy exposure multiplier",
"detail": "no confirmed policy-scope location match",
"points": 100,
"status": "met",
"details": [
{
"code": "jurisdiction_no_match",
"params": {}
}
],
"max_points": 100
}
]
}
],
"description": "Are visible security and supply-chain practices strong, with no malicious dependency and no unresolved high-risk jurisdiction exposure?"
},
{
"key": "ai_readiness",
"band": "moderate",
"name": "AI Readiness",
"value": 51,
"weight": 0,
"metrics": [
{
"key": "ai_agent_context",
"band": "critical",
"name": "Agent context & guidance",
"note": null,
"notes": [],
"value": 27,
"inputs": {
"has_llms_txt": false,
"legible_history_share": 0.5,
"agent_instruction_files": [],
"agent_instruction_max_bytes": null
},
"components": [
{
"key": "agent_instructions",
"name": "Agent instructions",
"detail": "no CLAUDE.md / AGENTS.md / editor rules",
"points": 0,
"status": "missed",
"details": [
{
"code": "no_agent_instructions",
"params": {}
}
],
"max_points": 45
},
{
"key": "machine_readable_docs_llms_txt",
"name": "Machine-readable docs (llms.txt)",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 15
},
{
"key": "legible_commit_history",
"name": "Legible commit history",
"detail": "22 of 44 human commits state their intent (structured subject or explanatory body)",
"points": 26.7,
"status": "partial",
"details": [
{
"code": "legible_history",
"params": {
"legible": 22,
"sampled": 44
}
}
],
"max_points": 40
}
]
},
{
"key": "ai_verify_loop",
"band": "moderate",
"name": "Verify loop (build / test / typecheck)",
"note": null,
"notes": [],
"value": 56,
"inputs": {
"has_nix": false,
"has_tests": true,
"lockfiles": [
"go.sum"
],
"has_dockerfile": false,
"typed_language": true,
"bootstrap_files": [],
"has_devcontainer": false,
"has_linter_config": false,
"typecheck_configs": [],
"agent_commit_share": 0,
"toolchain_manifests": [
"go.mod"
],
"dependency_bot_commit_share": 0
},
"components": [
{
"key": "one_command_bootstrap",
"name": "One-command bootstrap",
"detail": "go.mod (toolchain convention, no task runner)",
"points": 12.6,
"status": "partial",
"details": [
{
"code": "toolchain_convention",
"params": {
"files": "go.mod"
}
}
],
"max_points": 18
},
{
"key": "automated_tests",
"name": "Automated tests",
"detail": null,
"points": 22,
"status": "met",
"details": [],
"max_points": 22
},
{
"key": "lint_format_config",
"name": "Lint / format config",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 11
},
{
"key": "static_type_checking",
"name": "Static type checking",
"detail": "Go (statically typed)",
"points": 11,
"status": "met",
"details": [
{
"code": "statically_typed_language",
"params": {
"language": "Go"
}
}
],
"max_points": 11
},
{
"key": "reproducible_environment",
"name": "Reproducible environment",
"detail": "lockfile",
"points": 10,
"status": "met",
"details": [
{
"code": "file_list",
"params": {
"files": "lockfile"
}
}
],
"max_points": 10
},
{
"key": "demonstrated_agent_practice",
"name": "Demonstrated agent practice",
"detail": "no agent-authored commits among the last 44",
"points": 0,
"status": "missed",
"details": [
{
"code": "no_agent_authored_commits",
"params": {
"sampled": 44
}
}
],
"max_points": 10
},
{
"key": "automated_maintenance",
"name": "Automated maintenance",
"detail": "no automated dependency updates observed",
"points": 0,
"status": "missed",
"details": [
{
"code": "no_dependency_automation",
"params": {}
}
],
"max_points": 8
},
{
"key": "openssf_scorecard_pinned_dependencies",
"name": "OpenSSF Scorecard: Pinned-Dependencies",
"detail": "dependency not pinned by hash detected -- score normalized to 0",
"points": 0,
"status": "missed",
"details": [],
"max_points": 10
}
]
},
{
"key": "ai_code_legibility",
"band": "excellent",
"name": "Code legibility for models",
"note": null,
"notes": [],
"value": 87,
"inputs": {
"primary_language": "Go",
"largest_source_bytes": 7852594,
"source_files_sampled": 17,
"oversized_source_files": 4
},
"components": [
{
"key": "type_checkable_code",
"name": "Type-checkable code",
"detail": "Go (statically typed)",
"points": 45,
"status": "met",
"details": [
{
"code": "statically_typed_language",
"params": {
"language": "Go"
}
}
],
"max_points": 45
},
{
"key": "manageable_file_sizes",
"name": "Manageable file sizes",
"detail": "4/17 source files over 60KB",
"points": 42.1,
"status": "partial",
"details": [
{
"code": "oversized_source_files",
"params": {
"kb": 60,
"sampled": 17,
"oversized": 4
}
}
],
"max_points": 55
}
]
}
],
"description": "How well is the repo equipped to be developed and maintained with AI coding agents? An independent, experimental badge — weight 0.0, so it is surfaced on its own and does not affect the overall health score."
}
],
"metrics_version": "1.13.0"
},
"warnings": [
"GitHub dependency-graph SBOM unavailable (404); the dependency graph may be disabled for this repository"
],
"report_type": "repository",
"generated_at": "2026-07-22T04:00:03.666069Z",
"schema_version": "0.26.0",
"badge_url": "https://raw.githubusercontent.com/inspect-software/badges/main/v1/t/tiktoken-go/tokenizer.svg",
"full_name": "tiktoken-go/tokenizer",
"license_state": "standard",
"license_spdx": "MIT"
}Scores are signals, not warranties. They reflect publicly visible practices on GitHub — not a code audit, and not a security guarantee.
Missing data is excluded and weights renormalized, never scored as zero. Methodology is versioned and open: metrics v1.13.0, schema v0.26.0 — full methodology · metrics wiki.
How one result sits in the wider record: aggregate statistics — Go.