Öffentliches RegisterMittel
Software-GesundheitsberichtSchema 0.26.0 · Metriken 1.13.0 · 2026-07-22 04:00 UTC
tiktoken-go / tokenizer
Pure Go implementation of OpenAI's tiktoken tokenizer
tiktoken-go/tokenizer erreicht einen Gesundheitsindex von 54 von 100 und liegt damit im Bereich Mittel. Am stärksten schneidet es bei Vitality (64/100) ab, am schwächsten bei Security (35/100). Zuletzt vor 4 Tagen aktualisiert. Ein einzelner Mitwirkender trägt den Großteil der jüngsten Arbeit.
54
gesamt / 100Software-Gesundheitsindex
Metriken werden auf einer Skala von 1–100 in gewichtete Kategorien gruppiert. Der Gesamtwert beginnt als ihr Mittel; sobald öffentliche Evidenz die Richtlinie für Hochrisikojurisdiktionen auslöst, wird die Bewertung angepasst und erhält die Obergrenze 49 (Gefährdet). AI Readiness liegt außerhalb.
54
Exzellent85-100Vorbildlich; erfüllt im Wesentlichen alle geprüften Kriterien
Gut70-84Gesund; geringfügige Lücken
Mittel50-69Akzeptabel mit deutlichen Lücken; Überprüfung empfohlen
Gefährdet30-49Erhebliche Schwächen; eine Übernahme erfordert Vorsicht
Kritisch1-29Schwerwiegende Probleme (aufgegeben, nur ein Maintainer, keine Hygiene)
Bewertungsprofil
Jede Achse ist eine Kategorie. Die Form zählt mehr als der Durchschnitt — ein gesundes Projekt füllt die gesamte Fläche, während ein Profil aus Spitzen und Kratern bedeutet, dass Stärke in einer Dimension Risiken in einer anderen verdeckt.
Eigentümerschaft
tiktoken-goOrganisation
5 Follower1 öffentliches Reposeit Apr. 2023
Dieses Repository wird von einer Organisation getragen — geteilte, rechenschaftspflichtige Trägerschaft, die jeden einzelnen Maintainer überdauern kann.
Paket-Ökosysteme
| Registry | Paket | Version | Downloads / Monat | Versionen | Zuletzt veröffentlicht |
|---|---|---|---|---|---|
| Go | github.com/tiktoken-go/tokenizer | v0.8.1 | - | 16 | vor 4 Tagen |
Metriken nach Kategorie
Vitalität
Lebt das Projekt — wird Code geschrieben und werden Releases ausgeliefert?
64
Entwicklungsaktivität
46Gefährdet
Wie die Bewertung erfolgt
| 36/36 | Push-Aktualität — letzter Push vor 4 Tagen |
| 2.1/36 | Commit-Rhythmus — 3/52 Wochen mit Commits |
| 7/18 | Commit-Volumen — 5 Commits im letzten Jahr |
| 1/10 | OpenSSF Scorecard: Maintained — 2 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 1 |
Verwendete Eingangsdaten
| commits_last_year | 5 |
| human_commit_share | 1 |
| days_since_last_push | 4 |
| active_weeks_last_year | 3 |
Release-Disziplin
92Exzellent
Wie die Bewertung erfolgt
| 27/27 | Liefert Releases aus — 13 Releases veröffentlicht |
| 36/36 | Release-Aktualität — letztes Release vor 4 Tagen |
| 19.8/27 | Release-Rhythmus — ein Release etwa alle 66,9 Tage |
| 0/10 | OpenSSF Scorecard: Signed-Releases — keine Daten |
Verwendete Eingangsdaten
| releases_count | 13 |
| latest_release_tag | v0.8.1 |
| releases_from_tags | nein |
| days_since_latest_release | 4 |
| mean_days_between_releases | 66,9 |
Von der Bewertung ausgeschlossen (keine Daten oder nicht anwendbar): OpenSSF Scorecard: Signed-Releases. Die verbleibenden Gewichte wurden renormalisiert.
Community & Verbreitung
Hat das Projekt Nutzer, Downloads, Aufmerksamkeit und ein einladendes Umfeld für Beitragende?
55
Popularität & Verbreitung
59Mittel
Wie die Bewertung erfolgt
| 42.9/60 | Stars — 445 Stars |
| 12.4/25 | Forks — 32 Forks |
| 3.9/15 | Watcher — 6 Watcher |
Verwendete Eingangsdaten
| forks | 32 |
| stars | 445 |
| watchers | 6 |
| growth_state | organic |
| growth_factor_pct | 100 |
Community-Gesundheit
50Mittel
Wie die Bewertung erfolgt
| 22.5/22.5 | README |
| 22.5/22.5 | Lizenz — anerkannte Lizenz (MIT) |
| 0/18 | CONTRIBUTING-Leitfaden |
| 0/13.5 | Verhaltenskodex |
| 0/7.2 | Issue-Vorlage |
| 0/6.3 | PR-Vorlage |
Verwendete Eingangsdaten
| has_readme | ja |
| has_license | ja |
| has_contributing | nein |
| has_issue_template | nein |
| has_code_of_conduct | nein |
| has_pull_request_template | nein |
Nachhaltigkeit & Governance
Überdauert das Projekt die Menschen, die es tragen — Bus-Faktor, Reaktionsfähigkeit, Trägerschaft und Paketpflege?
60
Maintainer-Resilienz (Bus-Faktor)
29Kritisch
Wie die Bewertung erfolgt
| 9/54 | Bus-Faktor — 1 Beitragende decken die Hälfte aller Commits ab |
| 7.7/22.5 | Commit-Verteilung — wichtigste beitragende Person verfasste 66 % der Commits |
| 12.2/13.5 | Breite der Beitragenden — 9 Beitragende |
| 0/10 | OpenSSF Scorecard: Contributors — project has 0 contributing companies or organizations -- score normalized to 0 |
Verwendete Eingangsdaten
| bus_factor | 1 |
| contributors_sampled | 9 |
| top_contributor_share | 0,658 |
Wie die Bewertung erfolgt
| 39.6/46.8 | Issue-Lösungsquote — 85 % der Issues geschlossen |
| 34.8/38.3 | PR-Annahme — 10/11 entschiedene PRs gemergt |
| 7.5/15 | OpenSSF Scorecard: Code-Review — Found 8/16 approved changesets -- score normalized to 5 |
Verwendete Eingangsdaten
| merged_prs | 10 |
| open_issues | 2 |
| closed_issues | 11 |
| issue_closed_ratio | 0,846 |
| closed_unmerged_prs | 1 |
Eigentümerschaft & Trägerschaft
44Gefährdet
Wie die Bewertung erfolgt
| 30/30 | Organisatorische Trägerschaft — im Besitz einer Organisation |
| 0/20 | Verifizierte Domain |
| 5.6/25 | Reichweite des Inhabers — 5 Follower von tiktoken-go |
| 8.8/25 | Kontohistorie — 1 öffentliche Repos, Kontoalter ca. 3 Jahre |
Verwendete Eingangsdaten
| followers | 5 |
| owner_type | Organization |
| is_verified | — |
| owner_login | tiktoken-go |
| public_repos | 1 |
| account_age_days | 1.202 |
Paketpflege
100Exzellent
Wie die Bewertung erfolgt
| 25/25 | Veröffentlicht & auflösbar — 1 Paket(e) auf go |
| 35/35 | Veröffentlichungsaktualität — letzte Veröffentlichung vor 4 Tagen |
| 20/20 | Versionshistorie — 16 veröffentlichte Versionen |
| 20/20 | Nicht veraltet — aktiv, nicht veraltet oder zurückgezogen |
Verwendete Eingangsdaten
| packages | github.com/tiktoken-go/tokenizer |
| ecosystems | go |
| any_deprecated | nein |
| min_days_since_publish | 4 |
Engineering-Qualität
Sind grundlegende Engineering- und Dokumentationspraktiken vorhanden?
50
Engineering-Praktiken
50Mittel
Wie die Bewertung erfolgt
| 24/24 | CI-Workflows — 1 Workflow(s) |
| 24/24 | Tests vorhanden |
| 0/16 | Linter-Konfiguration |
| 0/9.6 | Pre-Commit-Hooks |
| 0/6.4 | .editorconfig |
| 2/20 | OpenSSF Scorecard: CI-Tests — 1 out of 8 merged PRs checked by a CI test -- score normalized to 1 |
Verwendete Eingangsdaten
| has_ci | ja |
| has_tests | ja |
| has_editorconfig | nein |
| has_linter_config | nein |
| has_precommit_config | nein |
Dokumentation
50Mittel
Wie die Bewertung erfolgt
| 30/30 | README |
| 0/25 | Dokumentationsverzeichnis |
| 0/15 | Dokumentations-/Homepage-Site |
| 10/10 | Repository-Beschreibung |
| 0/10 | Topics |
| 10/10 | Wiki |
Verwendete Eingangsdaten
| topics | — |
| has_wiki | ja |
| homepage | — |
| has_readme | ja |
| has_docs_dir | nein |
| has_description | ja |
Sicherheit
Sind die sichtbaren Sicherheits- und Lieferkettenpraktiken belastbar, ohne ungeklärte Exposition gegenüber Hochrisikojurisdiktionen?
35
Sicherheitslage
35Gefährdet
Wie die Bewertung erfolgt
| 7.5/7.5 | Binary-Artifacts — no binaries found in the repo |
| 0/7.5 | Branch-Protection — branch protection not enabled on development/release branches |
| 0.2/2.5 | CI-Tests — 1 out of 8 merged PRs checked by a CI test -- score normalized to 1 |
| 0/2.5 | CII-Best-Practices — no effort to earn an OpenSSF best practices badge detected |
| 3.8/7.5 | Code-Review — Found 8/16 approved changesets -- score normalized to 5 |
| 0/2.5 | Contributors — project has 0 contributing companies or organizations -- score normalized to 0 |
| 10/10 | Dangerous-Workflow — no dangerous workflow patterns detected |
| 0/7.5 | Dependency-Update-Tool — no update tool detected |
| 0/5 | Fuzzing — project is not fuzzed |
| 2.5/2.5 | Lizenz — license file detected |
| 0.8/7.5 | Maintained — 2 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 1 |
| 0/5 | Packaging — keine Daten |
| 0/5 | Pinned-Dependencies — dependency not pinned by hash detected -- score normalized to 0 |
| 0/5 | SAST — SAST tool is not run on all commits -- score normalized to 0 |
| 0/5 | Security-Policy — security policy file not detected |
| 0/7.5 | Signed-Releases — keine Daten |
| 0/7.5 | Token-Permissions — detected GitHub workflow tokens with excessive permissions |
| 7.5/7.5 | Vulnerabilities — 0 existing vulnerabilities detected |
Verwendete Eingangsdaten
| source | openssf_scorecard |
| checks_evaluated | 16 |
| scorecard_version | v5.5.0 |
| checks_inconclusive | 2 |
| scorecard_aggregate | 3,5 |
Von der Bewertung ausgeschlossen (keine Daten oder nicht anwendbar): packaging, signed_releases. Die verbleibenden Gewichte wurden renormalisiert.
AI Readiness
Wie gut ist das Repository dafür ausgestattet, mit KI-Coding-Agenten entwickelt und gepflegt zu werden? Ein unabhängiges, experimentelles Badge — Gewicht 0,0, es wird eigenständig ausgewiesen und verändert den Gesamt-Gesundheitswert nicht.
51
AI-Agentenkontext & Leitlinien
27Kritisch
Wie die Bewertung erfolgt
| 0/45 | Agentenanweisungen — keine CLAUDE.md / AGENTS.md / Editor-Regeln |
| 0/15 | Maschinenlesbare Doku (llms.txt) |
| 26.7/40 | Lesbare Commit-Historie — 22 von 44 menschlichen Commits benennen ihre Absicht (strukturierter Betreff oder erläuternder Text) |
Verwendete Eingangsdaten
| has_llms_txt | nein |
| legible_history_share | 0,5 |
| agent_instruction_files | — |
| agent_instruction_max_bytes | — |
Wie die Bewertung erfolgt
| 12.6/18 | Bootstrap mit einem Befehl — go.mod (Toolchain-Konvention, kein Task-Runner) |
| 22/22 | Automatisierte Tests |
| 0/11 | Lint-/Format-Konfiguration |
| 11/11 | Statische Typprüfung — Go (statisch typisiert) |
| 10/10 | Reproduzierbare Umgebung — lockfile |
| 0/10 | Belegte Agentenpraxis — keine von Agenten verfassten Commits unter den letzten 44 |
| 0/8 | Automatisierte Wartung — keine automatisierten Abhängigkeits-Updates beobachtet |
| 0/10 | OpenSSF Scorecard: Pinned-Dependencies — dependency not pinned by hash detected -- score normalized to 0 |
Verwendete Eingangsdaten
| has_nix | nein |
| has_tests | ja |
| lockfiles | go.sum |
| has_dockerfile | nein |
| typed_language | ja |
| bootstrap_files | — |
| has_devcontainer | nein |
| has_linter_config | nein |
| typecheck_configs | — |
| agent_commit_share | 0 |
| toolchain_manifests | go.mod |
| dependency_bot_commit_share | 0 |
AI-Code-Lesbarkeit für Modelle
87Exzellent
Wie die Bewertung erfolgt
| 45/45 | Typprüfbarer Code — Go (statisch typisiert) |
| 42.1/55 | Handhabbare Dateigrößen — 4/17 Quelldateien über 60 KB |
Verwendete Eingangsdaten
| primary_language | Go |
| largest_source_bytes | 7.852.594 |
| source_files_sampled | 17 |
| oversized_source_files | 4 |
Eckdaten
445GitHub-Sterne
9Mitwirkende
5Commits, letzte 12 Monate
4Tage seit letztem Push
13Releases
1Bus-Faktor
2offene Issues
GoPaket-Ökosysteme
Warnungen zur Datenerhebung
- GitHub dependency-graph SBOM unavailable (404); the dependency graph may be disabled for this repository
Weitere Details
Stern- und Fork-Verlauf 445 ★ / 32 ⇿
445Sterne
32Forks
13Releases
Wann jeder Stern und Fork hinzugefügt wurde, von GitHub erfasst und nach Tagen gruppiert. Das kumulierte Wachstum steht direkt über den täglichen Zugängen, aus denen es besteht, sodass beide gegeneinander lesbar sind: stetiger organischer Zuwachs sieht ganz anders aus als ein abrupter, kurzlebiger Ausschlag. Wo dieser Unterschied messbar ist, wird er als Wachstumsauthentizität ausgewiesen.
Major 0Minor 6Patch 7
Jeder Punkt umfasst 4 Tage.
OpenSSF Scorecard 3.5 / 10
3.5Gesamtwert
Unabhängige, werkzeugneutrale Sicherheitsbewertung durch das quelloffene OpenSSF Scorecard. Jede Prüfung honoriert eine Sicherheits-Praxis, nicht das Werkzeug eines bestimmten Anbieters. Prüfungen, die Scorecard nicht ermitteln konnte, sind mit k. A. markiert und vom Sicherheitswert ausgeschlossen (nie als null gezählt).
| 10 | Binary-Artifacts | no binaries found in the repo |
| 0 | Branch-Protection | branch protection not enabled on development/release branches |
| 1 | CI-Tests | 1 out of 8 merged PRs checked by a CI test -- score normalized to 1 |
| 0 | CII-Best-Practices | no effort to earn an OpenSSF best practices badge detected |
| 5 | Code-Review | Found 8/16 approved changesets -- score normalized to 5 |
| 0 | Contributors | project has 0 contributing companies or organizations -- score normalized to 0 |
| 10 | Dangerous-Workflow | no dangerous workflow patterns detected |
| 0 | Dependency-Update-Tool | no update tool detected |
| 0 | Fuzzing | project is not fuzzed |
| 10 | License | license file detected |
| 1 | Maintained | 2 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 1 |
| k. A. | Packaging | packaging workflow not detected |
| 0 | Pinned-Dependencies | dependency not pinned by hash detected -- score normalized to 0 |
| 0 | SAST | SAST tool is not run on all commits -- score normalized to 0 |
| 0 | Security-Policy | security policy file not detected |
| k. A. | Signed-Releases | no releases found |
| 0 | Token-Permissions | detected GitHub workflow tokens with excessive permissions |
| 10 | Vulnerabilities | 0 existing vulnerabilities detected |
Direkte Abhängigkeiten 1
| Registry | Paket | Versionsvorgabe | Manifest |
|---|---|---|---|
| Go | github.com/dlclark/regexp2/v2 | v2.5.1 | go.mod |
Alle Abhängigkeiten nicht erhoben
Der aufgelöste Abhängigkeitssatz konnte für diesen Bericht nicht erhoben werden: GitHub dependency-graph SBOM unavailable (404); the dependency graph may be disabled for this repository
JSON-Rohbericht maschinenlesbar
{
"data": {
"repo": {
"topics": [],
"is_fork": false,
"size_kb": 4481,
"has_wiki": true,
"homepage": null,
"languages": {
"Go": 57424
},
"pushed_at": "2026-07-17T16:07:27Z",
"created_at": "2023-04-06T16:56:55Z",
"owner_type": "Organization",
"updated_at": "2026-07-21T17:35:27Z",
"description": "Pure Go implementation of OpenAI's tiktoken tokenizer",
"is_archived": false,
"is_disabled": false,
"license_spdx": "MIT",
"default_branch": "main",
"license_spdx_raw": "MIT",
"primary_language": "Go",
"significant_languages": [
"Go"
]
},
"owner": {
"blog": null,
"name": null,
"type": "Organization",
"login": "tiktoken-go",
"company": null,
"location": null,
"followers": 5,
"avatar_url": "https://avatars.githubusercontent.com/u/130084571?v=4",
"created_at": "2023-04-06T16:54:55Z",
"is_verified": null,
"public_repos": 1,
"account_age_days": 1202
},
"license": {
"state": "standard",
"spdx_id": "MIT",
"raw_spdx": "MIT",
"file_present": true,
"scorecard_found": true,
"profile_has_license": true
},
"activity": {
"releases": [
{
"tag": "v0.8.1",
"kind": "patch",
"published_at": "2026-07-17T16:07:27Z"
},
{
"tag": "v0.7.0",
"kind": "minor",
"published_at": "2025-08-08T21:09:51Z"
},
{
"tag": "v0.6.2",
"kind": "patch",
"published_at": "2025-05-09T22:25:24Z"
},
{
"tag": "v0.6.1",
"kind": "patch",
"published_at": "2025-03-27T14:38:34Z"
},
{
"tag": "v0.6.0",
"kind": "minor",
"published_at": "2025-03-07T14:52:12Z"
},
{
"tag": "v0.5.1",
"kind": "patch",
"published_at": "2025-02-25T15:57:06Z"
},
{
"tag": "v0.5.0",
"kind": "minor",
"published_at": "2025-02-17T20:46:57Z"
},
{
"tag": "v0.4.0",
"kind": "minor",
"published_at": "2025-02-12T23:00:11Z"
},
{
"tag": "v0.3.0",
"kind": "minor",
"published_at": "2025-01-28T18:45:20Z"
},
{
"tag": "v0.2.1",
"kind": "patch",
"published_at": "2024-11-22T22:16:28Z"
},
{
"tag": "v0.2.0",
"kind": "minor",
"published_at": "2024-09-13T15:32:23Z"
},
{
"tag": "v0.1.1",
"kind": "patch",
"published_at": "2024-05-10T19:49:15Z"
},
{
"tag": "v0.0.8",
"kind": "patch",
"published_at": "2023-04-09T21:48:24Z"
}
],
"recent_commits": [
{
"oid": "ca39f5c7bff9edfc9d70012c5dfdf23b44f7971e",
"body": null,
"is_bot": false,
"headline": "🐛 Bugfix with generated regular expressions",
"author_name": "bluescreen10",
"author_login": "bluescreen10",
"committed_at": "2026-07-17T16:05:43Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "a7606a16d9e9235d97b6ad0e25f3237cbdb8b3cf",
"body": "With this commit we bumped to v2 of the regular\nexpression library.",
"is_bot": false,
"headline": "Use pre-compiled regexp for more performace",
"author_name": "bluescreen10",
"author_login": "bluescreen10",
"committed_at": "2026-05-25T19:03:43Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "629192d03fc5026d8d04030a93f4aee3b3458f65",
"body": "Added GPT-5 models",
"is_bot": false,
"headline": "Merge pull request #21 from ando-masaki/gpt-5-fix",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2025-08-08T21:09:07Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "3ac0d0b657027c27041af52613b31edd4f6b3420",
"body": "Update tokenizer.go",
"is_bot": false,
"headline": "Merge pull request #2 from ando-masaki/ando-masaki-patch-1",
"author_name": "ando-masaki",
"author_login": "ando-masaki",
"committed_at": "2025-08-08T02:53:48Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "5cc8c3ce694c2fc3b3f04e475b07e565a16edf57",
"body": null,
"is_bot": false,
"headline": "Update tokenizer.go",
"author_name": "ando-masaki",
"author_login": "ando-masaki",
"committed_at": "2025-08-08T02:52:01Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "07878231645aadb13fdea5ce1cd3603177a6ce42",
"body": "Add o3, o4-min, gpt-4.1",
"is_bot": false,
"headline": "Merge pull request #18 from ando-masaki/add-o3-and-o4-mini-fix",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2025-06-05T18:34:31Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "4e6fabdf3c93d28645d98f81403f21fdea81f863",
"body": "Add o3, o4-min, gpt-4.1",
"is_bot": false,
"headline": "Update tokenizer.go",
"author_name": "ando-masaki",
"author_login": "ando-masaki",
"committed_at": "2025-06-05T06:16:33Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "d82759aa66c5ab2fbc972525d7775141b0099065",
"body": "This reverts commit a26622ed5c27d9dd14a06ac61bb5e20a7cddd2ac.",
"is_bot": false,
"headline": "Revert \"small performance improvement\"",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2025-05-09T22:24:16Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "a26622ed5c27d9dd14a06ac61bb5e20a7cddd2ac",
"body": null,
"is_bot": false,
"headline": "small performance improvement",
"author_name": "Mariano Wahlmann",
"author_login": null,
"committed_at": "2025-03-27T14:28:02Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "e56ab3c0656c9d36f5a7483b5949f55ec8e280ac",
"body": null,
"is_bot": false,
"headline": "Remove defer, also small improvement using strings for bpe",
"author_name": "Mariano Wahlmann",
"author_login": null,
"committed_at": "2025-03-27T14:18:22Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "bc8ba39dc0bbee16968778c1a6e792e79b94f486",
"body": null,
"is_bot": false,
"headline": "🚀 feat: added support for O3-mini model",
"author_name": "Mariano Wahlmann",
"author_login": null,
"committed_at": "2025-03-07T14:50:38Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "e4624aa318a9fce453eebc18536c2f96b33e35c5",
"body": null,
"is_bot": false,
"headline": "🐛 problem with regexp generated regexp code dependencies",
"author_name": "Mariano Wahlmann",
"author_login": null,
"committed_at": "2025-02-25T15:55:43Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "8ac6fdfcb9cb83eb69eb62709090abf5a6fc6080",
"body": "feat: memory-efficient token counting functionality",
"is_bot": false,
"headline": "Merge pull request #13 from amalucelli/count",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2025-02-17T20:46:05Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "d47b68072e97bb7663b46f802b7a1835127b059a",
"body": "This uses iterators for the common logic and the function Count and Encode would do different things",
"is_bot": false,
"headline": "Refactored code to remove duplication",
"author_name": "Mariano Wahlmann",
"author_login": null,
"committed_at": "2025-02-13T17:50:18Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "cd27b29fe9bcefbfde7b860b81776f27132b15a7",
"body": null,
"is_bot": false,
"headline": "feat: add count method for tokens",
"author_name": "Alexandre Guimarães Malucelli",
"author_login": "amalucelli",
"committed_at": "2025-02-13T17:50:16Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "03e0e5f2a364bc0069e17f15107ad16ddd443355",
"body": "pre-compile regexp2 with regexp2cg",
"is_bot": false,
"headline": "Merge pull request #11 from terwey/regexp2cg",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2025-02-12T22:59:04Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "2e19c1421ebbcdd3d959ebd04fc8e44b70e64696",
"body": null,
"is_bot": false,
"headline": "Added a go generate command for the compiled regular expression",
"author_name": "Mariano Wahlmann",
"author_login": null,
"committed_at": "2025-02-12T22:57:06Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "025023d0144d79b96726319c4375e21a8b80837e",
"body": "Add support of chatgpt-4o-latest model",
"is_bot": false,
"headline": "Merge pull request #12 from DenisPalnitsky/chatgpt-4o-support",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2025-01-28T18:43:40Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "fdcd81a21a133e40754a66a8346439cfa3b87350",
"body": null,
"is_bot": false,
"headline": "support of chatgpt-4o-latest",
"author_name": "Denis Palnitsky",
"author_login": "DenisPalnitsky",
"committed_at": "2025-01-24T18:29:54Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "45878105be78c8394b2cbd428bbbe2049640029d",
"body": null,
"is_bot": false,
"headline": "pre-compile regexp2 with regexp2cg",
"author_name": "Yorick Terweijden",
"author_login": "terwey",
"committed_at": "2024-12-04T20:56:37Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "2ad5811ad0b7d7c869c93cc33f4f161390464fe8",
"body": "Added o1-preview and o1-mini",
"is_bot": false,
"headline": "Merge pull request #10 from ando-masaki/o1-preview-and-mini-fix",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2024-11-22T22:12:55Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "60153be10d9a8246e1623eb6cf2e6e269e34bba3",
"body": null,
"is_bot": false,
"headline": "Simplified model selection",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2024-11-22T22:10:42Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "1ca7752d6d32ae47d743f933ac9c8783dd4a26c6",
"body": null,
"is_bot": false,
"headline": "Update tokenizer.go",
"author_name": "ando-masaki",
"author_login": "ando-masaki",
"committed_at": "2024-11-19T03:42:24Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "60becaa41f4a53a291b43420c9c88dac330880b5",
"body": "Update tokenizer.go",
"is_bot": false,
"headline": "Merge pull request #8 from GOOD21/main",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2024-11-07T14:32:20Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "ad6a208b2232926024e7f642fd617ff8e4ca9d71",
"body": null,
"is_bot": false,
"headline": "Update tokenizer.go",
"author_name": "GOOD21",
"author_login": "GOOD21",
"committed_at": "2024-11-07T10:09:50Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "b74b6b8b3a6705a0d7cb58ef9df45f6aba8404a4",
"body": "Fix panic caused by model name length",
"is_bot": false,
"headline": "Update tokenizer.go",
"author_name": "GOOD21",
"author_login": "GOOD21",
"committed_at": "2024-11-07T09:48:04Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "53141e47ff8b62f2b1f97a11f024f3cb515d09b7",
"body": "This commit should add support for all OpenAI\ncurrently available models.",
"is_bot": false,
"headline": "Added support for GPT4o and o200k_base models",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2024-09-13T15:28:04Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "7f65fe7d582fc7772cd74271d20dfad52fb8611a",
"body": "Generate embedded vocabulary maps lazily",
"is_bot": false,
"headline": "Merge pull request #6 from espadolini/lazy-load",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2024-05-10T19:46:47Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "93a91392ef75e411fc080030b02c448d09c8250e",
"body": null,
"is_bot": false,
"headline": "Format the code after codegen",
"author_name": "Edoardo Spadolini",
"author_login": "espadolini",
"committed_at": "2024-05-10T18:27:19Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "944cac3e81265a66bea9a725714c1698565e346c",
"body": null,
"is_bot": false,
"headline": "Generate embedded vocabulary maps lazily",
"author_name": "Edoardo Spadolini",
"author_login": "espadolini",
"committed_at": "2024-05-09T09:40:58Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "70d9b595feda6b98d6339ee4bab5f7a0c7c997f0",
"body": "Update alternatives",
"is_bot": false,
"headline": "Merge pull request #3 from philippgille/patch-1",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-19T13:07:29Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "4f793dc73f725f0c821ada3bcc30a84659850a5a",
"body": null,
"is_bot": false,
"headline": "Update alternatives",
"author_name": "Philipp Gillé",
"author_login": "philippgille",
"committed_at": "2023-04-18T20:32:25Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "94e23ef2a7f6a2832126c7ace1001258220d3cd5",
"body": "Go's regexp doesn't allow for look-around so had to re-write the\nregexp but that had a bug with multiple spaces. Switched to regexp2\nthat supports the native tiktoken regexp",
"is_bot": false,
"headline": "🐛 fixed bug with spaces",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-11T13:19:07Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "876e66b44d595542c8a0251b0a5bbadfdbb9b205",
"body": "before it was not possible to install the cli",
"is_bot": false,
"headline": "🐛 moved cli into its own folder",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-11T02:28:56Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "4868b0b5dc763f5886b8956dd4a9c9749aa7ae7e",
"body": "Renamed variables struct to reflect what they do a little better.",
"is_bot": false,
"headline": "🪄 clean up the bpe algorithm",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-09T21:43:58Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "12a3097aa3fad93104c64a7e774d974879e42927",
"body": "📘 fixed usage example in readme",
"is_bot": false,
"headline": "Merge pull request #1 from MarvinJWendt/patch-1",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-08T15:00:03Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "2886eb0986a0e4fe3220a8430ef4ccbe1a0560b5",
"body": "- Indents in the import block were not consistent\r\n- `()` was missing in the main function\r\n- `token` was unused",
"is_bot": false,
"headline": "🛂 fixed usage example in readme",
"author_name": "Marvin Wendt",
"author_login": "MarvinJWendt",
"committed_at": "2023-04-08T02:07:20Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "859c61e7dc23560fefb62403e163a1afd80814cf",
"body": "After comfirming the expected behavior changed tests to match OpenAI's\nbehavior. It should now work as expected.",
"is_bot": false,
"headline": "🛂 fixed tests",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-07T15:06:52Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "1b538b580912c6e4e1c8e8ba72e82aece232bd9f",
"body": "this would be helpful for other people to test and use the tokenizer.\nEven from other languages because this can be distributed as a single\nbinary.",
"is_bot": false,
"headline": "🌟 added a cli to use the tokenizer via the command Lines",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-06T21:56:04Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "79aa80d63f09a0d74b360df077d204f7755bc714",
"body": "The internal generate command was using log.Fatal instead of the\ncorrect log.Fatalf.",
"is_bot": false,
"headline": "🐛 fixed an issue in the internal generate command",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-06T20:58:15Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "ea2b0eea6b85ee333cdf235e71442c8196c742ad",
"body": "Right now emits the same output as OpenAI's tiktoken but I'm not sure\nthat's the right output.",
"is_bot": false,
"headline": "🤫 temporarily skipping failing test until I confirm the expected output",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-06T20:10:08Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "2e69c6a4db9547039ca536c281a73c350a20b57b",
"body": "This will make it more obvious the status of the build when reading the landing page",
"is_bot": false,
"headline": "📗 adding test results badge to README.md",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-06T20:02:57Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "a4eb7ed16c9dbe62ab99a614157759f64fd99dac",
"body": "This should help verify future releases. The test are not passing as of now because I'm not sure tiktoken's (original python library) output is correct.",
"is_bot": false,
"headline": "🛂 add github actions",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-06T19:58:32Z",
"body_truncated": false,
"is_coding_agent": false
},
{
"oid": "72848e8b530ee3bcb5f464232e39d905e06109a4",
"body": "This implementations should work for most of OpenAI's model. However\nWe are not yet dealing with the special tokens and/or gpt2 encoding.",
"is_bot": false,
"headline": "🚢 Initial Release",
"author_name": "Mariano Wahlmann",
"author_login": "bluescreen10",
"committed_at": "2023-04-06T19:53:05Z",
"body_truncated": false,
"is_coding_agent": false
}
],
"releases_count": 13,
"commits_last_year": 5,
"latest_release_at": "2026-07-17T16:07:27Z",
"latest_release_tag": "v0.8.1",
"releases_from_tags": false,
"days_since_last_push": 4,
"active_weeks_last_year": 3,
"days_since_latest_release": 4,
"mean_days_between_releases": 66.9
},
"community": {
"has_readme": true,
"has_license": true,
"has_description": true,
"has_contributing": false,
"health_percentage": 50,
"has_issue_template": false,
"has_code_of_conduct": false,
"has_pull_request_template": false
},
"ecosystem": {
"packages": [
{
"name": "github.com/tiktoken-go/tokenizer",
"exists": true,
"license": null,
"keywords": [],
"ecosystem": "go",
"matches_repo": true,
"registry_url": "https://pkg.go.dev/github.com/tiktoken-go/tokenizer",
"is_deprecated": false,
"latest_version": "v0.8.1",
"repository_url": "https://github.com/tiktoken-go/tokenizer",
"versions_count": 16,
"total_downloads": null,
"dependents_count": null,
"deprecation_note": null,
"maintainers_count": null,
"monthly_downloads": null,
"first_published_at": null,
"latest_published_at": "2026-07-17T16:05:43Z",
"latest_version_yanked": null,
"days_since_latest_publish": 4
}
]
},
"popularity": {
"forks": 32,
"stars": 445,
"watchers": 6,
"fork_history": {
"days": [
{
"date": "2023-04-07",
"count": 2
},
{
"date": "2023-04-09",
"count": 1
},
{
"date": "2023-04-11",
"count": 1
},
{
"date": "2023-04-18",
"count": 1
},
{
"date": "2023-05-05",
"count": 1
},
{
"date": "2023-05-24",
"count": 1
},
{
"date": "2023-06-28",
"count": 1
},
{
"date": "2023-11-09",
"count": 1
},
{
"date": "2023-12-18",
"count": 1
},
{
"date": "2024-01-17",
"count": 1
},
{
"date": "2024-04-15",
"count": 1
},
{
"date": "2024-06-06",
"count": 1
},
{
"date": "2024-06-09",
"count": 1
},
{
"date": "2024-06-13",
"count": 1
},
{
"date": "2024-06-15",
"count": 1
},
{
"date": "2024-11-06",
"count": 1
},
{
"date": "2024-11-08",
"count": 1
},
{
"date": "2024-11-19",
"count": 1
},
{
"date": "2024-12-04",
"count": 1
},
{
"date": "2025-01-10",
"count": 1
},
{
"date": "2025-01-24",
"count": 1
},
{
"date": "2025-02-10",
"count": 1
},
{
"date": "2025-06-10",
"count": 1
},
{
"date": "2025-07-10",
"count": 1
},
{
"date": "2025-08-03",
"count": 1
},
{
"date": "2025-09-04",
"count": 1
},
{
"date": "2025-09-12",
"count": 1
},
{
"date": "2026-02-02",
"count": 1
},
{
"date": "2026-02-11",
"count": 1
},
{
"date": "2026-06-23",
"count": 1
},
{
"date": "2026-07-16",
"count": 1
}
],
"complete": true,
"collected": 32,
"total_forks": 32
},
"star_history": {
"days": [
{
"date": "2023-04-07",
"count": 23
},
{
"date": "2023-04-08",
"count": 18
},
{
"date": "2023-04-09",
"count": 7
},
{
"date": "2023-04-10",
"count": 6
},
{
"date": "2023-04-11",
"count": 28
},
{
"date": "2023-04-12",
"count": 20
},
{
"date": "2023-04-13",
"count": 6
},
{
"date": "2023-04-14",
"count": 1
},
{
"date": "2023-04-15",
"count": 2
},
{
"date": "2023-04-16",
"count": 4
},
{
"date": "2023-04-17",
"count": 9
},
{
"date": "2023-04-18",
"count": 4
},
{
"date": "2023-04-19",
"count": 2
},
{
"date": "2023-04-20",
"count": 1
},
{
"date": "2023-04-22",
"count": 2
},
{
"date": "2023-04-23",
"count": 1
},
{
"date": "2023-04-24",
"count": 2
},
{
"date": "2023-04-25",
"count": 1
},
{
"date": "2023-04-26",
"count": 2
},
{
"date": "2023-04-27",
"count": 2
},
{
"date": "2023-04-29",
"count": 2
},
{
"date": "2023-05-03",
"count": 2
},
{
"date": "2023-05-04",
"count": 2
},
{
"date": "2023-05-05",
"count": 1
},
{
"date": "2023-05-06",
"count": 1
},
{
"date": "2023-05-10",
"count": 1
},
{
"date": "2023-05-11",
"count": 1
},
{
"date": "2023-05-12",
"count": 2
},
{
"date": "2023-05-14",
"count": 1
},
{
"date": "2023-05-16",
"count": 1
},
{
"date": "2023-05-17",
"count": 2
},
{
"date": "2023-05-18",
"count": 1
},
{
"date": "2023-05-29",
"count": 1
},
{
"date": "2023-05-30",
"count": 2
},
{
"date": "2023-06-01",
"count": 2
},
{
"date": "2023-06-02",
"count": 1
},
{
"date": "2023-06-03",
"count": 1
},
{
"date": "2023-06-05",
"count": 1
},
{
"date": "2023-06-07",
"count": 1
},
{
"date": "2023-06-08",
"count": 1
},
{
"date": "2023-06-09",
"count": 2
},
{
"date": "2023-06-11",
"count": 1
},
{
"date": "2023-06-13",
"count": 1
},
{
"date": "2023-06-17",
"count": 1
},
{
"date": "2023-06-28",
"count": 1
},
{
"date": "2023-06-29",
"count": 1
},
{
"date": "2023-07-04",
"count": 1
},
{
"date": "2023-07-06",
"count": 1
},
{
"date": "2023-07-09",
"count": 1
},
{
"date": "2023-07-10",
"count": 2
},
{
"date": "2023-07-13",
"count": 2
},
{
"date": "2023-07-22",
"count": 2
},
{
"date": "2023-07-29",
"count": 1
},
{
"date": "2023-08-03",
"count": 1
},
{
"date": "2023-08-08",
"count": 1
},
{
"date": "2023-08-14",
"count": 1
},
{
"date": "2023-08-21",
"count": 1
},
{
"date": "2023-08-23",
"count": 2
},
{
"date": "2023-08-29",
"count": 1
},
{
"date": "2023-09-04",
"count": 1
},
{
"date": "2023-09-15",
"count": 1
},
{
"date": "2023-09-21",
"count": 1
},
{
"date": "2023-10-05",
"count": 1
},
{
"date": "2023-10-12",
"count": 1
},
{
"date": "2023-10-13",
"count": 1
},
{
"date": "2023-10-14",
"count": 1
},
{
"date": "2023-10-27",
"count": 1
},
{
"date": "2023-11-03",
"count": 1
},
{
"date": "2023-11-07",
"count": 1
},
{
"date": "2023-11-10",
"count": 1
},
{
"date": "2023-11-14",
"count": 2
},
{
"date": "2023-11-22",
"count": 1
},
{
"date": "2023-11-28",
"count": 1
},
{
"date": "2023-11-29",
"count": 1
},
{
"date": "2023-12-01",
"count": 1
},
{
"date": "2023-12-04",
"count": 1
},
{
"date": "2023-12-16",
"count": 1
},
{
"date": "2023-12-18",
"count": 1
},
{
"date": "2023-12-31",
"count": 1
},
{
"date": "2024-01-10",
"count": 1
},
{
"date": "2024-01-31",
"count": 1
},
{
"date": "2024-02-05",
"count": 1
},
{
"date": "2024-02-17",
"count": 1
},
{
"date": "2024-02-21",
"count": 1
},
{
"date": "2024-03-01",
"count": 1
},
{
"date": "2024-03-05",
"count": 1
},
{
"date": "2024-03-07",
"count": 1
},
{
"date": "2024-04-11",
"count": 1
},
{
"date": "2024-04-16",
"count": 1
},
{
"date": "2024-04-18",
"count": 1
},
{
"date": "2024-04-26",
"count": 1
},
{
"date": "2024-05-04",
"count": 1
},
{
"date": "2024-05-12",
"count": 1
},
{
"date": "2024-05-20",
"count": 1
},
{
"date": "2024-05-27",
"count": 1
},
{
"date": "2024-05-29",
"count": 1
},
{
"date": "2024-06-06",
"count": 1
},
{
"date": "2024-06-11",
"count": 7
},
{
"date": "2024-06-12",
"count": 6
},
{
"date": "2024-06-13",
"count": 9
},
{
"date": "2024-06-14",
"count": 2
},
{
"date": "2024-06-15",
"count": 2
},
{
"date": "2024-06-16",
"count": 1
},
{
"date": "2024-06-17",
"count": 2
},
{
"date": "2024-06-19",
"count": 2
},
{
"date": "2024-06-27",
"count": 1
},
{
"date": "2024-07-09",
"count": 1
},
{
"date": "2024-07-10",
"count": 1
},
{
"date": "2024-07-11",
"count": 1
},
{
"date": "2024-07-16",
"count": 1
},
{
"date": "2024-07-21",
"count": 1
},
{
"date": "2024-07-22",
"count": 1
},
{
"date": "2024-08-12",
"count": 1
},
{
"date": "2024-08-14",
"count": 1
},
{
"date": "2024-08-15",
"count": 1
},
{
"date": "2024-08-21",
"count": 1
},
{
"date": "2024-08-24",
"count": 1
},
{
"date": "2024-08-28",
"count": 1
},
{
"date": "2024-09-01",
"count": 1
},
{
"date": "2024-09-06",
"count": 1
},
{
"date": "2024-09-11",
"count": 1
},
{
"date": "2024-09-13",
"count": 1
},
{
"date": "2024-09-25",
"count": 1
},
{
"date": "2024-09-30",
"count": 1
},
{
"date": "2024-10-03",
"count": 1
},
{
"date": "2024-10-12",
"count": 1
},
{
"date": "2024-10-30",
"count": 1
},
{
"date": "2024-11-05",
"count": 3
},
{
"date": "2024-11-10",
"count": 1
},
{
"date": "2024-11-16",
"count": 2
},
{
"date": "2024-11-20",
"count": 1
},
{
"date": "2024-11-23",
"count": 1
},
{
"date": "2024-11-25",
"count": 3
},
{
"date": "2024-11-27",
"count": 1
},
{
"date": "2024-11-29",
"count": 1
},
{
"date": "2024-12-02",
"count": 1
},
{
"date": "2024-12-08",
"count": 1
},
{
"date": "2024-12-18",
"count": 1
},
{
"date": "2024-12-19",
"count": 2
},
{
"date": "2024-12-30",
"count": 1
},
{
"date": "2025-01-12",
"count": 1
},
{
"date": "2025-01-24",
"count": 1
},
{
"date": "2025-01-27",
"count": 1
},
{
"date": "2025-01-29",
"count": 2
},
{
"date": "2025-01-30",
"count": 2
},
{
"date": "2025-01-31",
"count": 1
},
{
"date": "2025-02-04",
"count": 1
},
{
"date": "2025-02-09",
"count": 1
},
{
"date": "2025-02-11",
"count": 2
},
{
"date": "2025-02-16",
"count": 1
},
{
"date": "2025-02-17",
"count": 1
},
{
"date": "2025-02-18",
"count": 1
},
{
"date": "2025-02-19",
"count": 2
},
{
"date": "2025-02-23",
"count": 1
},
{
"date": "2025-03-01",
"count": 1
},
{
"date": "2025-03-03",
"count": 2
},
{
"date": "2025-03-07",
"count": 3
},
{
"date": "2025-03-11",
"count": 2
},
{
"date": "2025-03-15",
"count": 1
},
{
"date": "2025-03-17",
"count": 2
},
{
"date": "2025-03-19",
"count": 1
},
{
"date": "2025-03-20",
"count": 2
},
{
"date": "2025-03-21",
"count": 1
},
{
"date": "2025-03-22",
"count": 1
},
{
"date": "2025-03-27",
"count": 1
},
{
"date": "2025-03-28",
"count": 1
},
{
"date": "2025-04-03",
"count": 1
},
{
"date": "2025-04-06",
"count": 1
},
{
"date": "2025-04-09",
"count": 1
},
{
"date": "2025-04-10",
"count": 1
},
{
"date": "2025-04-11",
"count": 2
},
{
"date": "2025-04-17",
"count": 1
},
{
"date": "2025-04-19",
"count": 1
},
{
"date": "2025-04-24",
"count": 1
},
{
"date": "2025-05-04",
"count": 1
},
{
"date": "2025-05-15",
"count": 1
},
{
"date": "2025-05-21",
"count": 1
},
{
"date": "2025-05-27",
"count": 1
},
{
"date": "2025-05-30",
"count": 2
},
{
"date": "2025-06-09",
"count": 1
},
{
"date": "2025-06-10",
"count": 2
},
{
"date": "2025-06-17",
"count": 2
},
{
"date": "2025-06-18",
"count": 1
},
{
"date": "2025-06-19",
"count": 1
},
{
"date": "2025-06-25",
"count": 2
},
{
"date": "2025-06-27",
"count": 1
},
{
"date": "2025-06-28",
"count": 1
},
{
"date": "2025-06-29",
"count": 1
},
{
"date": "2025-07-01",
"count": 1
},
{
"date": "2025-07-02",
"count": 1
},
{
"date": "2025-07-08",
"count": 2
},
{
"date": "2025-07-10",
"count": 1
},
{
"date": "2025-07-13",
"count": 1
},
{
"date": "2025-07-24",
"count": 1
},
{
"date": "2025-07-27",
"count": 1
},
{
"date": "2025-08-01",
"count": 2
},
{
"date": "2025-08-03",
"count": 1
},
{
"date": "2025-08-15",
"count": 1
},
{
"date": "2025-08-19",
"count": 2
},
{
"date": "2025-08-21",
"count": 1
},
{
"date": "2025-08-26",
"count": 1
},
{
"date": "2025-09-03",
"count": 1
},
{
"date": "2025-09-09",
"count": 2
},
{
"date": "2025-09-22",
"count": 1
},
{
"date": "2025-09-23",
"count": 1
},
{
"date": "2025-10-14",
"count": 2
},
{
"date": "2025-10-19",
"count": 1
},
{
"date": "2025-10-27",
"count": 1
},
{
"date": "2025-10-28",
"count": 1
},
{
"date": "2025-10-29",
"count": 1
},
{
"date": "2025-11-03",
"count": 1
},
{
"date": "2025-11-04",
"count": 1
},
{
"date": "2025-11-06",
"count": 2
},
{
"date": "2025-11-12",
"count": 1
},
{
"date": "2025-11-14",
"count": 1
},
{
"date": "2025-11-26",
"count": 1
},
{
"date": "2025-11-27",
"count": 3
},
{
"date": "2025-12-02",
"count": 3
},
{
"date": "2025-12-04",
"count": 1
},
{
"date": "2025-12-29",
"count": 2
},
{
"date": "2026-01-05",
"count": 1
},
{
"date": "2026-01-12",
"count": 1
},
{
"date": "2026-01-19",
"count": 2
},
{
"date": "2026-01-31",
"count": 1
},
{
"date": "2026-02-06",
"count": 1
},
{
"date": "2026-02-09",
"count": 2
},
{
"date": "2026-02-12",
"count": 1
},
{
"date": "2026-02-15",
"count": 2
},
{
"date": "2026-02-24",
"count": 1
},
{
"date": "2026-03-11",
"count": 1
},
{
"date": "2026-03-15",
"count": 1
},
{
"date": "2026-03-18",
"count": 1
},
{
"date": "2026-03-27",
"count": 1
},
{
"date": "2026-04-01",
"count": 1
},
{
"date": "2026-04-07",
"count": 1
},
{
"date": "2026-04-22",
"count": 1
},
{
"date": "2026-04-26",
"count": 1
},
{
"date": "2026-04-30",
"count": 1
},
{
"date": "2026-05-06",
"count": 1
},
{
"date": "2026-05-11",
"count": 1
},
{
"date": "2026-05-17",
"count": 1
},
{
"date": "2026-05-28",
"count": 1
},
{
"date": "2026-05-29",
"count": 1
},
{
"date": "2026-06-07",
"count": 1
},
{
"date": "2026-06-08",
"count": 1
},
{
"date": "2026-06-09",
"count": 1
},
{
"date": "2026-06-26",
"count": 1
},
{
"date": "2026-06-27",
"count": 1
},
{
"date": "2026-06-30",
"count": 1
},
{
"date": "2026-07-01",
"count": 1
},
{
"date": "2026-07-05",
"count": 1
},
{
"date": "2026-07-15",
"count": 1
},
{
"date": "2026-07-21",
"count": 1
}
],
"complete": true,
"collected": 445,
"total_stars": 445
},
"open_issues_and_prs": 3
},
"ai_readiness": {
"has_nix": false,
"example_dirs": [],
"has_llms_txt": false,
"has_dockerfile": false,
"has_mcp_signal": false,
"bootstrap_files": [],
"api_schema_files": [],
"has_devcontainer": false,
"typecheck_configs": [],
"toolchain_manifests": [
"go.mod"
],
"largest_source_bytes": 7852594,
"source_files_sampled": 17,
"oversized_source_files": 4,
"agent_instruction_files": [],
"agent_instruction_max_bytes": null
},
"dependencies": {
"manifests": [
"go.mod"
],
"advisories": {
"error": null,
"scope": null,
"source": null,
"findings": [],
"collected": false,
"malicious": [],
"truncated": false,
"by_severity": {},
"advisory_count": 0,
"affected_count": 0,
"assessed_count": 0,
"malicious_count": 0,
"assessed_package": null,
"unassessed_count": 0,
"direct_affected_count": 0
},
"ecosystems": [
"go"
],
"dependencies": [
{
"name": "github.com/dlclark/regexp2/v2",
"manifest": "go.mod",
"ecosystem": "go",
"version_constraint": "v2.5.1"
}
],
"all_dependencies": {
"error": "GitHub dependency-graph SBOM unavailable (404); the dependency graph may be disabled for this repository",
"source": null,
"packages": [],
"collected": false,
"truncated": false,
"total_count": null,
"direct_count": null,
"indirect_count": null
}
},
"maintainership": {
"issues": {
"open_prs": 1,
"merged_prs": 10,
"open_issues": 2,
"closed_ratio": 0.846,
"closed_issues": 11,
"closed_unmerged_prs": 1
},
"bus_factor": 1,
"bot_contributors": 0,
"top_contributors": [
{
"type": "User",
"login": "bluescreen10",
"commits": 25,
"avatar_url": "https://avatars.githubusercontent.com/u/86094?v=4"
},
{
"type": "User",
"login": "ando-masaki",
"commits": 4,
"avatar_url": "https://avatars.githubusercontent.com/u/4375029?v=4"
},
{
"type": "User",
"login": "espadolini",
"commits": 2,
"avatar_url": "https://avatars.githubusercontent.com/u/886075?v=4"
},
{
"type": "User",
"login": "GOOD21",
"commits": 2,
"avatar_url": "https://avatars.githubusercontent.com/u/3470273?v=4"
},
{
"type": "User",
"login": "amalucelli",
"commits": 1,
"avatar_url": "https://avatars.githubusercontent.com/u/9258376?v=4"
},
{
"type": "User",
"login": "DenisPalnitsky",
"commits": 1,
"avatar_url": "https://avatars.githubusercontent.com/u/154290?v=4"
},
{
"type": "User",
"login": "MarvinJWendt",
"commits": 1,
"avatar_url": "https://avatars.githubusercontent.com/u/31022056?v=4"
},
{
"type": "User",
"login": "philippgille",
"commits": 1,
"avatar_url": "https://avatars.githubusercontent.com/u/170670?v=4"
},
{
"type": "User",
"login": "terwey",
"commits": 1,
"avatar_url": "https://avatars.githubusercontent.com/u/1596124?v=4"
}
],
"contributors_sampled": 9,
"top_contributor_share": 0.658
},
"quality_signals": {
"has_ci": true,
"has_tests": true,
"ci_workflows": [
"go.yml"
],
"has_docs_dir": false,
"linter_configs": [],
"has_editorconfig": false,
"has_linter_config": false,
"has_precommit_config": false
},
"security_signals": {
"lockfiles": [
"go.sum"
],
"scorecard": {
"checks": [
{
"name": "Binary-Artifacts",
"score": 10,
"reason": "no binaries found in the repo",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#binary-artifacts"
},
{
"name": "Branch-Protection",
"score": 0,
"reason": "branch protection not enabled on development/release branches",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#branch-protection"
},
{
"name": "CI-Tests",
"score": 1,
"reason": "1 out of 8 merged PRs checked by a CI test -- score normalized to 1",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#ci-tests"
},
{
"name": "CII-Best-Practices",
"score": 0,
"reason": "no effort to earn an OpenSSF best practices badge detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#cii-best-practices"
},
{
"name": "Code-Review",
"score": 5,
"reason": "Found 8/16 approved changesets -- score normalized to 5",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#code-review"
},
{
"name": "Contributors",
"score": 0,
"reason": "project has 0 contributing companies or organizations -- score normalized to 0",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#contributors"
},
{
"name": "Dangerous-Workflow",
"score": 10,
"reason": "no dangerous workflow patterns detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#dangerous-workflow"
},
{
"name": "Dependency-Update-Tool",
"score": 0,
"reason": "no update tool detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#dependency-update-tool"
},
{
"name": "Fuzzing",
"score": 0,
"reason": "project is not fuzzed",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#fuzzing"
},
{
"name": "License",
"score": 10,
"reason": "license file detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#license"
},
{
"name": "Maintained",
"score": 1,
"reason": "2 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 1",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#maintained"
},
{
"name": "Packaging",
"score": null,
"reason": "packaging workflow not detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#packaging"
},
{
"name": "Pinned-Dependencies",
"score": 0,
"reason": "dependency not pinned by hash detected -- score normalized to 0",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#pinned-dependencies"
},
{
"name": "SAST",
"score": 0,
"reason": "SAST tool is not run on all commits -- score normalized to 0",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#sast"
},
{
"name": "Security-Policy",
"score": 0,
"reason": "security policy file not detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#security-policy"
},
{
"name": "Signed-Releases",
"score": null,
"reason": "no releases found",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#signed-releases"
},
{
"name": "Token-Permissions",
"score": 0,
"reason": "detected GitHub workflow tokens with excessive permissions",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#token-permissions"
},
{
"name": "Vulnerabilities",
"score": 10,
"reason": "0 existing vulnerabilities detected",
"documentation_url": "https://github.com/ossf/scorecard/blob/c395761df6afe1a69e476bc60a013a94bcbc153f/docs/checks.md#vulnerabilities"
}
],
"commit": "ca39f5c7bff9edfc9d70012c5dfdf23b44f7971e",
"ran_at": "2026-07-22T03:59:48Z",
"aggregate_score": 3.5,
"scorecard_version": "v5.5.0"
},
"has_codeql_workflow": false,
"has_security_policy": false,
"has_dependabot_config": false
},
"contribution_flow": {
"collected": true,
"ci_last_run_at": "2026-07-17T16:06:22Z",
"oldest_open_prs": [
{
"number": 25,
"created_at": "2026-07-16T03:56:27Z",
"last_comment_at": "2026-07-17T15:44:04Z",
"last_comment_author": "bluescreen10"
}
],
"last_merged_pr_at": "2025-08-08T21:09:07Z",
"ci_last_conclusion": "SUCCESS",
"oldest_open_issues": [
{
"number": 22,
"created_at": "2025-12-04T08:48:17Z",
"last_comment_at": "2025-12-05T03:54:50Z",
"last_comment_author": "ppipada"
},
{
"number": 23,
"created_at": "2026-02-19T11:17:31Z",
"last_comment_at": null,
"last_comment_author": null
}
]
}
},
"config": {
"disabled_metrics": [],
"disabled_categories": [],
"disabled_components": {}
},
"source": {
"url": "https://github.com/tiktoken-go/tokenizer",
"host": "github.com",
"name": "tokenizer",
"owner": "tiktoken-go"
},
"metrics": {
"overall": {
"key": "overall",
"band": "moderate",
"name": "Overall health",
"note": null,
"notes": [],
"value": 54,
"inputs": {
"security": 35,
"vitality": 64,
"community": 55,
"governance": 60,
"engineering": 50
},
"components": []
},
"categories": [
{
"key": "vitality",
"band": "moderate",
"name": "Vitality",
"value": 64,
"weight": 0.22,
"metrics": [
{
"key": "development_activity",
"band": "at_risk",
"name": "Development activity",
"note": null,
"notes": [],
"value": 46,
"inputs": {
"commits_last_year": 5,
"human_commit_share": 1,
"days_since_last_push": 4,
"active_weeks_last_year": 3
},
"components": [
{
"key": "push_recency",
"name": "Push recency",
"detail": "last push 4 days ago",
"points": 36,
"status": "met",
"details": [
{
"code": "push_recency",
"params": {
"days": 4
}
}
],
"max_points": 36
},
{
"key": "commit_cadence",
"name": "Commit cadence",
"detail": "3/52 weeks with commits",
"points": 2.1,
"status": "partial",
"details": [
{
"code": "commit_cadence_weeks",
"params": {
"weeks": 3
}
}
],
"max_points": 36
},
{
"key": "commit_volume",
"name": "Commit volume",
"detail": "5 commits in the last year",
"points": 7,
"status": "partial",
"details": [
{
"code": "commits_last_year",
"params": {
"count": 5
}
}
],
"max_points": 18
},
{
"key": "openssf_scorecard_maintained",
"name": "OpenSSF Scorecard: Maintained",
"detail": "2 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 1",
"points": 1,
"status": "partial",
"details": [],
"max_points": 10
}
]
},
{
"key": "release_discipline",
"band": "excellent",
"name": "Release discipline",
"note": "Excluded from scoring (no data or not applicable): OpenSSF Scorecard: Signed-Releases. Remaining weights renormalized.",
"notes": [
{
"code": "excluded_no_data",
"params": {
"components": [
"openssf_scorecard_signed_releases"
]
}
},
{
"code": "weights_renormalized",
"params": {}
}
],
"value": 92,
"inputs": {
"releases_count": 13,
"latest_release_tag": "v0.8.1",
"releases_from_tags": false,
"days_since_latest_release": 4,
"mean_days_between_releases": 66.9
},
"components": [
{
"key": "ships_releases",
"name": "Ships releases",
"detail": "13 releases published",
"points": 27,
"status": "met",
"details": [
{
"code": "releases_published",
"params": {
"count": 13
}
}
],
"max_points": 27
},
{
"key": "release_recency",
"name": "Release recency",
"detail": "latest release 4 days ago",
"points": 36,
"status": "met",
"details": [
{
"code": "release_recency",
"params": {
"days": 4
}
}
],
"max_points": 36
},
{
"key": "release_cadence",
"name": "Release cadence",
"detail": "a release every ~66.9 days",
"points": 19.8,
"status": "partial",
"details": [
{
"code": "release_cadence",
"params": {
"gap": 66.9
}
}
],
"max_points": 27
},
{
"key": "openssf_scorecard_signed_releases",
"name": "OpenSSF Scorecard: Signed-Releases",
"detail": "no releases found",
"points": 0,
"status": "excluded",
"details": [
{
"code": "no_data",
"params": {}
}
],
"max_points": 10
}
]
},
{
"key": "abandonment",
"band": "excellent",
"name": "Abandonment",
"note": null,
"notes": [],
"value": 100,
"inputs": {
"cap": null,
"state": "maintained",
"guards": [],
"signals": [],
"red_flag": false,
"multiplier_pct": 100,
"declared_reason": null,
"unverified_reason": null,
"unanswered_open_prs": null,
"unanswered_open_issues": null,
"days_since_last_merged_pr": null,
"days_since_last_human_commit": 4,
"days_since_last_human_commit_is_floor": false
},
"components": [
{
"key": "project_is_still_maintained",
"name": "Project is still maintained",
"detail": "last human commit 4 days ago",
"points": 100,
"status": "met",
"details": [
{
"code": "abandonment_maintained",
"params": {
"days": 4
}
}
],
"max_points": 100
}
]
}
],
"description": "Is the project alive — is code being written and are releases shipping?"
},
{
"key": "community",
"band": "moderate",
"name": "Community & Adoption",
"value": 55,
"weight": 0.18,
"metrics": [
{
"key": "popularity",
"band": "moderate",
"name": "Popularity & adoption",
"note": null,
"notes": [],
"value": 59,
"inputs": {
"forks": 32,
"stars": 445,
"watchers": 6,
"growth_state": "organic",
"growth_factor_pct": 100
},
"components": [
{
"key": "stars",
"name": "Stars",
"detail": "445 stars",
"points": 42.9,
"status": "partial",
"details": [
{
"code": "stars",
"params": {
"count": 445
}
}
],
"max_points": 60
},
{
"key": "forks",
"name": "Forks",
"detail": "32 forks",
"points": 12.4,
"status": "partial",
"details": [
{
"code": "forks",
"params": {
"count": 32
}
}
],
"max_points": 25
},
{
"key": "watchers",
"name": "Watchers",
"detail": "6 watchers",
"points": 3.9,
"status": "partial",
"details": [
{
"code": "watchers",
"params": {
"count": 6
}
}
],
"max_points": 15
}
]
},
{
"key": "community_health",
"band": "moderate",
"name": "Community health",
"note": null,
"notes": [],
"value": 50,
"inputs": {
"has_readme": true,
"has_license": true,
"has_contributing": false,
"has_issue_template": false,
"has_code_of_conduct": false,
"has_pull_request_template": false
},
"components": [
{
"key": "readme",
"name": "README",
"detail": null,
"points": 22.5,
"status": "met",
"details": [],
"max_points": 22.5
},
{
"key": "license",
"name": "License",
"detail": "recognized license (MIT)",
"points": 22.5,
"status": "met",
"details": [
{
"code": "license_standard",
"params": {}
},
{
"code": "license_spdx",
"params": {
"spdx": "MIT"
}
}
],
"max_points": 22.5
},
{
"key": "contributing_guide",
"name": "CONTRIBUTING guide",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 18
},
{
"key": "code_of_conduct",
"name": "Code of conduct",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 13.5
},
{
"key": "issue_template",
"name": "Issue template",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 7.2
},
{
"key": "pr_template",
"name": "PR template",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 6.3
}
]
}
],
"description": "Does the project have users, downloads, attention, and a welcoming setup for contributors?"
},
{
"key": "governance",
"band": "moderate",
"name": "Sustainability & Governance",
"value": 60,
"weight": 0.24,
"metrics": [
{
"key": "maintainer_resilience",
"band": "critical",
"name": "Maintainer resilience (bus factor)",
"note": null,
"notes": [],
"value": 29,
"inputs": {
"bus_factor": 1,
"contributors_sampled": 9,
"top_contributor_share": 0.658
},
"components": [
{
"key": "bus_factor",
"name": "Bus factor",
"detail": "1 contributor(s) cover half of all commits",
"points": 9,
"status": "partial",
"details": [
{
"code": "bus_factor",
"params": {
"count": 1
}
}
],
"max_points": 54
},
{
"key": "commit_distribution",
"name": "Commit distribution",
"detail": "top contributor authored 66% of commits",
"points": 7.7,
"status": "partial",
"details": [
{
"code": "top_contributor_share",
"params": {
"share": 66
}
}
],
"max_points": 22.5
},
{
"key": "contributor_breadth",
"name": "Contributor breadth",
"detail": "9 contributors",
"points": 12.2,
"status": "partial",
"details": [
{
"code": "contributors_sampled",
"params": {
"count": 9
}
}
],
"max_points": 13.5
},
{
"key": "openssf_scorecard_contributors",
"name": "OpenSSF Scorecard: Contributors",
"detail": "project has 0 contributing companies or organizations -- score normalized to 0",
"points": 0,
"status": "missed",
"details": [],
"max_points": 10
}
]
},
{
"key": "responsiveness",
"band": "good",
"name": "Issue & PR responsiveness",
"note": null,
"notes": [],
"value": 82,
"inputs": {
"merged_prs": 10,
"open_issues": 2,
"closed_issues": 11,
"issue_closed_ratio": 0.846,
"closed_unmerged_prs": 1
},
"components": [
{
"key": "issue_resolution",
"name": "Issue resolution",
"detail": "85% of issues closed",
"points": 39.6,
"status": "partial",
"details": [
{
"code": "issues_closed_share",
"params": {
"share": 85
}
}
],
"max_points": 46.75
},
{
"key": "pr_acceptance",
"name": "PR acceptance",
"detail": "10/11 decided PRs merged",
"points": 34.8,
"status": "partial",
"details": [
{
"code": "decided_prs_merged",
"params": {
"merged": 10,
"decided": 11
}
}
],
"max_points": 38.25
},
{
"key": "openssf_scorecard_code_review",
"name": "OpenSSF Scorecard: Code-Review",
"detail": "Found 8/16 approved changesets -- score normalized to 5",
"points": 7.5,
"status": "partial",
"details": [],
"max_points": 15
}
]
},
{
"key": "stewardship",
"band": "at_risk",
"name": "Ownership & stewardship",
"note": null,
"notes": [],
"value": 44,
"inputs": {
"followers": 5,
"owner_type": "Organization",
"is_verified": null,
"owner_login": "tiktoken-go",
"public_repos": 1,
"account_age_days": 1202
},
"components": [
{
"key": "ownership_backing",
"name": "Ownership backing",
"detail": "organization-owned",
"points": 30,
"status": "met",
"details": [
{
"code": "owner_organization",
"params": {}
}
],
"max_points": 30
},
{
"key": "verified_domain",
"name": "Verified domain",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 20
},
{
"key": "owner_reach",
"name": "Owner reach",
"detail": "5 followers of tiktoken-go",
"points": 5.6,
"status": "partial",
"details": [
{
"code": "owner_followers",
"params": {
"count": 5,
"login": "tiktoken-go"
}
}
],
"max_points": 25
},
{
"key": "track_record",
"name": "Track record",
"detail": "1 public repos, account ~3 yr old",
"points": 8.8,
"status": "partial",
"details": [
{
"code": "public_repos",
"params": {
"count": 1
}
},
{
"code": "account_age_years",
"params": {
"years": 3
}
}
],
"max_points": 25
}
]
},
{
"key": "package_maintenance",
"band": "excellent",
"name": "Package maintenance",
"note": null,
"notes": [],
"value": 100,
"inputs": {
"packages": [
"github.com/tiktoken-go/tokenizer"
],
"ecosystems": "go",
"any_deprecated": false,
"min_days_since_publish": 4
},
"components": [
{
"key": "published_resolvable",
"name": "Published & resolvable",
"detail": "1 package(s) on go",
"points": 25,
"status": "met",
"details": [
{
"code": "packages_published",
"params": {
"count": 1,
"ecosystems": "go"
}
}
],
"max_points": 25
},
{
"key": "publish_recency",
"name": "Publish recency",
"detail": "latest publish 4 days ago",
"points": 35,
"status": "met",
"details": [
{
"code": "publish_recency",
"params": {
"days": 4
}
}
],
"max_points": 35
},
{
"key": "version_history",
"name": "Version history",
"detail": "16 published versions",
"points": 20,
"status": "met",
"details": [
{
"code": "published_versions",
"params": {
"count": 16
}
}
],
"max_points": 20
},
{
"key": "not_deprecated",
"name": "Not deprecated",
"detail": "active, not deprecated or yanked",
"points": 20,
"status": "met",
"details": [
{
"code": "package_not_deprecated",
"params": {}
}
],
"max_points": 20
}
]
}
],
"description": "Will the project survive its people — bus factor, responsiveness, who backs it, and package upkeep?"
},
{
"key": "engineering",
"band": "moderate",
"name": "Engineering Quality",
"value": 50,
"weight": 0.2,
"metrics": [
{
"key": "engineering_practices",
"band": "moderate",
"name": "Engineering practices",
"note": null,
"notes": [],
"value": 50,
"inputs": {
"has_ci": true,
"has_tests": true,
"has_editorconfig": false,
"has_linter_config": false,
"has_precommit_config": false
},
"components": [
{
"key": "ci_workflows",
"name": "CI workflows",
"detail": "1 workflow(s)",
"points": 24,
"status": "met",
"details": [
{
"code": "ci_workflows",
"params": {
"count": 1
}
}
],
"max_points": 24
},
{
"key": "tests_present",
"name": "Tests present",
"detail": null,
"points": 24,
"status": "met",
"details": [],
"max_points": 24
},
{
"key": "linter_config",
"name": "Linter config",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 16
},
{
"key": "pre_commit_hooks",
"name": "Pre-commit hooks",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 9.6
},
{
"key": "editorconfig",
"name": ".editorconfig",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 6.4
},
{
"key": "openssf_scorecard_ci_tests",
"name": "OpenSSF Scorecard: CI-Tests",
"detail": "1 out of 8 merged PRs checked by a CI test -- score normalized to 1",
"points": 2,
"status": "partial",
"details": [],
"max_points": 20
}
]
},
{
"key": "documentation",
"band": "moderate",
"name": "Documentation",
"note": null,
"notes": [],
"value": 50,
"inputs": {
"topics": [],
"has_wiki": true,
"homepage": null,
"has_readme": true,
"has_docs_dir": false,
"has_description": true
},
"components": [
{
"key": "readme",
"name": "README",
"detail": null,
"points": 30,
"status": "met",
"details": [],
"max_points": 30
},
{
"key": "documentation_directory",
"name": "Documentation directory",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 25
},
{
"key": "documentation_homepage_site",
"name": "Documentation / homepage site",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 15
},
{
"key": "repository_description",
"name": "Repository description",
"detail": null,
"points": 10,
"status": "met",
"details": [],
"max_points": 10
},
{
"key": "topics",
"name": "Topics",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 10
},
{
"key": "wiki",
"name": "Wiki",
"detail": null,
"points": 10,
"status": "met",
"details": [],
"max_points": 10
}
]
}
],
"description": "Are baseline engineering and documentation practices in place?"
},
{
"key": "security",
"band": "at_risk",
"name": "Security",
"value": 35,
"weight": 0.16,
"metrics": [
{
"key": "security_posture",
"band": "at_risk",
"name": "Security posture",
"note": "Excluded from scoring (no data or not applicable): Packaging, Signed-Releases. Remaining weights renormalized.",
"notes": [
{
"code": "excluded_no_data",
"params": {
"components": [
"packaging",
"signed_releases"
]
}
},
{
"code": "weights_renormalized",
"params": {}
}
],
"value": 35,
"inputs": {
"source": "openssf_scorecard",
"checks_evaluated": 16,
"scorecard_version": "v5.5.0",
"checks_inconclusive": 2,
"scorecard_aggregate": 3.5
},
"components": [
{
"key": "binary_artifacts",
"name": "Binary-Artifacts",
"detail": "no binaries found in the repo",
"points": 7.5,
"status": "met",
"details": [],
"max_points": 7.5
},
{
"key": "branch_protection",
"name": "Branch-Protection",
"detail": "branch protection not enabled on development/release branches",
"points": 0,
"status": "missed",
"details": [],
"max_points": 7.5
},
{
"key": "ci_tests",
"name": "CI-Tests",
"detail": "1 out of 8 merged PRs checked by a CI test -- score normalized to 1",
"points": 0.2,
"status": "partial",
"details": [],
"max_points": 2.5
},
{
"key": "cii_best_practices",
"name": "CII-Best-Practices",
"detail": "no effort to earn an OpenSSF best practices badge detected",
"points": 0,
"status": "missed",
"details": [],
"max_points": 2.5
},
{
"key": "code_review",
"name": "Code-Review",
"detail": "Found 8/16 approved changesets -- score normalized to 5",
"points": 3.8,
"status": "partial",
"details": [],
"max_points": 7.5
},
{
"key": "contributors",
"name": "Contributors",
"detail": "project has 0 contributing companies or organizations -- score normalized to 0",
"points": 0,
"status": "missed",
"details": [],
"max_points": 2.5
},
{
"key": "dangerous_workflow",
"name": "Dangerous-Workflow",
"detail": "no dangerous workflow patterns detected",
"points": 10,
"status": "met",
"details": [],
"max_points": 10
},
{
"key": "dependency_update_tool",
"name": "Dependency-Update-Tool",
"detail": "no update tool detected",
"points": 0,
"status": "missed",
"details": [],
"max_points": 7.5
},
{
"key": "fuzzing",
"name": "Fuzzing",
"detail": "project is not fuzzed",
"points": 0,
"status": "missed",
"details": [],
"max_points": 5
},
{
"key": "license",
"name": "License",
"detail": "license file detected",
"points": 2.5,
"status": "met",
"details": [],
"max_points": 2.5
},
{
"key": "maintained",
"name": "Maintained",
"detail": "2 commit(s) and 0 issue activity found in the last 90 days -- score normalized to 1",
"points": 0.8,
"status": "partial",
"details": [],
"max_points": 7.5
},
{
"key": "packaging",
"name": "Packaging",
"detail": "packaging workflow not detected",
"points": 0,
"status": "excluded",
"details": [
{
"code": "no_data",
"params": {}
}
],
"max_points": 5
},
{
"key": "pinned_dependencies",
"name": "Pinned-Dependencies",
"detail": "dependency not pinned by hash detected -- score normalized to 0",
"points": 0,
"status": "missed",
"details": [],
"max_points": 5
},
{
"key": "sast",
"name": "SAST",
"detail": "SAST tool is not run on all commits -- score normalized to 0",
"points": 0,
"status": "missed",
"details": [],
"max_points": 5
},
{
"key": "security_policy",
"name": "Security-Policy",
"detail": "security policy file not detected",
"points": 0,
"status": "missed",
"details": [],
"max_points": 5
},
{
"key": "signed_releases",
"name": "Signed-Releases",
"detail": "no releases found",
"points": 0,
"status": "excluded",
"details": [
{
"code": "no_data",
"params": {}
}
],
"max_points": 7.5
},
{
"key": "token_permissions",
"name": "Token-Permissions",
"detail": "detected GitHub workflow tokens with excessive permissions",
"points": 0,
"status": "missed",
"details": [],
"max_points": 7.5
},
{
"key": "vulnerabilities",
"name": "Vulnerabilities",
"detail": "0 existing vulnerabilities detected",
"points": 7.5,
"status": "met",
"details": [],
"max_points": 7.5
}
]
},
{
"key": "high_risk_jurisdiction_exposure",
"band": "excellent",
"name": "High-Risk Jurisdiction Exposure",
"note": "Only high-confidence self-published location evidence affects this multiplier. Ambiguous matches are review-only; country evidence is not proof of nationality, citizenship, legal registration, malicious intent, or sanctions status.",
"notes": [
{
"code": "jurisdiction_evidence_limits",
"params": {}
}
],
"value": 100,
"inputs": {
"meaning": "self-published location evidence; not nationality or citizenship",
"red_flag": false,
"exposures": [],
"policy_countries": [
"Russia",
"Iran",
"North Korea"
],
"review_only_matches": 0,
"assessed_self_published_locations": 13
},
"components": [
{
"key": "policy_exposure_multiplier",
"name": "Policy exposure multiplier",
"detail": "no confirmed policy-scope location match",
"points": 100,
"status": "met",
"details": [
{
"code": "jurisdiction_no_match",
"params": {}
}
],
"max_points": 100
}
]
}
],
"description": "Are visible security and supply-chain practices strong, with no malicious dependency and no unresolved high-risk jurisdiction exposure?"
},
{
"key": "ai_readiness",
"band": "moderate",
"name": "AI Readiness",
"value": 51,
"weight": 0,
"metrics": [
{
"key": "ai_agent_context",
"band": "critical",
"name": "Agent context & guidance",
"note": null,
"notes": [],
"value": 27,
"inputs": {
"has_llms_txt": false,
"legible_history_share": 0.5,
"agent_instruction_files": [],
"agent_instruction_max_bytes": null
},
"components": [
{
"key": "agent_instructions",
"name": "Agent instructions",
"detail": "no CLAUDE.md / AGENTS.md / editor rules",
"points": 0,
"status": "missed",
"details": [
{
"code": "no_agent_instructions",
"params": {}
}
],
"max_points": 45
},
{
"key": "machine_readable_docs_llms_txt",
"name": "Machine-readable docs (llms.txt)",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 15
},
{
"key": "legible_commit_history",
"name": "Legible commit history",
"detail": "22 of 44 human commits state their intent (structured subject or explanatory body)",
"points": 26.7,
"status": "partial",
"details": [
{
"code": "legible_history",
"params": {
"legible": 22,
"sampled": 44
}
}
],
"max_points": 40
}
]
},
{
"key": "ai_verify_loop",
"band": "moderate",
"name": "Verify loop (build / test / typecheck)",
"note": null,
"notes": [],
"value": 56,
"inputs": {
"has_nix": false,
"has_tests": true,
"lockfiles": [
"go.sum"
],
"has_dockerfile": false,
"typed_language": true,
"bootstrap_files": [],
"has_devcontainer": false,
"has_linter_config": false,
"typecheck_configs": [],
"agent_commit_share": 0,
"toolchain_manifests": [
"go.mod"
],
"dependency_bot_commit_share": 0
},
"components": [
{
"key": "one_command_bootstrap",
"name": "One-command bootstrap",
"detail": "go.mod (toolchain convention, no task runner)",
"points": 12.6,
"status": "partial",
"details": [
{
"code": "toolchain_convention",
"params": {
"files": "go.mod"
}
}
],
"max_points": 18
},
{
"key": "automated_tests",
"name": "Automated tests",
"detail": null,
"points": 22,
"status": "met",
"details": [],
"max_points": 22
},
{
"key": "lint_format_config",
"name": "Lint / format config",
"detail": null,
"points": 0,
"status": "missed",
"details": [],
"max_points": 11
},
{
"key": "static_type_checking",
"name": "Static type checking",
"detail": "Go (statically typed)",
"points": 11,
"status": "met",
"details": [
{
"code": "statically_typed_language",
"params": {
"language": "Go"
}
}
],
"max_points": 11
},
{
"key": "reproducible_environment",
"name": "Reproducible environment",
"detail": "lockfile",
"points": 10,
"status": "met",
"details": [
{
"code": "file_list",
"params": {
"files": "lockfile"
}
}
],
"max_points": 10
},
{
"key": "demonstrated_agent_practice",
"name": "Demonstrated agent practice",
"detail": "no agent-authored commits among the last 44",
"points": 0,
"status": "missed",
"details": [
{
"code": "no_agent_authored_commits",
"params": {
"sampled": 44
}
}
],
"max_points": 10
},
{
"key": "automated_maintenance",
"name": "Automated maintenance",
"detail": "no automated dependency updates observed",
"points": 0,
"status": "missed",
"details": [
{
"code": "no_dependency_automation",
"params": {}
}
],
"max_points": 8
},
{
"key": "openssf_scorecard_pinned_dependencies",
"name": "OpenSSF Scorecard: Pinned-Dependencies",
"detail": "dependency not pinned by hash detected -- score normalized to 0",
"points": 0,
"status": "missed",
"details": [],
"max_points": 10
}
]
},
{
"key": "ai_code_legibility",
"band": "excellent",
"name": "Code legibility for models",
"note": null,
"notes": [],
"value": 87,
"inputs": {
"primary_language": "Go",
"largest_source_bytes": 7852594,
"source_files_sampled": 17,
"oversized_source_files": 4
},
"components": [
{
"key": "type_checkable_code",
"name": "Type-checkable code",
"detail": "Go (statically typed)",
"points": 45,
"status": "met",
"details": [
{
"code": "statically_typed_language",
"params": {
"language": "Go"
}
}
],
"max_points": 45
},
{
"key": "manageable_file_sizes",
"name": "Manageable file sizes",
"detail": "4/17 source files over 60KB",
"points": 42.1,
"status": "partial",
"details": [
{
"code": "oversized_source_files",
"params": {
"kb": 60,
"sampled": 17,
"oversized": 4
}
}
],
"max_points": 55
}
]
}
],
"description": "How well is the repo equipped to be developed and maintained with AI coding agents? An independent, experimental badge — weight 0.0, so it is surfaced on its own and does not affect the overall health score."
}
],
"metrics_version": "1.13.0"
},
"warnings": [
"GitHub dependency-graph SBOM unavailable (404); the dependency graph may be disabled for this repository"
],
"report_type": "repository",
"generated_at": "2026-07-22T04:00:03.666069Z",
"schema_version": "0.26.0",
"badge_url": "https://raw.githubusercontent.com/inspect-software/badges/main/v1/t/tiktoken-go/tokenizer.svg",
"full_name": "tiktoken-go/tokenizer",
"license_state": "standard",
"license_spdx": "MIT"
}Bewertungen sind Signale, keine Garantien. Sie spiegeln öffentlich sichtbare Praxis auf GitHub wider — kein Code-Audit und keine Sicherheitsgarantie.
Fehlende Daten werden ausgeschlossen und die Gewichte neu normiert, nie als null bewertet. Die Methodik ist versioniert und offen: Metriken v1.13.0, Schema v0.26.0 — vollständige Methodik · Metriken-Wiki.
Wie ein einzelnes Ergebnis im Gesamtregister steht: aggregierte Statistiken — Go.