All tags
Catalogue tag

#tokenizers

Every repository in the public record carrying this tag — from its GitHub topics or the keywords its package registries publish. Health is measured under the same versioned methodology as the rest of the record.

1 record
Tagged “tokenizers”Ranked by health index
PyPI · crates.io
50Moderatehealth index
marcelroed/gigatoken
Language model tokenization at GB/s
Rust · Python★ 115↓ 2,741/moJul 22, 2026
MITJul 22, 2026 · metrics 1.13.0