All tags
Catalogue tag

#tokenize

Every repository in the public record carrying this tag — from its GitHub topics or the keywords its package registries publish. Health is measured under the same versioned methodology as the rest of the record.

5 records
Tagged “tokenize”Ranked by health index
Packagist
61Moderatehealth index
loupe-php/matcher
A library to highlight and crop terms and context within strings such as search results
PHP★ 10↓ 20.6K/moJul 17, 2026
MITJul 17, 2026 · metrics 1.13.0
npm
55Moderatehealth index
micromark/micromark
small, safe, and great commonmark (optionally gfm, mdx) compliant markdown parser
JavaScript★ 2,195↓ 1.3B/moJul 22, 2026
MITJul 22, 2026 · metrics 1.13.0
PyPI
48At riskhealth index
OpenVoiceOS/quebra_frases
chunks strings into byte sized pieces
Python★ 1Jul 21, 2026
Apache-2.0Jul 21, 2026 · metrics 1.13.0
PyPI
32At riskhealth index
gristlabs/asttokens
Annotate Python AST trees with source text and token information
Python★ 192Jul 20, 2026
Apache-2.0Jul 20, 2026 · metrics 1.13.0