npm
flex-development/mark
a specification for building syntax-aware parsers, compilers, and text processing systems
BSD-3-ClauseSep 18, 2026 · metrics 2.10.0
All tags
Catalogue tag
#tokenize
Every repository in the public record carrying this tag — from its GitHub topics or the keywords its package registries publish. Health is measured under the same versioned methodology as the rest of the record.
12 records
npm
micromark/micromark
small, safe, and great commonmark (optionally gfm, mdx) compliant markdown parser
MITSep 16, 2026 · metrics 2.10.0
PyPI
gristlabs/asttokens
Annotate Python AST trees with source text and token information
Apache-2.0Sep 12, 2026 · metrics 2.10.0
npm
syntax-tree/mdast-util-from-markdown
mdast utility to parse markdown
MITSep 16, 2026 · metrics 2.10.0
Packagist
loupe-php/matcher
A library to highlight and crop terms and context within strings such as search results
MITSep 10, 2026 · metrics 2.10.0
npm
syntax-tree/hast-util-from-html
hast utility to parse from HTML
MITSep 15, 2026 · metrics 2.10.0
npm
syntax-tree/esast-util-from-js
estree (and esast) utility to parse from JavaScript
MITSep 15, 2026 · metrics 2.10.0
PyPI
OpenVoiceOS/quebra_frases
chunks strings into byte sized pieces
Apache-2.0Sep 12, 2026 · metrics 2.10.0
npm
here-be/snapdragon
snapdragon is an extremely pluggable, powerful and easy-to-use parser-renderer factory.
MITSep 15, 2026 · metrics 2.10.0
crates.io
stevedonovan/scanlex
A simple lexical scanner in Rust
MITSep 13, 2026 · metrics 2.10.0
PyPI
myint/untokenize
Transforms tokens into original source code (while preserving whitespace)
MITSep 15, 2026 · metrics 2.10.0
npm
qatrehai/persian-normalize
Normalize Persian and Arabic-script text — ZWNJ, Arabic yeh/kaf, three digit sets, diacritics. Zero dependencies, TypeScript types.
MITOct 18, 2026 · metrics 2.10.0