crates.io
RazrFalcon/xmlparser
A low-level, pull-based, zero-allocation XML 1.0 parser.
Apache-2.018 sept 2026 · métricas 2.10.0
Todas las etiquetas
Etiqueta del catálogo
#tokenizer
Todos los repositorios del registro público que llevan esta etiqueta, procedente de sus topics de GitHub o de las palabras clave que publican sus registros de paquetes. La salud se mide con la misma metodología versionada que el resto del registro.
76 registros
npm · Maven · RubyGems
leonsilicon/react-native-jieba
Turbo Module integrating cppjieba for React Native.
MIT5 oct 2026 · métricas 2.10.0
crates.io
savannstm/language-tokenizer
Text tokenizer for linguistic purposes, such as text matching. Supports more than 40 languages, including English, French, Russian, Japanese, Thai etc.
WTFPL19 oct 2026 · métricas 2.10.0
npm
ericprud/ts-jison
Bison in JavaScript.
MIT7 oct 2026 · métricas 2.10.0
Packagist
sebastianbergmann/php-token-stream
Wrapper around PHP's tokenizer extension.
Licencia propia27 ago 2026 · métricas 2.10.0
PyPI
bitextor/bitextor
Bitextor generates translation memories from multilingual websites
GPL-3.012 sept 2026 · métricas 2.10.0
Packagist
phplrt/lexer
[READ ONLY] Subtree split of the phplrt lexer component (see phplrt/phplrt)
MIT13 sept 2026 · métricas 2.10.0
Packagist
phplrt/lexer-builder
[READ ONLY] Subtree split of the phplrt lexer component (see phplrt/phplrt)
MIT15 sept 2026 · métricas 2.10.0
Packagist
phplrt/lexer-contracts
[READ ONLY] Subtree split of the phplrt lexer contracts component (see phplrt/phplrt)
MIT13 sept 2026 · métricas 2.10.0
—
JuliaLang/Tokenize.jl
Tokenization for Julia source code
Licencia propia12 sept 2026 · métricas 2.10.0
crates.io
fschutt/simplecss
A very simple CSS 2.1 tokenizer
MPL-2.023 ago 2026 · métricas 2.10.0
crates.io
spider-rs/markup5ever
Send-able fork of markup5ever, depends on spider-tendril
Apache-2.015 sept 2026 · métricas 2.10.0
npm
Borewit/tokenizer-token
TypeScript definition for strtok3 token
Sin licencia23 sept 2026 · métricas 2.10.0
Packagist
chriskonnertz/string-calc
PHP calculator library for mathematical terms (expressions) passed as strings
MIT18 oct 2026 · métricas 2.10.0
Packagist
nette/tokenizer
[DISCONTINUED] Source code tokenizer
Licencia propia18 sept 2026 · métricas 2.10.0
npm
no-context/moo
Optimised tokenizer/lexer generator! 🐄 Uses /y for performance. Moo.
BSD-3-Clause19 sept 2026 · métricas 2.10.0