#nlp

Zero-dependency .NET library for RAG text preparation: multilingual cleaning, sensitive data masking, content filtering, and semantic chunking across 14 languages.

Microsoft.Recognizers.Text provides recognition and resolution of numbers, units, date/time, etc. in multiple languages (ZH, EN, FR, ES, PT, DE, IT, TR, HI, NL. Partial support for JA, KO, AR, SV). Packages available at: https://www.nuget.org/profiles/Recognizers.Text, https://www.npmjs.com/~recognizers.text

Language Sample Analysis Tools for CHAT

Zero-dependency script & phoneme-notation core — ISO-15924 detection & metadata, IPA ↔ ARPABET/X-SAMPA/Lexique/Kirshenbaum/Cotovía/RFE, Buckwalter ↔ Arabic, Hangul → jamo, kana

Common stop words in a variety of languages

🛸 Use pretrained transformers like BERT, XLNet and GPT-2 in spaCy

Fast Cantonese text-to-Jyutping (G2P) — Rust core, Python (PyO3). Numbers, dates, and HK English code-switching.

A fast and memory-efficient PHP text chunker

spokenform converts plain written text in one selected language into reviewable text intended for speech systems. It is a text-to-text frontend: written text in, spoken form out.

PHP Sentiment Analyzer is a lexicon and rule-based sentiment analysis tool for PHP, built on the VADER (Valence Aware Dictionary and Sentiment Reasoner) sentiment lexicon.

Collection of language detection libraries for .NET: FastText, CLD2, CLD3, MediaPipe, Lingua, Whatlang

🧹 Python package for text cleaning

Babel Street Analytics Client Library for Ruby

English Part-of-Speech Tagger Library; a Ruby port of Lingua::EN::Tagger

A wrapper module for using spaCy natural language processing library from the Ruby programming language via PyCall

Multilingual, context-aware abbreviation expansion for text normalization and speech.

中文 NLP 预处理、解析工具包，准确、高效、易用 A Chinese NLP Preprocessing & Parsing Package www.jionlp.com

Inference engine for GLiNER models, in Rust

Fast and Portable Character String Processing in R (with the Unicode ICU)

Semantic similarity testing for Elixir. Test LLM outputs, chatbots, and NLP in Elixir

No repository description published.

:book: A Golang library for text processing, including tokenization, part-of-speech tagging, and named-entity extraction.

A Rust port of Microsoft Presidio — PII detection & anonymization (analyzer + anonymizer + Python bindings)

Rust client for txtai

Node.js bindings for the NLP++ text-analysis engine (npm peer of the Python package NLPPlus)

Pre-built Snowball stemmers for Rust (tracks snowball main)

A tokenizer based on Unicode text segmentation (UAX #29), for Go. Split graphemes, words, sentences.

🚀 Catalyst is a C# Natural Language Processing library built for speed. Inspired by spaCy's design, it brings pre-trained models, out-of-the box support for training word and document embeddings, and flexible entity recognition models.

JavaScript client for txtai

Pure-Go LLM tokenizer and tiktoken-compatible token counter for OpenAI BPE, WordPiece, SentencePiece, Gemini, Llama, Mistral, and Hugging Face adapters.