#nlp

SWE-bench: Can Language Models Resolve Real-world Github Issues?

Fast and robust date extraction from web pages, with Python or on the command-line

React library to integrate a Memori in your app or website

A CoNLL-U parser that takes a CoNLL-U formatted string and turns it into a nested python dictionary.

LLM prompt language based on Jinja. Banks provides tools and functions to build prompts text and chat messages from generic blueprints. It allows attaching metadata to prompts to ease their management, and versioning is first-class citizen. Banks provides ways to store prompts on disk along with their metadata.

A fast, efficient natural language processing engine for Icelandic.

A Python library for exploring the Constitution of India.

CoreNLP: A Java suite of core NLP tools for tokenization, sentence segmentation, NER, parsing, coreference, sentiment analysis, etc.

🔮 A refreshing functional take on deep learning, compatible with your favorite libraries

Reference BLEU implementation that auto-downloads test sets and reports a version string to facilitate cross-lab comparisons

NLP engine code

Heuristic code and docs metrics for spotting AI slop, overengineering, and filler at a glance.

Zero-config NL-to-SQL data engine with multi-tenant isolation, built for embedding in multi-tenant SaaS apps.

Semantic similarity testing for Elixir. Test LLM outputs, chatbots, and NLP in Elixir

Pure-Go LLM tokenizer and tiktoken-compatible token counter for OpenAI BPE, WordPiece, SentencePiece, Gemini, Llama, Mistral, and Hugging Face adapters.

A fast, lightweight and easy-to-use Python library for splitting text into semantically meaningful chunks.

초경량 한국어 형태소 분석기 — 1MB 모델로 브라우저에서 실행(WASM). Ultra-lightweight Korean morphological analyzer in the browser. npm: garu-ko

Microsoft.Recognizers.Text provides recognition and resolution of numbers, units, date/time, etc. in multiple languages (ZH, EN, FR, ES, PT, DE, IT, TR, HI, NL. Partial support for JA, KO, AR, SV). Packages available at: https://www.nuget.org/profiles/Recognizers.Text, https://www.npmjs.com/~recognizers.text

Zero-dependency script & phoneme-notation core — ISO-15924 detection & metadata, IPA ↔ ARPABET/X-SAMPA/Lexique/Kirshenbaum/Cotovía/RFE, Buckwalter ↔ Arabic, Hangul → jamo, kana

🛸 Use pretrained transformers like BERT, XLNet and GPT-2 in spaCy

A specialist Python toolkit for Ancient Greek - alphabetic Greek NLP (incl. a neural pipeline) and the Aegean syllabic scripts (Linear A, Linear B, Cypriot, Cypro-Minoan)

Node.js bindings for the NLP++ text-analysis engine (npm peer of the Python package NLPPlus)

A command-line tool to extract plain text from Wikipedia dumps with category and section filtering

Smart expansion of Spanish legal acronyms. 647 verified acronyms from RAE, BOE and DPEJ. | Expansión inteligente de siglas legales españolas.

A Rust port of Microsoft Presidio — PII detection & anonymization (analyzer + anonymizer + Python bindings)

Provider-agnostic Go AI framework for building RAG, agents, and AI-powered applications. Swap LLM providers without code changes — supports Ollama, OpenAI, and more.

Python port of Moses tokenizer, truecaser and normalizer

Language model tokenization at GB/s

Fast and accurate natural language detection. Detector written in PHP. Nito-ELD, ELD.