PyPI · npm
NVIDIA-NeMo/Curator
Scalable data pre processing and curation toolkit for LLMs
Apache-2.0Sep 25, 2026 · metrics 2.10.0
All tags
Catalogue tag
#data-processing
Every repository in the public record carrying this tag — from its GitHub topics or the keywords its package registries publish. Health is measured under the same versioned methodology as the rest of the record.
35 records
npm · Go · crates.io
numaproj/numaflow
Kubernetes-native platform to run massively parallel data/streaming jobs
Apache-2.0Oct 21, 2026 · metrics 2.10.0
PyPI · Go
Lightning-AI/litData
Speed up model training by fixing data loading.
Apache-2.0Oct 20, 2026 · metrics 2.10.0
npm
Siteimprove/alfa
:wheelchair: Suite of open and standards-based tools for performing reliable accessibility conformance testing at scale
MITSep 15, 2026 · metrics 2.10.0
PyPI
datajuicer/data-juicer
Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷
Apache-2.0Sep 16, 2026 · metrics 2.10.0
Go · PyPI
johnkerl/miller
Miller is like awk, sed, cut, join, and sort for name-indexed data such as CSV, TSV, and tabular JSON
Custom licenseOct 4, 2026 · metrics 2.10.0
PyPI
Nike-Inc/koheesio
Python framework for building efficient data pipelines. It promotes modularity and collaboration, enabling the creation of complex pipelines from simple, reusable components.
Apache-2.0Oct 16, 2026 · metrics 2.10.0
PyPI · crates.io
pathwaycom/pathway
Python ETL framework for stream processing, real-time analytics, LLM pipelines, and RAG.
Custom licenseSep 20, 2026 · metrics 2.10.0
PyPI
unionai-oss/pandera
A light-weight, flexible, and expressive statistical data testing library
MITAug 28, 2026 · metrics 2.10.0
PyPI · npm
LiberTEM/LiberTEM
Open pixelated STEM framework
MITAug 23, 2026 · metrics 2.10.0
PyPI
OpenDCAI/DataFlow
[SIGMOD'27] Easy Data Preparation with latest LLMs-based Operators and Pipelines.
Apache-2.0Sep 19, 2026 · metrics 2.10.0
PyPI
andrewbolster/bolster
Brain of Bolster, Be Ware
Custom licenseSep 19, 2026 · metrics 2.10.0
PyPI
lasp/space_packet_parser
A CCSDS telemetry packet decoding library based on the XTCE packet format description standard.
Custom licenseSep 15, 2026 · metrics 2.10.0
Packagist · PyPI
cognesy/instructor-php
Unified LLM API, structured data outputs with LLMs, and agent SDK - in PHP
MITSep 12, 2026 · metrics 2.10.0
PyPI
datagouv/csv-detective
Inspection of tabular (csv, xls-like, parquet) files to guess the columns' content
MITOct 20, 2026 · metrics 2.10.0
PyPI
owid/etl
A compute graph for loading and transforming OWID's data
MITOct 13, 2026 · metrics 2.10.0
—
Puchaczov/Musoq
SQL Runtime without any database
MITSep 10, 2026 · metrics 2.10.0
npm
zazuko/barnard59
An intuitive and flexible RDF pipeline solution designed to simplify and automate ETL processes for efficient data management.
No licenseSep 15, 2026 · metrics 2.10.0
npm · PyPI
melihbirim/csvql
SQL queries for CSV files. The analytical CSV query engine for AI agents cli: csvql.
MITSep 13, 2026 · metrics 2.10.0
Hex
elixir-broadway/broadway
Concurrent and multi-stage data ingestion and data processing with Elixir
Apache-2.0Sep 19, 2026 · metrics 2.10.0
PyPI
okfn-brasil/querido-diario-data-processing
Text processing repository to free brazilian municipal gazettes from closed file formats for the Querido Diário project.
MITSep 13, 2026 · metrics 2.10.0
PyPI
Alex870521/AeroViz
A Python package designed for aerosol science, providing various data processing and visualization tools.
MITOct 20, 2026 · metrics 2.10.0
npm
Smoren/transferum-ts
A language for describing interactions between components
MITSep 18, 2026 · metrics 2.10.0
Packagist
veryfi/veryfi-php
PHP module for communicating with the Veryfi OCR API.
MITOct 19, 2026 · metrics 2.10.0
PyPI
mmore500/joinem
CLI for fast, flexbile concatenation of tabular data using polars
MITOct 15, 2026 · metrics 2.10.0
Go
rosscartlidge/ssql
SQL-style stream processing for the command line and Go
MITSep 6, 2026 · metrics 2.10.0
Packagist
koolreport/core
An Open Source PHP Reporting Framework that helps you to write perfect data reports or to construct awesome dashboards in PHP. Working great with all PHP versions from 5.6 to latest 8.0. Fully compatible with all kinds of MVC frameworks like Laravel, CodeIgniter, Symfony.
MITSep 17, 2026 · metrics 2.10.0
PyPI
NVIDIA/nvImageCodec
A nvImageCodec library of GPU- and CPU- accelerated codecs featuring a unified interface
Apache-2.0Sep 11, 2026 · metrics 2.10.0
Packagist
fab2s/YaEtl
Yet Another ETL in PHP
MITSep 15, 2026 · metrics 2.10.0
Packagist
fab2s/NodalFlow
A PHP Nodal (net)WorkFlow
MITSep 15, 2026 · metrics 2.10.0