Maven
apache/shardingsphere
Empowering Data Intelligence with Distributed SQL for Sharding, Scalability, and Security Across All Databases.
Apache-2.0Aug 5, 2026 · metrics 2.10.0
All tags
Catalogue tag
#data-pipeline
Every repository in the public record carrying this tag — from its GitHub topics or the keywords its package registries publish. Health is measured under the same versioned methodology as the rest of the record.
28 records
Go
rudderlabs/rudder-server
Privacy and Security focused Segment-alternative, in Golang and React
Custom licenseAug 28, 2026 · metrics 2.10.0
PyPI
elementary-data/elementary
The dbt-native data observability solution for data & analytics engineers. Monitor your data pipelines in minutes. Available as self-hosted or cloud service with premium features.
Apache-2.0Aug 13, 2026 · metrics 2.10.0
Maven · npm
airbytehq/airbyte
Open-source data movement for ELT pipelines and AI agents — from APIs, databases & files to warehouses, lakes, and AI applications. Both self-hosted and Cloud.
Custom licenseAug 26, 2026 · metrics 2.10.0
PyPI · Go · npm
bruin-data/ingestr
ingestr is a CLI tool to copy data between any databases with a single command seamlessly.
Custom licenseJul 17, 2026 · metrics 2.10.0
PyPI
datajuicer/data-juicer
Data processing for and with foundation models! 🍎 🍋 🌽 ➡️ ➡️🍸 🍹 🍷
Apache-2.0Aug 3, 2026 · metrics 2.10.0
Go
datazip-inc/olake
OLake - Fastest Databases, Kafka & S3 Replication to Apache Iceberg with Table optimization (Called OLake Fusion). ⚡ Efficient, quick and scalable data ingestion for real-time analytics. Supported sources : Postgres, MongoDB, MySQL, Oracle, MSSql, DB2, Kafka, S3.
Apache-2.0Aug 31, 2026 · metrics 2.10.0
Go
ConduitIO/conduit
Conduit streams data between data stores. Kafka Connect replacement. No JVM required.
Apache-2.0Jul 17, 2026 · metrics 2.10.0
Maven
debezium/debezium
Change data capture for a variety of databases. Please log issues at https://github.com/debezium/dbz/issues.
Apache-2.0Aug 27, 2026 · metrics 2.10.0
RubyGems · npm
multiwoven/multiwoven
🔥🔥🔥 Open source Reverse ETL - alternative to hightouch and census.
AGPL-3.0Jul 17, 2026 · metrics 2.10.0
Go · crates.io · npm +1
estuary/flow
🌊 Continuously synchronize the systems where your data lives, to the systems where you _want_ it to live, by managing your data flows with Estuary. 🌊
Custom licenseJul 17, 2026 · metrics 2.10.0
PyPI
Glitni/dlt-saga
Config-driven data ingestion and SCD2 historization framework built on dlt
Apache-2.0Aug 3, 2026 · metrics 2.10.0
Go
chalk-ai/chalk-go
Go client for Chalk
Apache-2.0Jul 16, 2026 · metrics 2.10.0
PyPI
maltzsama/dagster-authkit
Community Auth System for self-hosted Dagster OSS - simple RBAC, Audit-log, and Session Management
Apache-2.0Aug 22, 2026 · metrics 2.10.0
PyPI
olirice/flupy
Fluent data pipelines for python and your shell
Custom licenseJul 27, 2026 · metrics 2.10.0
PyPI
russalo/file-observer
Deterministic file observation for pipelines — one read-only pass over a directory emits a reproducible JSON manifest of every file's type, metadata, structure, and provenance.
Custom licenseJul 17, 2026 · metrics 2.10.0
PyPI
PFund-Software-Ltd/pfeed
Data Engine for AI/Algo Trading: Download/Stream -> Clean -> Store. Supports Data Lakehouse Architecture. Clean Once and Forget.
Apache-2.0Jul 20, 2026 · metrics 2.10.0
PyPI
ebarti/jobstreaming
Concurrent, resumable job collection for Python with typed events, durable checkpoints, normalized models, and a DataFrame API.
MITAug 29, 2026 · metrics 2.10.0
PyPI · crates.io
apitap/apitap-lib
Move whole tables between databases fast — Postgres, MySQL, ClickHouse, BigQuery. Rust engine, one-line Python API, bounded memory.
MITAug 18, 2026 · metrics 2.10.0
PyPI · Go · Maven
liam0205/pineapple
Multi-runtime DAG pipeline engine — declare in Python, execute in Go / Java / Python, decouple with JSON.
Apache-2.0Jul 17, 2026 · metrics 2.10.0
PyPI
Jackxiaozhiren/datasentry
Local-first AI copilot for data quality: 39 detectors, six-dimension scoring, AI repair with human approval, drift engine, cron scheduling with distributed worker pool (failover + parallel), MCP/REST/CLI/Web UI. Apache-2.0.
Apache-2.0Aug 16, 2026 · metrics 2.10.0
Go · npm
Tnsor-Labs/brokoli
Brokoli — self-hosted data pipeline orchestration
Apache-2.0Jul 25, 2026 · metrics 2.10.0
PyPI · npm
N283T/pdb-mine-builder
Build a Mine-schema database from PDB data. Load PDBj structural biology data into PostgreSQL with RDKit chemical search support.
MITJul 15, 2026 · metrics 2.10.0
PyPI
pydoit/doit
CLI task management & automation tool
MITAug 13, 2026 · metrics 2.10.0
Go
derickschaefer/reserve
A modern command-line interface (CLI) for interfacing with the Federal Reserve Bank of St. Louis FRED API.
Custom licenseAug 20, 2026 · metrics 2.10.0
PyPI
mr3od/bgate-unix
High-performance, hardware-aware Layer 0 deduplication engine. Uses 4-tier short-circuit logic (Size → Fringe → Full Hash) with xxHash128 to eliminate exact duplicates before they enter your processing pipeline. Python 3.11+, uv, SQLite.
MITAug 14, 2026 · metrics 2.10.0
Go · PyPI
zoyluoblue/cronova
Lightweight self-hosted workflow scheduler in a single Go binary — an open-source Airflow/Azkaban alternative. Embedded SQLite, DAGs (cron + dependencies + backfill), polyglot tasks (shell/Python/SQL/JAR/HTTP), web console, REST API, and a built-in MCP server for AI agents. Linux & macOS.
MITJul 23, 2026 · metrics 2.10.0
Maven · PyPI
whylabs/whylogs
An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-preserving data collection, ensuring safety & robustness. 📈
Apache-2.0Jul 21, 2026 · metrics 2.10.0