PyPI
Query-farm/vgi-python
Python SDK for the VGI (Vector Gateway Interface) protocol — host DuckDB scalar, table, aggregate, and table-in-out functions in an external worker over Arrow IPC.
Custom licenseSep 13, 2026 · metrics 2.10.0
All tags
Catalogue tag
#data-engineering
Every repository in the public record carrying this tag — from its GitHub topics or the keywords its package registries publish. Health is measured under the same versioned methodology as the rest of the record.
102 records
PyPI · crates.io · npm
glyf-data/glyf
Open Source Visualization build tool to data pipeline
Apache-2.0Oct 19, 2026 · metrics 2.10.0
PyPI
leonardovida/duckdb-sqlalchemy
Production-ready SQLAlchemy dialect for DuckDB and MotherDuck with operational defaults and migration support from duckdb_engine.
MITSep 15, 2026 · metrics 2.10.0
Go
okfcli/okf
Vendor-neutral Go CLI for the Open Knowledge Format. Create, validate, lint, index, search, and graph OKF bundles. Built to be driven by AI agents.
Apache-2.0Sep 18, 2026 · metrics 2.10.0
PyPI
PFund-Software-Ltd/pfeed
Data Engine for AI/Algo Trading: Download/Stream -> Clean -> Store. Supports Data Lakehouse Architecture. Clean Once and Forget.
Apache-2.0Sep 12, 2026 · metrics 2.10.0
Go · npm
flidai/leapview
Open-source BI for governed analytics, without the lock-in.
Apache-2.0Sep 20, 2026 · metrics 2.10.0
PyPI
JB-Analytica/model2data
model2data turns data models into analytics-ready datasets.
MITOct 2, 2026 · metrics 2.10.0
PyPI
PaulKov/dpone
Declarative ETL framework for YAML-driven data pipelines
Apache-2.0Sep 13, 2026 · metrics 2.10.0
PyPI
Query-farm/vgi-lint-check
Lint the documentation & metadata quality of VGI (Vector Gateway Interface) data workers — descriptions, column comments, tags, and example queries — with a quality score, per-version baselines, and agent-friendly output.
Custom licenseSep 11, 2026 · metrics 2.10.0
crates.io · PyPI
mag1cfrog/delta-funnel
Fast, lightweight Delta Lake to SQL Server loads without Spark or ODBC.
Apache-2.0Sep 9, 2026 · metrics 2.10.0
NuGet
npipeline/NPipeline
High-performance, streaming data pipelines for .NET
Custom licenseSep 15, 2026 · metrics 2.10.0
PyPI
Jackxiaozhiren/datasentry
Find, explain, and safely fix bad data. Local-first data quality for files, databases, CI pipelines, and AI agents.
Apache-2.0Sep 18, 2026 · metrics 2.10.0
npm
Query-farm/vgi-typescript
TypeScript SDK for the VGI (Vector Gateway Interface) protocol — host DuckDB scalar, table, aggregate, and table-in-out functions in an external worker over Arrow IPC.
Custom licenseSep 16, 2026 · metrics 2.10.0
Go · npm
Tnsor-Labs/brokoli
Brokoli — self-hosted data pipeline orchestration
Apache-2.0Sep 13, 2026 · metrics 2.10.0
PyPI · npm
datacharter/datacharter
Your data, explored locally — and your AI agents kept on a leash. A federated data explorer with governed agentic access.
Apache-2.0Sep 15, 2026 · metrics 2.10.0
PyPI · crates.io
BirchKwok/apexbase
A High-performance HTAP embedded database. A reliable work partner.
Apache-2.0Sep 9, 2026 · metrics 2.10.0
Go
realdatadriven/etlx
ETL / ELT / Reverse ETL Framework powered by DuckDB, designed to seamlessly integrate and process data from diverse sources. It leverages Markdown as a configuration medium, where YAML blocks define metadata for each data source, and embedded SQL blocks specify the extraction, transformation, and loading logic.
MITOct 21, 2026 · metrics 2.10.0
PyPI · crates.io
apitap/apitap-lib
Move whole tables between databases fast — Postgres, MySQL, ClickHouse, BigQuery. Rust engine, one-line Python API, bounded memory.
MITOct 19, 2026 · metrics 2.10.0
PyPI
sidequery/dlt-iceberg
An Iceberg destination for DLT that supports REST catalogs
MITOct 12, 2026 · metrics 2.10.0
PyPI
1cadumagalhaes/airsql
A decorator-based SQL execution framework for Airflow that provides clean, Python-like syntax for data operations.
MITSep 19, 2026 · metrics 2.10.0
Go
Query-farm/vgi-go
Go SDK for the VGI (Vector Gateway Interface) protocol — host DuckDB scalar, table, aggregate, and table-in-out functions in an external worker over Arrow IPC.
Custom licenseSep 8, 2026 · metrics 2.10.0
PyPI
adidas/lakehouse-engine
The Lakehouse Engine is a configuration driven Spark framework, written in Python, serving as a scalable and distributed engine for several lakehouse algorithms, data flows and utilities for Data Products.
Apache-2.0Oct 19, 2026 · metrics 2.10.0
PyPI
LakeLogic/open-lakehouse-contract
Open Lakehouse Contract (OLC) is an open YAML specification for declaring a data product's schema, quality rules, transformations, lineage, ownership, service levels (SLAs and SLOs) and materialisation.
Apache-2.0Oct 1, 2026 · metrics 2.10.0
PyPI
larsyencken/alcove
A personal ETL and data lake.
MITOct 20, 2026 · metrics 2.10.0
PyPI
Ebiquity/crystalia-collector
No repository description published.
Custom licenseSep 9, 2026 · metrics 2.10.0
PyPI
ashishsinha1602/schemagate
Identity-scoped schema selection for text-to-SQL: give the model only the tables this caller may read, and 75% fewer prompt tokens.
Custom licenseSep 18, 2026 · metrics 2.10.0
PyPI
eric-thomas-dagster/dagster-community-components-cli
No repository description published.
MITOct 20, 2026 · metrics 2.10.0
PyPI · crates.io · npm
sercanatalik/cereyan
Orchestration for Python data pipelines, without the platform. A fast Rust core drives flows, schedules, backfills, retries, and a live UI. Agentic-ready: MCP tools, agent-attributed runs, human-in-the-loop approvals. One wheel, no runtime dependencies.
Custom licenseOct 2, 2026 · metrics 2.10.0
Go · npm
arinbalyan/scrappy
Bulk job-board scraper with 100+ sites, email enrichment, deterministic quality scoring, and multi-format exports (CSV/JSONL/XLSX/Parquet). Designed for scheduled bulk-first operations with per-site rate limiting, proxy pools, and resume support.
Custom licenseSep 19, 2026 · metrics 2.10.0
PyPI · crates.io
mmirzafahmi/quickhouse
Fast PostgreSQL / MySQL / BigQuery → ClickHouse or BigQuery ETL: a native Rust engine driven from a small, typed Python API.
MITSep 14, 2026 · metrics 2.10.0