Orchestration for Python data pipelines, without the platform. A fast Rust core drives flows, schedules, backfills, retries, and a live UI. Agentic-ready: MCP tools, agent-attributed runs, human-in-the-loop approvals. One wheel, no runtime dependencies.
High-performance, hardware-aware Layer 0 deduplication engine. Uses 4-tier short-circuit logic (Size → Fringe → Full Hash) with xxHash128 to eliminate exact duplicates before they enter your processing pipeline. Python 3.11+, uv, SQLite.
Lightweight self-hosted workflow scheduler in a single Go binary — an open-source Airflow/Azkaban alternative. Embedded SQLite, DAGs (cron + dependencies + backfill), polyglot tasks (shell/Python/SQL/JAR/HTTP), web console, REST API, and a built-in MCP server for AI agents. Linux & macOS.
An open-source data logging library for machine learning models and data pipelines. 📚 Provides visibility into data quality & model performance over time. 🛡️ Supports privacy-preserving data collection, ensuring safety & robustness. 📈