All tags
Catalogue tag

#deduplication

Every repository in the public record carrying this tag — from its GitHub topics or the keywords its package registries publish. Health is measured under the same versioned methodology as the rest of the record.

24 records
Tagged “deduplication”Ranked by health index
Go · npm
98Exceptionalhealth index
prometheus/alertmanager
Prometheus Alertmanager
Go · Elm★ 8,597Aug 28, 2026
Apache-2.0Aug 28, 2026 · metrics 2.10.0
PyPI
97Exceptionalhealth index
borgbackup/borg
Deduplicating archiver with compression and authenticated encryption.
Python★ 13.7K↓ 70.8K/moAug 27, 2026
Custom licenseAug 27, 2026 · metrics 2.10.0
PyPI
96Exceptionalhealth index
moj-analytical-services/splink
Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends
Python · JavaScript★ 2,336Aug 13, 2026
MITAug 13, 2026 · metrics 2.10.0
Go
94Exceptionalhealth index
PlakarKorp/plakar
plakar is a backup solution powered by Kloset and ptar
Go★ 2,030Aug 22, 2026
ISCAug 22, 2026 · metrics 2.10.0
PyPI · crates.io
94Exceptionalhealth index
huggingface/xet-core
xet client tech, used in huggingface_hub
Rust★ 548↓ 257M/moAug 4, 2026
Apache-2.0Aug 4, 2026 · metrics 2.10.0
PyPI
93Exceptionalhealth index
opensanctions/nomenklatura
Framework and command-line tools for integrating FollowTheMoney data streams from multiple sources
Python★ 260↓ 34.7K/moAug 19, 2026
MITAug 19, 2026 · metrics 2.10.0
crates.io
89Excellenthealth index
rustic-rs/rustic_core
rustic_core - library for fast, encrypted, deduplicated backups that powers rustic-rs
Rust★ 85↓ 16.4K/moAug 18, 2026
Apache-2.0Aug 18, 2026 · metrics 2.10.0
npm · PyPI
88Excellenthealth index
benseverndev-oss/goldenmatch
Zero-config entity resolution feeding a durable identity layer: messy records from any source become stable golden entities, a Customer 360 with provenance, merge/split and audit. Fellegi-Sunter beats hand-tuned Splink. Arrow-native/Rust, 250M rows in 11.2 min. Python + edge TypeScript (WASM), SQL-native in Postgres & DuckDB, 97 MCP tools + REST.
Python · TypeScript★ 131Sep 6, 2026
MITSep 6, 2026 · metrics 2.10.0
PyPI
84Excellenthealth index
sebastienrousseau/bankstatementparser
Parse bank statements — CAMT/ISO 20022, PAIN.001, CSV (EN/DE/FR/ES), OFX/QFX, MT940, PDFs (digital + scanned) — into unified Transactions. Deterministic parsers, LLM/vision fallback, Golden Rule + multi-currency + continuity verification, page provenance, dedupe, review, accuracy evals, CLI + REST API. 100% test coverage.
Python★ 38Jul 31, 2026
Apache-2.0Jul 31, 2026 · metrics 2.10.0
Go
81Excellenthealth index
cplieger/docker-fclones-scheduler
Find and deduplicate files on a schedule — reclaim wasted disk space automatically
Go★ 0Sep 5, 2026
Apache-2.0Sep 5, 2026 · metrics 2.10.0
npm
80Excellenthealth index
git-stunts/git-cas
Git, freebased: pure CAS that’ll knock your SHAs off. LFS hates this repo!
JavaScript★ 0↓ 2,858/moJul 25, 2026
Apache-2.0Jul 25, 2026 · metrics 2.10.0
Go
78Goodhealth index
franchoy/coldkeep
Coldkeep is a correctness-first, content-addressed cold-storage engine with deterministic restore, staged integrity verification, safe garbage collection, deduplication, transform-aware packed storage, stable storage semantics, and a reusable engine/catalog core. v1.x is complete; v2.x focuses on local-first productization.
Go★ 0Aug 16, 2026
Apache-2.0Aug 16, 2026 · metrics 2.10.0
npm
73Goodhealth index
mikkelscheike/email-provider-links
Email Provider Links automatically detects which email service (Gmail, Outlook, Yahoo, etc.) someone uses and provides a direct link to their login page, making password reset flows and email-related UX seamless.
TypeScript★ 3↓ 19K/moJul 19, 2026
MITJul 19, 2026 · metrics 2.10.0
71Goodhealth index
mhx/dwarfs
A fast high-compression read-only file system for Linux, FreeBSD, macOS and Windows
C++ · C★ 2,582Jul 17, 2026
Custom licenseJul 17, 2026 · metrics 2.10.0
crates.io
71Goodhealth index
nlfiedler/fastcdc-rs
FastCDC implementation in Rust
Rust★ 202↓ 312K/moAug 28, 2026
MITAug 28, 2026 · metrics 2.10.0
Go · npm
60Moderatehealth index
alexander-kolodka/crestic
Crestic simplifies working with restic. Restic is fast and secure, but managing multiple folders and backup destinations via its CLI can get repetitive. Crestic lets you define repositories and jobs in a single YAML config and run them with simple commands — no magic, just a cleaner way to use restic.
Go★ 2Jul 18, 2026
MITJul 18, 2026 · metrics 2.10.0
PyPI
59Moderatehealth index
beshu-tech/deltaglider
Store 4TB in 5GB: S3-compatible storage with 99.9% compression for versioned files
Python★ 33↓ 99/moAug 17, 2026
MITAug 17, 2026 · metrics 2.10.0
crates.io
59Moderatehealth index
revidiumhq/biblib
Parse, manage, and deduplicate academic citations
Rust★ 8↓ 25.6K/moJul 21, 2026
Apache-2.0Jul 21, 2026 · metrics 2.10.0
Go
57Moderatehealth index
samekind/codexfold
Local-first deduplication and transparent storage for Codex sessions
Go★ 0Aug 19, 2026
Apache-2.0Aug 19, 2026 · metrics 2.10.0
PyPI
53Moderatehealth index
Fallen-Breath/pyfastcdc
A high-performance FastCDC 2020 implementation written in Python + Cython
Python · Cython★ 2↓ 6,109/moSep 5, 2026
MITSep 5, 2026 · metrics 2.10.0
PyPI
44Weakhealth index
mr3od/bgate-unix
High-performance, hardware-aware Layer 0 deduplication engine. Uses 4-tier short-circuit logic (Size → Fringe → Full Hash) with xxHash128 to eliminate exact duplicates before they enter your processing pipeline. Python 3.11+, uv, SQLite.
Python★ 1↓ 32/moAug 14, 2026
MITAug 14, 2026 · metrics 2.10.0
npm
41Weakhealth index
electrovir/fuzzy-vir
No repository description published.
TypeScript★ 0↓ 3,678/moSep 6, 2026
CC0-1.0Sep 6, 2026 · metrics 2.10.0
PyPI
25At Riskhealth index
elemental-lf/benji
Benji Backup: A block based deduplicating backup software for Ceph RBD images, iSCSI targets, image files and block devices
Python★ 150Aug 14, 2026
Custom licenseAug 14, 2026 · metrics 2.10.0