PyPI
moj-analytical-services/splink
Fast, accurate and scalable probabilistic data linkage with support for multiple SQL backends
MIT17. Sept. 2026 · Metriken 2.10.0
Alle Tags
Katalog-Tag
#deduplication
Alle Repositories im öffentlichen Register, die dieses Tag tragen — aus ihren GitHub-Topics oder den von ihren Paket-Registries veröffentlichten Schlagwörtern. Die Gesundheit wird nach derselben versionierten Methodik gemessen wie im übrigen Register.
34 Einträge
Go · npm
prometheus/alertmanager
Prometheus Alertmanager
Apache-2.028. Aug. 2026 · Metriken 2.10.0
PyPI · npm
NVIDIA-NeMo/Curator
Scalable data pre processing and curation toolkit for LLMs
Apache-2.025. Sept. 2026 · Metriken 2.10.0
PyPI
borgbackup/borg
Deduplicating archiver with compression and authenticated encryption.
Eigene Lizenz27. Aug. 2026 · Metriken 2.10.0
PyPI · crates.io
huggingface/xet-core
xet client tech, used in huggingface_hub
Apache-2.016. Sept. 2026 · Metriken 2.10.0
Go
PlakarKorp/plakar
plakar is a backup solution powered by Kloset and ptar
ISC18. Sept. 2026 · Metriken 2.10.0
PyPI
opensanctions/nomenklatura
Framework and command-line tools for integrating FollowTheMoney data streams from multiple sources
MIT19. Okt. 2026 · Metriken 2.10.0
npm · PyPI
benseverndev-oss/goldenmatch
Zero-config entity resolution feeding a durable identity layer: messy records from any source become stable golden entities, a Customer 360 with provenance, merge/split and audit. Fellegi-Sunter beats hand-tuned Splink. Arrow-native/Rust, 250M rows in 11.2 min. Python + edge TypeScript (WASM), SQL-native in Postgres & DuckDB, 97 MCP tools + REST.
MIT6. Sept. 2026 · Metriken 2.10.0
crates.io
rustic-rs/rustic_core
rustic_core - library for fast, encrypted, deduplicated backups that powers rustic-rs
Apache-2.019. Okt. 2026 · Metriken 2.10.0
npm
trieb-work/nextjs-turbo-redis-cache
Designed for speed, scalability, and optimized performance, nextjs-turbo-redis-cache is your go-to solution Next.js cache handler for demanding production environments. If you use the app router and self-host, you will probably need this. Development sponsored by https://TRWK.de
MIT15. Sept. 2026 · Metriken 2.10.0
Go
cplieger/docker-fclones-scheduler
Hardened, rootless container for fclones that dedupes your folders on a schedule
Apache-2.021. Okt. 2026 · Metriken 2.10.0
PyPI
sebastienrousseau/bankstatementparser
Parse bank statements — CAMT/ISO 20022, PAIN.001, CSV (EN/DE/FR/ES), OFX/QFX, MT940, PDFs (digital + scanned) — into unified Transactions. Deterministic parsers, LLM/vision fallback, Golden Rule + multi-currency + continuity verification, page provenance, dedupe, review, accuracy evals, CLI + REST API. 100% test coverage.
Eigene Lizenz14. Sept. 2026 · Metriken 2.10.0
crates.io · npm · PyPI
WithAutonomi/self_encryption
Self-encrypting, content-addressed storage - convergent encryption that splits data into obfuscated chunks. The peer-reviewed core of Autonomi.
Apache-2.010. Okt. 2026 · Metriken 2.10.0
Go
franchoy/coldkeep
Coldkeep is a correctness-first, content-addressed cold-storage engine with deterministic restore, integrity verification, safe garbage collection, deduplication, transform-aware packed storage, and a reusable engine/catalog core. v1.x is complete and frozen; v2.x is planned for local-first productization.
Apache-2.019. Okt. 2026 · Metriken 2.10.0
npm
git-stunts/git-cas
Git, freebased: pure CAS that’ll knock your SHAs off. LFS hates this repo!
Apache-2.013. Sept. 2026 · Metriken 2.10.0
PyPI
crunchtools/petit
Log analysis program for use from the command line
Eigene Lizenz3. Okt. 2026 · Metriken 2.10.0
Go
LarsArtmann/go-idempotency
Go idempotency/deduplication store for CQRS command keys
MIT18. Okt. 2026 · Metriken 2.10.0
npm
MikkelScheike/email-provider-links
Email Provider Links automatically detects which email service (Gmail, Outlook, Yahoo, etc.) someone uses and provides a direct link to their login page, making password reset flows and email-related UX seamless.
MIT11. Sept. 2026 · Metriken 2.10.0
crates.io
nlfiedler/fastcdc-rs
FastCDC implementation in Rust
MIT28. Aug. 2026 · Metriken 2.10.0
PyPI · crates.io · Go
The-Billy-Company/relate
Code similarity search by compression - find near-duplicate files, clone families, and where a pasted snippet came from. No embeddings, no model, no vector database.
Apache-2.01. Okt. 2026 · Metriken 2.10.0
—
mhx/dwarfs
A fast high-compression read-only file system for Linux, FreeBSD, macOS and Windows
Eigene Lizenz9. Sept. 2026 · Metriken 2.10.0
npm
SIARHEI-SMANTSAR/redux-requests-factory
Keine Repository-Beschreibung veröffentlicht.
MIT21. Okt. 2026 · Metriken 2.10.0
Go
samekind/codexfold
Local-first deduplication and transparent storage for Codex sessions
Apache-2.019. Okt. 2026 · Metriken 2.10.0
PyPI
beshu-tech/deltaglider
Store 4TB in 5GB: S3-compatible storage with 99.9% compression for versioned files
MIT19. Okt. 2026 · Metriken 2.10.0
crates.io
revidiumhq/biblib
Parse, manage, and deduplicate academic citations
Apache-2.012. Sept. 2026 · Metriken 2.10.0
crates.io
nootr/tracing-throttle
High-performance log deduplication and rate limiting for the Rust tracing ecosystem.
MIT15. Sept. 2026 · Metriken 2.10.0
RubyGems
rosette-api/ruby
Babel Street Analytics Client Library for Ruby
Eigene Lizenz6. Okt. 2026 · Metriken 2.10.0
Go · npm
alexander-kolodka/crestic
Crestic simplifies working with restic. Restic is fast and secure, but managing multiple folders and backup destinations via its CLI can get repetitive. Crestic lets you define repositories and jobs in a single YAML config and run them with simple commands — no magic, just a cleaner way to use restic.
MIT11. Sept. 2026 · Metriken 2.10.0
crates.io
jRimbault/yadf
Yet Another Dupes Finder
MIT19. Sept. 2026 · Metriken 2.10.0
PyPI
Fallen-Breath/pyfastcdc
A high-performance FastCDC 2020 implementation written in Python + Cython
MIT5. Sept. 2026 · Metriken 2.10.0