#compression

A blazingly fast multi-language serialization framework for idiomatic domain objects, schema IDL, and cross-language data exchange.

Neural Network Compression Framework for enhanced OpenVINO™ inference

Transformers-compatible library for applying various compression algorithms to LLMs for optimized deployment with vLLM

Deduplicating archiver with compression and authenticated encryption.

Accessible large language models via k-bit quantization for PyTorch.

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Fastify compression utils

An extensible, state-of-the-art framework for columnar compression, and the fastest FOSS columnar file format. Formerly at @spiraldb, now an Incubation Stage project at LFAI&Data, part of the Linux Foundation.

Compress tool outputs, logs, files, and RAG chunks before they reach the LLM. 20% fewer tokens for coding agents, 60-95% fewer tokens for JSON, same answers. Library, proxy, MCP server.

Tantivy is a full-text search engine library inspired by Apache Lucene and written in Rust

A high-performance library for compressed ND arrays and columnar tables, with compute and indexing engines

The Cyber Swiss Army Knife - a web app for encryption, encoding, compression and data analysis

MCAP is a modular, performant, and serialization-agnostic container file format, useful for pub/sub and robotics applications.

Persistent Context Across Sessions for Every Agent – Captures everything your agent does during sessions, compresses it with AI, and injects relevant context back into future sessions. Works with Claude Code, OpenClaw, Codex, Gemini, Hermes, Copilot, OpenCode + More

Control what your AI can see. LeanCTX (Lean Context) is the context intelligence layer for AI agents — one local Rust binary that decides what they read, remembers what they learn, guards what they touch, and proves what they save. 60–90% fewer tokens as the receipt. 76 MCP tools, 30+ agents, local-first.

Zip implementation in Rust