Go · crates.io · npm
teranos/QNTX
QNTX = Experiential ꩜ Learning ⌬ System ≡ Attestation +
MITAug 1, 2026 · metrics 2.10.0
All tags
Catalogue tag
#inference
Every repository in the public record carrying this tag — from its GitHub topics or the keywords its package registries publish. Health is measured under the same versioned methodology as the rest of the record.
106 records
PyPI
varjoranta/turboquant-vllm
TurboQuant+ KV cache compression for vLLM. 3.8x smaller KV cache, same conversation quality. Fused CUDA kernels with automatic PyTorch fallback.
MITJul 22, 2026 · metrics 2.10.0
Packagist
RubixML/ML
A high-level machine learning and deep learning library for the PHP language.
MITJul 22, 2026 · metrics 2.10.0
PyPI · npm
jjang-ai/vmlx
vMLX - JANGTQ Uber Compressed MLX Models - L2 Disk Cache (survives restart) + L1 Paged (super fast ttft) + Hybrid SSM Scheduler + Cont Batching + etc!
Apache-2.0Jul 24, 2026 · metrics 2.10.0
Go
Ericson246/npu-optimize
Hardware-aware CLI that detects NPUs/GPUs, finds compatible GGUF models, and recommends optimal llama.cpp inference configs
MITJul 17, 2026 · metrics 2.10.0
PyPI
NevermindNilas/Nelux
NeLux: High-performance video processing for Python, powered by FFmpeg and PyTorch. Ultra-fast video decoding directly to tensors.
AGPL-3.0Jul 19, 2026 · metrics 2.10.0
Go · npm
OpenCSGs/csglite
CSGLite is a lightweight local LLM inference platform. One command downloads, loads, and chats with models. It ships a web UI, OpenAI-compatible API, llama.cpp inference, resumable downloads, marketplace browsing, and one-click AI app and coding-agent setup—all in a single cross-platform binary.
Apache-2.0Aug 22, 2026 · metrics 2.10.0
PyPI
RDFLib/OWL-RL
A simple implementation of the OWL2 RL Profile on top of RDFLib: it expands the graph with all possible triples that OWL RL defines. It can be used together with RDFLib to expand an RDFLib Graph object, or as a stand alone service with its own serialization.
Custom licenseAug 8, 2026 · metrics 2.10.0
npm
gvergnaud/ts-pattern
🎨 The exhaustive Pattern Matching library for TypeScript, with smart type inference.
MITAug 12, 2026 · metrics 2.10.0
npm
wundercorp/openmodel
Use any AI model, locally or in the cloud, while tracking on-device token usage and collecting token telemetry data.
Apache-2.0Aug 7, 2026 · metrics 2.10.0
npm
amanharshx/ultralytics-mcp
MCP for Ultralytics Platform workflows, datasets, training, prediction, and model operations.
MITJul 23, 2026 · metrics 2.10.0
Go · PyPI
anthony-chaudhary/fak
fak — the Fused Agent Kernel: one Go binary for AI agent loops. Wrap Claude Code/Codex/Cursor, keep long sessions cache-efficient, route work per call, run local GGUF models, and adjudicate tool calls.
Apache-2.0Jul 17, 2026 · metrics 2.10.0
PyPI
skylight-org/sparse-attention-hub
Advancing the frontier of efficient AI
Apache-2.0Jul 17, 2026 · metrics 2.10.0
PyPI · crates.io
FedericoTs/quantprobe
Run a 110B on a 2016 PC with 16 GB RAM. Know your tok/s before you download. Placement beats budget: predicts speed + memory fit for any GGUF on your exact hardware, self-calibrates, emits the exact llama.cpp command — or 'quantprobe auto' does it all. Falsification-tested laws; misses published at full size. pip install quantprobe
MITAug 20, 2026 · metrics 2.10.0
PyPI
RobTand/gridbook
Out-of-tree vLLM plugin and open format spec for NVFP4-CB / FP8-CB product-codebook weights — 2-6 bit-per-weight LLM quantization served on native Blackwell tensor cores.
Apache-2.0Aug 15, 2026 · metrics 2.10.0
PyPI
SYSTRAN/faster-whisper
Faster Whisper transcription with CTranslate2
MITAug 5, 2026 · metrics 2.10.0
PyPI
druide67/asiai
Multi-engine LLM benchmark & monitoring CLI for Apple Silicon
Apache-2.0Jul 18, 2026 · metrics 2.10.0
PyPI
jagmarques/nexusquant
Training-free KV cache compression via E8 lattice VQ. 2-bit KV that preserves retrieval (30/30 NIAH vs TurboQuant 0/30). Calibration-free, 9 architectures validated.
Custom licenseJul 16, 2026 · metrics 2.10.0
npm
mcmcjs/mcmcjs
Command-line tools for Bayesian modelling, MCMC inference, and post-inference diagnostics across probabilistic programming languages.
MITSep 5, 2026 · metrics 2.10.0
PyPI · npm
seantalts/stanli
stanli, the Stan Language Interpreter: op-graph executor over precompiled stan-math kernels. Compile and sample Stan models with no C++ toolchain.
BSD-3-ClauseAug 30, 2026 · metrics 2.10.0
npm
inference-sh/sdk-js
No repository description published.
MITJul 17, 2026 · metrics 2.10.0
crates.io · PyPI
oramasearch/oramacore
OramaCore is the complete runtime you need for your projects, answer engines, copilots, and search. It includes a fully-fledged full-text search engine, vector database, LLM interface, and many more utilities.
AGPL-3.0Aug 28, 2026 · metrics 2.10.0
PyPI · npm
mauriciobenjamin700/ort-vision-sdk
No repository description published.
MITAug 3, 2026 · metrics 2.10.0
npm
tangle-network/tcloud
TypeScript SDK, CLI, agent, and relayer for Tangle AI Cloud — decentralized LLM inference
No licenseSep 5, 2026 · metrics 2.10.0
PyPI
ai-hypercomputer/jetstream
JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).
Apache-2.0Jul 18, 2026 · metrics 2.10.0
PyPI
pictograph-io/pictograph-sdk
Complete python SDK, CLI, and API reference for integrating with the Pictograph.io platform.
MITAug 29, 2026 · metrics 2.10.0
PyPI
Hai-Wenxiang/fusedtok
Fused CUDA kernels for LLM inference - RMSNorm / RoPE / SwiGLU
MITSep 1, 2026 · metrics 2.10.0
crates.io · PyPI
dshakes/firstpass
Route every LLM request to the cheapest model that provably passes your quality gate — with a signed, tamper-evident receipt for every decision. Proof over prediction.
Apache-2.0Jul 26, 2026 · metrics 2.10.0
PyPI
google/jetstream
JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).
Apache-2.0Jul 18, 2026 · metrics 2.10.0
npm
mohitsoni48/TurboLLM
Run any local LLM engine, auto-tuned to your GPU — polished web UI + OpenAI/Anthropic-compatible API. Point Claude Code at your own machine in one command. No Electron, no Python, offline-first.
No licenseJul 16, 2026 · metrics 2.10.0