PyPI
asher/mlx-kquant
Native K-quant support for MLX, with a quantization and fine-tuning toolchain for Apple Silicon
MITAug 23, 2026 · metrics 2.10.0
All tags
Catalogue tag
#llm-inference
Every repository in the public record carrying this tag — from its GitHub topics or the keywords its package registries publish. Health is measured under the same versioned methodology as the rest of the record.
51 records
PyPI
pjordanandrsn/experts4bit-qlora
Train and serve MoE models that do not fit in VRAM: fused 4-bit experts, QLoRA, CPU/NVMe offload, and fast inference on consumer NVIDIA GPUs.
MITSep 15, 2026 · metrics 2.10.0
Go · PyPI
anthony-chaudhary/fak
Agentic Runtime
Apache-2.0Sep 10, 2026 · metrics 2.10.0
PyPI
pjordanandrsn/grouped-nf4-gemm
Triton kernels for 4-bit MoE inference: grouped NF4/MXFP4 GEMM, INT4 GEMV, FP8 paged attention, and CPU/NVMe expert streaming.
MITSep 24, 2026 · metrics 2.10.0
PyPI
asher/gmlx
Local inference platform for K/IQ-quant GGUF models on Apple Silicon
Custom licenseSep 25, 2026 · metrics 2.10.0
Go
townsendmerino/goinfer
Pure-Go, no-cgo local LLM inference — run Gemma, Qwen, Llama and friends from safetensors or GGUF in a single static binary.
MITOct 19, 2026 · metrics 2.10.0
Go
flexigpt/inference-go
A single interface in Go to get inference from multiple llm/ai providers using their official SDKs
MITSep 16, 2026 · metrics 2.10.0
PyPI
jagmarques/nexusquant
Training-free KV cache compression via E8 lattice VQ. 2-bit KV that preserves retrieval (30/30 NIAH vs TurboQuant 0/30). Calibration-free, 9 architectures validated.
Custom licenseJul 16, 2026 · metrics 2.10.0
PyPI
RobTand/gridbook
Out-of-tree vLLM plugin and open format spec for NVFP4-CB / FP8-CB product-codebook weights — 2-6 bit-per-weight LLM quantization served on native Blackwell tensor cores.
Apache-2.0Sep 18, 2026 · metrics 2.10.0
—
lean-dojo/LeanCopilot
LLMs as Copilots for Theorem Proving in Lean
MITSep 16, 2026 · metrics 2.10.0
PyPI · crates.io
theoddden/Terradev
An imperative command-line-interface for AI workload orchestration
Apache-2.0Aug 23, 2026 · metrics 2.10.0
—
animicaorg/all
Animica — post-quantum (ML-DSA-65) proof-of-work Layer-1 blockchain by Alexander Lehman: Python smart contracts, ANM-native L2 rollup, mining pool, free OpenAI-compatible AI API, x402 pay-per-call agent APIs, MCP server. Monorepo, Apache-2.0.
Apache-2.0Sep 17, 2026 · metrics 2.10.0
Go
flexigpt/llmtools-go
LLM Tool implementations for Golang
MITOct 19, 2026 · metrics 2.10.0
Go · npm
nchapman/lleme
A CLI that makes it easy way to find, run, and manage local LLMs.
MITSep 20, 2026 · metrics 2.10.0
—
eastriverlee/LLM.swift
LLM.swift is a simple and readable library that allows you to interact with large language models locally with ease for macOS, iOS, watchOS, tvOS, and visionOS.
MITSep 14, 2026 · metrics 2.10.0
PyPI
AI-Hypercomputer/JetStream
JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).
Apache-2.0Sep 10, 2026 · metrics 2.10.0
Go
mostlydev/cllama
The blood-brain barrier for autonomous agents. A context-aware LLM governance proxy that enforces credential starvation — identity-verified, provider-routed, cost-tracked, and audit-logged.
No licenseSep 19, 2026 · metrics 2.10.0
PyPI
thisisthepy/torchnative
Run the real PyTorch ecosystem on device, not reimplementing it
Apache-2.0Sep 18, 2026 · metrics 2.10.0
—
MekayelAnik/vllm-cpu
Wheels & Docker images for running vLLM on CPU-only systems, optimized for different CPU instruction sets
GPL-3.0Sep 17, 2026 · metrics 2.10.0
PyPI
openvinotoolkit/openvino
OpenVINO™ is an open source toolkit for optimizing and deploying AI inference
Apache-2.0Sep 17, 2026 · metrics 2.10.0
npm
empirical-run/empirical
Test and evaluate LLMs and model configurations, across all the scenarios that matter for your application
MITSep 8, 2026 · metrics 2.10.0