PyPI
asher/mlx-kquant
Native K-quant support for MLX, with a quantization and fine-tuning toolchain for Apple Silicon
MIT23. Aug. 2026 · Metriken 2.10.0
Alle Tags
Katalog-Tag
#llm-inference
Alle Repositories im öffentlichen Register, die dieses Tag tragen — aus ihren GitHub-Topics oder den von ihren Paket-Registries veröffentlichten Schlagwörtern. Die Gesundheit wird nach derselben versionierten Methodik gemessen wie im übrigen Register.
51 Einträge
PyPI
pjordanandrsn/experts4bit-qlora
Train and serve MoE models that do not fit in VRAM: fused 4-bit experts, QLoRA, CPU/NVMe offload, and fast inference on consumer NVIDIA GPUs.
MIT15. Sept. 2026 · Metriken 2.10.0
Go · PyPI
anthony-chaudhary/fak
Agentic Runtime
Apache-2.010. Sept. 2026 · Metriken 2.10.0
PyPI
pjordanandrsn/grouped-nf4-gemm
Triton kernels for 4-bit MoE inference: grouped NF4/MXFP4 GEMM, INT4 GEMV, FP8 paged attention, and CPU/NVMe expert streaming.
MIT24. Sept. 2026 · Metriken 2.10.0
PyPI
asher/gmlx
Local inference platform for K/IQ-quant GGUF models on Apple Silicon
Eigene Lizenz25. Sept. 2026 · Metriken 2.10.0
Go
townsendmerino/goinfer
Pure-Go, no-cgo local LLM inference — run Gemma, Qwen, Llama and friends from safetensors or GGUF in a single static binary.
MIT19. Okt. 2026 · Metriken 2.10.0
Go
flexigpt/inference-go
A single interface in Go to get inference from multiple llm/ai providers using their official SDKs
MIT16. Sept. 2026 · Metriken 2.10.0
PyPI
jagmarques/nexusquant
Training-free KV cache compression via E8 lattice VQ. 2-bit KV that preserves retrieval (30/30 NIAH vs TurboQuant 0/30). Calibration-free, 9 architectures validated.
Eigene Lizenz16. Juli 2026 · Metriken 2.10.0
PyPI
RobTand/gridbook
Out-of-tree vLLM plugin and open format spec for NVFP4-CB / FP8-CB product-codebook weights — 2-6 bit-per-weight LLM quantization served on native Blackwell tensor cores.
Apache-2.018. Sept. 2026 · Metriken 2.10.0
—
lean-dojo/LeanCopilot
LLMs as Copilots for Theorem Proving in Lean
MIT16. Sept. 2026 · Metriken 2.10.0
PyPI · crates.io
theoddden/Terradev
An imperative command-line-interface for AI workload orchestration
Apache-2.023. Aug. 2026 · Metriken 2.10.0
—
animicaorg/all
Animica — post-quantum (ML-DSA-65) proof-of-work Layer-1 blockchain by Alexander Lehman: Python smart contracts, ANM-native L2 rollup, mining pool, free OpenAI-compatible AI API, x402 pay-per-call agent APIs, MCP server. Monorepo, Apache-2.0.
Apache-2.017. Sept. 2026 · Metriken 2.10.0
Go
flexigpt/llmtools-go
LLM Tool implementations for Golang
MIT19. Okt. 2026 · Metriken 2.10.0
Go · npm
nchapman/lleme
A CLI that makes it easy way to find, run, and manage local LLMs.
MIT20. Sept. 2026 · Metriken 2.10.0
—
eastriverlee/LLM.swift
LLM.swift is a simple and readable library that allows you to interact with large language models locally with ease for macOS, iOS, watchOS, tvOS, and visionOS.
MIT14. Sept. 2026 · Metriken 2.10.0
PyPI
AI-Hypercomputer/JetStream
JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).
Apache-2.010. Sept. 2026 · Metriken 2.10.0
Go
mostlydev/cllama
The blood-brain barrier for autonomous agents. A context-aware LLM governance proxy that enforces credential starvation — identity-verified, provider-routed, cost-tracked, and audit-logged.
Keine Lizenz19. Sept. 2026 · Metriken 2.10.0
PyPI
thisisthepy/torchnative
Run the real PyTorch ecosystem on device, not reimplementing it
Apache-2.018. Sept. 2026 · Metriken 2.10.0
—
MekayelAnik/vllm-cpu
Wheels & Docker images for running vLLM on CPU-only systems, optimized for different CPU instruction sets
GPL-3.017. Sept. 2026 · Metriken 2.10.0
PyPI
openvinotoolkit/openvino
OpenVINO™ is an open source toolkit for optimizing and deploying AI inference
Apache-2.017. Sept. 2026 · Metriken 2.10.0
npm
empirical-run/empirical
Test and evaluate LLMs and model configurations, across all the scenarios that matter for your application
MIT8. Sept. 2026 · Metriken 2.10.0