PyPI · crates.io
ai-dynamo/dynamo
A Datacenter Scale Distributed Inference Serving Framework
Custom licenseAug 28, 2026 · metrics 2.10.0
All tags
Catalogue tag
#vllm
Every repository in the public record carrying this tag — from its GitHub topics or the keywords its package registries publish. Health is measured under the same versioned methodology as the rest of the record.
51 records
PyPI · Go
LMCache/LMCache
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
Apache-2.0Sep 18, 2026 · metrics 2.10.0
Go · PyPI
kserve/kserve
Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes
Apache-2.0Sep 18, 2026 · metrics 2.10.0
PyPI
kvcache-ai/Mooncake
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
Apache-2.0Sep 17, 2026 · metrics 2.10.0
PyPI
vllm-project/vllm-ascend
Community maintained hardware plugin for vLLM on Huawei Ascend
Apache-2.0Oct 9, 2026 · metrics 2.10.0
PyPI · npm
xorbitsai/inference
Swap GPT for any LLM by changing a single line of code. Xinference lets you run open-source, speech, and multimodal models on cloud, on-prem, or your laptop — all through one unified, production-ready inference API.
Apache-2.0Sep 20, 2026 · metrics 2.10.0
PyPI
gpustack/gpustack
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
Apache-2.0Sep 15, 2026 · metrics 2.10.0
PyPI
intel/auto-round
A SOTA quantization toolkit for high-accuracy low-bit LLM inference|简洁且高效的量化工具包
Apache-2.0Sep 8, 2026 · metrics 2.10.0
PyPI · npm
modelscope/FunASR
Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
MITSep 16, 2026 · metrics 2.10.0
Go
praetorian-inc/julius
Simple LLM service identification - translate IP:Port to Ollama, vLLM, LiteLLM, or 60+ other AI services in seconds
Apache-2.0Sep 17, 2026 · metrics 2.10.0
PyPI · crates.io
smg-project/smg
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.
Apache-2.0Sep 9, 2026 · metrics 2.10.0
PyPI
vllm-project/speculators
A unified library for building, evaluating, and storing speculative decoding algorithms for LLM inference in vLLM
Apache-2.0Oct 8, 2026 · metrics 2.10.0
PyPI
ModelCloud/GPTQModel
LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.
Custom licenseSep 10, 2026 · metrics 2.10.0
Go · npm
matrixhub-ai/matrixhub
An Open-source, self-hosted AI model hub with Hugging Face compatibility, accelerating vLLM/SGLang performance.
Apache-2.0Sep 10, 2026 · metrics 2.10.0
PyPI
waybarrios/vllm-mlx
High-performance OpenAI and Anthropic compatible LLM inference server for Apple Silicon. Native MLX, continuous batching, multimodal models, MCP tool calling, and Claude Code support.
Apache-2.0Oct 20, 2026 · metrics 2.10.0
Go · PyPI
pmady/keda-gpu-scaler
KEDA External gRPC Scaler for GPU workloads - native NVML metrics via DaemonSet, no Prometheus required
Apache-2.0Sep 18, 2026 · metrics 2.10.0
Go
defilantech/LLMKube
Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.
Apache-2.0Sep 5, 2026 · metrics 2.10.0
Go · npm
ome-projects/ome
Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton
Apache-2.0Sep 12, 2026 · metrics 2.10.0
npm
pensarai/apex
AI-powered offensive security testing using autonomous agents, directly in your terminal.
Apache-2.0Sep 6, 2026 · metrics 2.10.0
PyPI
Sahil170595/Chimeraforge
PyPI capacity-planning CLI for LLM deployment. pip install chimeraforge.
MITOct 20, 2026 · metrics 2.10.0
Go · PyPI
kubeai-project/kubeai
AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text.
Apache-2.0Sep 15, 2026 · metrics 2.10.0
Go · npm
raids-lab/crater
Crater is a cloud-native AI training & inference platform.
Apache-2.0Sep 14, 2026 · metrics 2.10.0
Go · npm
lordmathis/llamactl
Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard.
MITSep 20, 2026 · metrics 2.10.0
Go · npm
voidmind-io/voidllm
Privacy-first LLM proxy and AI gateway - load balancing, multi-provider routing, API key management, usage tracking, rate limiting. Self-hosted. Zero knowledge of your prompts.
Custom licenseSep 16, 2026 · metrics 2.10.0
npm
sybil-solutions/local-studio
Control panel for VLLM, Sglang, llama.cpp, exllamav3
Apache-2.0Sep 9, 2026 · metrics 2.10.0
PyPI · crates.io
ahb-sjsu/turboquant-pro
Consumer-aware compression for embedding indexes and LLM KV caches — compress by the metric the downstream consumer actually uses. PCA-Matryoshka + TurboQuant (27x @ 99.8% recall@10), asymmetric K/V, CUDA/Triton kernels, vLLM plugin, replayable CI-gated claims. MIT.
MITSep 5, 2026 · metrics 2.10.0
Go
airiclenz/apogee
Terminal coding agent for local LLMs (llama.cpp, Ollama, vLLM) and any OpenAI-compatible API. OS-sandboxed autonomy, MCP, sessions. Go.
MITOct 20, 2026 · metrics 2.10.0
PyPI
freesolo-co/flash
LoRA post-training for open-weight models: SFT, GRPO, and on-policy distillation. Describe a run in TOML; Flash allocates a GPU, trains, streams checkpoints, and serves the adapter.
Apache-2.0Oct 21, 2026 · metrics 2.10.0
PyPI · npm
jjang-ai/vmlx
vMLX - Use MLX models easily - JANGQ (GGUF for MLX) - Not dependant on mlx_vlm
Apache-2.0Sep 13, 2026 · metrics 2.10.0
crates.io · npm
soapbucket/sbproxy
Open source Enterprise AI Gateway for API, MCP and agent, and AI model traffic. One Apache-2.0 binary: 72 native providers behind an OpenAI-compatible API, or serve vLLM and llama.cpp on your own GPUs. Keys, budgets, guardrails, semantic cache, WAF.
Apache-2.0Sep 14, 2026 · metrics 2.10.0