PyPI · crates.io
ai-dynamo/dynamo
A Datacenter Scale Distributed Inference Serving Framework
自定义许可证2026年8月28日 · 指标 2.10.0
全部标签
目录标签
#vllm
公开记录中带有此标签的全部仓库——标签来自其 GitHub 主题或软件包注册表发布的关键词。健康度量遵循与记录其余部分相同的版本化方法论。
38 条记录
PyPI · Go
LMCache/LMCache
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
Apache-2.02026年8月15日 · 指标 2.10.0
PyPI
kvcache-ai/Mooncake
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
Apache-2.02026年8月12日 · 指标 2.10.0
PyPI
intel/auto-round
A SOTA quantization algorithm for high-accuracy low-bit LLM inference, seamlessly optimized for CPU/XPU/CUDA, with multi-datatype support and full compatibility with vLLM, SGLang, and Transformers.
Apache-2.02026年7月16日 · 指标 2.10.0
Go · PyPI
kserve/kserve
Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes
Apache-2.02026年8月28日 · 指标 2.10.0
PyPI
gpustack/gpustack
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
Apache-2.02026年8月2日 · 指标 2.10.0
PyPI · npm
modelscope/FunASR
Open-source speech recognition toolkit for training, inference, streaming ASR, VAD, punctuation, speaker diarization pipelines, and OpenAI-compatible/MCP serving.
MIT2026年8月5日 · 指标 2.10.0
crates.io · PyPI
lightseekorg/smg
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.
Apache-2.02026年7月16日 · 指标 2.10.0
Go
praetorian-inc/julius
Simple LLM service identification - translate IP:Port to Ollama, vLLM, LiteLLM, or 60+ other AI services in seconds
Apache-2.02026年8月8日 · 指标 2.10.0
crates.io · PyPI
smg-project/smg
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.
Apache-2.02026年8月2日 · 指标 2.10.0
PyPI
ModelCloud/GPTQModel
LLM model quantization (compression) toolkit with HW acceleration support for Nvidia, AMD, Intel GPU and Intel/AMD/Apple CPU via HF, vLLM, and SGLang.
自定义许可证2026年7月17日 · 指标 2.10.0
Go
defilantech/LLMKube
Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.
Apache-2.02026年9月5日 · 指标 2.10.0
Go · npm
matrixhub-ai/matrixhub
An Open-source, self-hosted AI model hub with Hugging Face compatibility, accelerating vLLM/SGLang performance.
Apache-2.02026年7月17日 · 指标 2.10.0
Go · npm
ome-projects/ome
Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton
Apache-2.02026年7月21日 · 指标 2.10.0
npm
pensarai/apex
AI-powered offensive security testing using autonomous agents, directly in your terminal.
Apache-2.02026年9月6日 · 指标 2.10.0
Go · PyPI
kubeai-project/kubeai
AI Inference Operator for Kubernetes. The easiest way to serve ML models in production. Supports VLMs, LLMs, embeddings, and speech-to-text.
Apache-2.02026年7月31日 · 指标 2.10.0
Go · npm
raids-lab/crater
Crater is a cloud-native AI training & inference platform.
Apache-2.02026年7月28日 · 指标 2.10.0
Go · npm
voidmind-io/voidllm
Privacy-first LLM proxy and AI gateway - load balancing, multi-provider routing, API key management, usage tracking, rate limiting. Self-hosted. Zero knowledge of your prompts.
自定义许可证2026年8月2日 · 指标 2.10.0
PyPI
Sahil170595/Chimeraforge
PyPI capacity-planning CLI for LLM deployment. pip install chimeraforge.
MIT2026年8月22日 · 指标 2.10.0
npm
0xsero/vllm-studio
Control panel for VLLM, Sglang, llama.cpp, exllamav3
Apache-2.02026年7月16日 · 指标 2.10.0
PyPI · crates.io
ahb-sjsu/turboquant-pro
Consumer-aware compression for embedding indexes and LLM KV caches — compress by the metric the downstream consumer actually uses. PCA-Matryoshka + TurboQuant (27x @ 99.8% recall@10), asymmetric K/V, CUDA/Triton kernels, vLLM plugin, replayable CI-gated claims. MIT.
MIT2026年9月5日 · 指标 2.10.0
crates.io · npm
soapbucket/sbproxy
Self-hosted AI gateway and LLM proxy. OpenAI-compatible API for OpenAI, Anthropic, Gemini, Bedrock and 60+ providers, or serve vLLM/llama.cpp on your GPUs. Keys, budgets, guardrails, semantic cache, MCP
Apache-2.02026年7月30日 · 指标 2.10.0
PyPI
freesolo-co/flash
LoRA post-training for open-weight models: SFT, GRPO, and on-policy distillation. Describe a run in TOML; Flash allocates a GPU, trains, streams checkpoints, and serves the adapter.
Apache-2.02026年8月29日 · 指标 2.10.0
PyPI
CocoRoF/geny-executor
Manifest-driven 21-stage agent pipeline for Python — 5 LLM backends (Anthropic / OpenAI / Google / vLLM / Claude Code CLI), tools, skills, memory, sandboxed CLI runs, MCP. The engine behind Geny. Apache-2.0.
Apache-2.02026年9月5日 · 指标 2.10.0
PyPI
varjoranta/turboquant-vllm
TurboQuant+ KV cache compression for vLLM. 3.8x smaller KV cache, same conversation quality. Fused CUDA kernels with automatic PyTorch fallback.
MIT2026年7月22日 · 指标 2.10.0
npm
yeemio/owlcoda
OwlCoda — independent local-first AI coding workbench. Native REPL, 42+ tools, learned skills, GPL-3.0-or-later.
GPL-3.02026年7月25日 · 指标 2.10.0
PyPI
Aitherium/awdk
Build AI agent fleets. 3 lines, any backend, local or cloud. Effort-based model routing, 48 identities, knowledge graph memory, fleet orchestration.
自定义许可证2026年9月5日 · 指标 2.10.0
PyPI · npm
jjang-ai/vmlx
vMLX - JANGTQ Uber Compressed MLX Models - L2 Disk Cache (survives restart) + L1 Paged (super fast ttft) + Hybrid SSM Scheduler + Cont Batching + etc!
Apache-2.02026年7月24日 · 指标 2.10.0
PyPI
hackspaces/blueshark-forge
Model-agnostic agentic runtime for the terminal — any local model becomes a capable agent. The intelligence lives in the harness, not the weights.
MIT2026年9月5日 · 指标 2.10.0
PyPI
RobTand/gridbook
Out-of-tree vLLM plugin and open format spec for NVFP4-CB / FP8-CB product-codebook weights — 2-6 bit-per-weight LLM quantization served on native Blackwell tensor cores.
Apache-2.02026年8月15日 · 指标 2.10.0