PyPI · crates.io
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
Apache-2.02026年9月16日 · 指标 2.10.0
全部标签
目录标签
#model-serving
公开记录中带有此标签的全部仓库——标签来自其 GitHub 主题或软件包注册表发布的关键词。健康度量遵循与记录其余部分相同的版本化方法论。
13 条记录
PyPI · crates.io
basetenlabs/truss
The simplest way to serve AI/ML models in production
MIT2026年8月27日 · 指标 2.10.0
Go · PyPI
kserve/kserve
Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes
Apache-2.02026年9月18日 · 指标 2.10.0
PyPI
vllm-project/vllm-ascend
Community maintained hardware plugin for vLLM on Huawei Ascend
Apache-2.02026年10月9日 · 指标 2.10.0
PyPI
bentoml/BentoML
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
Apache-2.02026年9月17日 · 指标 2.10.0
PyPI
mlrun/mlrun
MLRun is an open source MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your development and CI/CD environment and automates the delivery of production data, ML pipelines, and online applications.
Apache-2.02026年9月15日 · 指标 2.10.0
PyPI
sgl-project/sglang-omni
SGLang-Omni is a high-performance serving framework for audio models (TTS, ASR) and unified multimodal models.
Apache-2.02026年9月28日 · 指标 2.10.0
Go · npm
ome-projects/ome
Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton
Apache-2.02026年9月12日 · 指标 2.10.0
PyPI
Tejas-TA/predikit
The missing bridge between your ML models and your AI agents.
MIT2026年9月15日 · 指标 2.10.0
PyPI
pjordanandrsn/grouped-nf4-gemm
Triton kernels for 4-bit MoE inference: grouped NF4/MXFP4 GEMM, INT4 GEMV, FP8 paged attention, and CPU/NVMe expert streaming.
MIT2026年9月24日 · 指标 2.10.0
PyPI · npm · Maven
hpnkv/a11
A streaming action runtime for AI agents, model serving, and multimodal APIs
Apache-2.02026年10月21日 · 指标 2.10.0
PyPI
AI-Hypercomputer/JetStream
JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).
Apache-2.02026年9月10日 · 指标 2.10.0
PyPI
Lightning-Universe/stable-diffusion-deploy
Learn to serve Stable Diffusion models on cloud infrastructure at scale. This Lightning App shows load-balancing, orchestrating, pre-provisioning, dynamic batching, GPU-inference, micro-services working together via the Lightning Apps framework.
Apache-2.02026年9月12日 · 指标 2.10.0