PyPI · crates.io
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
Apache-2.0Sep 16, 2026 · metrics 2.10.0
All tags
Catalogue tag
#model-serving
Every repository in the public record carrying this tag — from its GitHub topics or the keywords its package registries publish. Health is measured under the same versioned methodology as the rest of the record.
13 records
PyPI · crates.io
basetenlabs/truss
The simplest way to serve AI/ML models in production
MITAug 27, 2026 · metrics 2.10.0
Go · PyPI
kserve/kserve
Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes
Apache-2.0Sep 18, 2026 · metrics 2.10.0
PyPI
vllm-project/vllm-ascend
Community maintained hardware plugin for vLLM on Huawei Ascend
Apache-2.0Oct 9, 2026 · metrics 2.10.0
PyPI
bentoml/BentoML
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
Apache-2.0Sep 17, 2026 · metrics 2.10.0
PyPI
mlrun/mlrun
MLRun is an open source MLOps platform for quickly building and managing continuous ML applications across their lifecycle. MLRun integrates into your development and CI/CD environment and automates the delivery of production data, ML pipelines, and online applications.
Apache-2.0Sep 15, 2026 · metrics 2.10.0
PyPI
sgl-project/sglang-omni
SGLang-Omni is a high-performance serving framework for audio models (TTS, ASR) and unified multimodal models.
Apache-2.0Sep 28, 2026 · metrics 2.10.0
Go · npm
ome-projects/ome
Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton
Apache-2.0Sep 12, 2026 · metrics 2.10.0
PyPI
Tejas-TA/predikit
The missing bridge between your ML models and your AI agents.
MITSep 15, 2026 · metrics 2.10.0
PyPI
pjordanandrsn/grouped-nf4-gemm
Triton kernels for 4-bit MoE inference: grouped NF4/MXFP4 GEMM, INT4 GEMV, FP8 paged attention, and CPU/NVMe expert streaming.
MITSep 24, 2026 · metrics 2.10.0
PyPI · npm · Maven
hpnkv/a11
A streaming action runtime for AI agents, model serving, and multimodal APIs
Apache-2.0Oct 21, 2026 · metrics 2.10.0
PyPI
AI-Hypercomputer/JetStream
JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).
Apache-2.0Sep 10, 2026 · metrics 2.10.0
PyPI
Lightning-Universe/stable-diffusion-deploy
Learn to serve Stable Diffusion models on cloud infrastructure at scale. This Lightning App shows load-balancing, orchestrating, pre-provisioning, dynamic batching, GPU-inference, micro-services working together via the Lightning Apps framework.
Apache-2.0Sep 12, 2026 · metrics 2.10.0