PyPI · crates.io
ai-dynamo/dynamo
A Datacenter Scale Distributed Inference Serving Framework
Custom licenseAug 28, 2026 · metrics 2.10.0
All tags
Catalogue tag
#inference
Every repository in the public record carrying this tag — from its GitHub topics or the keywords its package registries publish. Health is measured under the same versioned methodology as the rest of the record.
106 records
PyPI
huggingface/transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Apache-2.0Aug 4, 2026 · metrics 2.10.0
PyPI · crates.io
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
Apache-2.0Aug 4, 2026 · metrics 2.10.0
PyPI · Go
LMCache/LMCache
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
Apache-2.0Aug 15, 2026 · metrics 2.10.0
PyPI · crates.io
apache/tvm-ffi
Open ABI and FFI for Machine Learning Systems
Apache-2.0Aug 27, 2026 · metrics 2.10.0
PyPI
kvcache-ai/Mooncake
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
Apache-2.0Aug 12, 2026 · metrics 2.10.0
PyPI · crates.io
sgl-project/sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
Apache-2.0Aug 4, 2026 · metrics 2.10.0
PyPI
triton-inference-server/server
The Triton Inference Server provides an optimized cloud and edge inferencing solution.
BSD-3-ClauseAug 27, 2026 · metrics 2.10.0
PyPI · RubyGems
deepspeedai/DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Apache-2.0Aug 13, 2026 · metrics 2.10.0
PyPI
pytorch/ao
PyTorch native quantization and sparsity for training and inference
Custom licenseJul 21, 2026 · metrics 2.10.0
PyPI
mozilla-ai/any-llm
Communicate with an LLM provider using a single interface
Apache-2.0Jul 21, 2026 · metrics 2.10.0
npm
vercel/ai
The AI Toolkit for TypeScript. From the creators of Next.js, the AI SDK is a free open-source library for building AI-powered applications and agents
Custom licenseAug 5, 2026 · metrics 2.10.0
PyPI · crates.io
ai-dynamo/aiconfigurator
Offline optimization of your disaggregated Dynamo graph
Apache-2.0Jul 26, 2026 · metrics 2.10.0
crates.io
ai-dynamo/modelexpress
Model Express is a Rust-based component meant to be placed next to existing model inference systems to speed up their startup times and improve overall performance.
Apache-2.0Aug 1, 2026 · metrics 2.10.0
PyPI
gpustack/gpustack
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
Apache-2.0Aug 2, 2026 · metrics 2.10.0
npm
huggingface/huggingface.js
Use Hugging Face with JavaScript
MITAug 27, 2026 · metrics 2.10.0
PyPI
huggingface/optimum
🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools
Apache-2.0Jul 21, 2026 · metrics 2.10.0
crates.io · npm · PyPI
pykeio/ort
Fast ML inference & training for ONNX models in Rust
Apache-2.0Aug 27, 2026 · metrics 2.10.0
npm · RubyGems
vega/vega
A visualization grammar.
BSD-3-ClauseAug 27, 2026 · metrics 2.10.0
PyPI · npm · RubyGems
mlc-ai/xgrammar
Fast, Flexible and Portable Structured Generation
Apache-2.0Aug 27, 2026 · metrics 2.10.0
PyPI
pytorch/TensorRT
PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT
BSD-3-ClauseJul 30, 2026 · metrics 2.10.0
PyPI
raullenchai/Rapid-MLX
The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Drop-in OpenAI replacement. Works with Claude Code, Cursor, Aider.
Apache-2.0Aug 2, 2026 · metrics 2.10.0
Go
run-ai/karta
Translation layer that maps any Kubernetes framework's Custom Resource Definitions (CRDs) into a standardized, generic structure.
Apache-2.0Aug 5, 2026 · metrics 2.10.0
crates.io · PyPI
lightseekorg/smg
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.
Apache-2.0Jul 16, 2026 · metrics 2.10.0
PyPI · npm
roboflow/inference
Turn any computer or edge device into a command center for your computer vision projects.
Custom licenseSep 5, 2026 · metrics 2.10.0
PyPI
triton-inference-server/model_analyzer
Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.
Apache-2.0Jul 29, 2026 · metrics 2.10.0
Go
utkuozdemir/nvidia_gpu_exporter
Nvidia GPU exporter for prometheus using nvidia-smi binary
MITJul 22, 2026 · metrics 2.10.0
—
NVIDIA/nvcf
Platform for deploying and routing GPU-accelerated inference, streaming, and batch workloads at scale.
Apache-2.0Jul 17, 2026 · metrics 2.10.0
PyPI
Tencent/ncnn
ncnn is a high-performance neural network inference framework optimized for the mobile platform
Custom licenseAug 12, 2026 · metrics 2.10.0
npm
colinhacks/zod
TypeScript-first schema validation with static type inference
MITAug 4, 2026 · metrics 2.10.0