PyPI · crates.io
ai-dynamo/dynamo
A Datacenter Scale Distributed Inference Serving Framework
Licencia propia28 ago 2026 · métricas 2.10.0
Todas las etiquetas
Etiqueta del catálogo
#inference
Todos los repositorios del registro público que llevan esta etiqueta, procedente de sus topics de GitHub o de las palabras clave que publican sus registros de paquetes. La salud se mide con la misma metodología versionada que el resto del registro.
106 registros
PyPI
huggingface/transformers
🤗 Transformers: the model-definition framework for state-of-the-art machine learning models in text, vision, audio, and multimodal models, for both inference and training.
Apache-2.04 ago 2026 · métricas 2.10.0
PyPI · crates.io
vllm-project/vllm
A high-throughput and memory-efficient inference and serving engine for LLMs
Apache-2.04 ago 2026 · métricas 2.10.0
PyPI · Go
LMCache/LMCache
LMCache: Supercharge Your LLM with the Fastest KV Cache Layer
Apache-2.015 ago 2026 · métricas 2.10.0
PyPI · crates.io
apache/tvm-ffi
Open ABI and FFI for Machine Learning Systems
Apache-2.027 ago 2026 · métricas 2.10.0
PyPI
kvcache-ai/Mooncake
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
Apache-2.012 ago 2026 · métricas 2.10.0
PyPI · crates.io
sgl-project/sglang
SGLang is a high-performance serving framework for large language models and multimodal models.
Apache-2.04 ago 2026 · métricas 2.10.0
PyPI
triton-inference-server/server
The Triton Inference Server provides an optimized cloud and edge inferencing solution.
BSD-3-Clause27 ago 2026 · métricas 2.10.0
PyPI · RubyGems
deepspeedai/DeepSpeed
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.
Apache-2.013 ago 2026 · métricas 2.10.0
PyPI
pytorch/ao
PyTorch native quantization and sparsity for training and inference
Licencia propia21 jul 2026 · métricas 2.10.0
PyPI
mozilla-ai/any-llm
Communicate with an LLM provider using a single interface
Apache-2.021 jul 2026 · métricas 2.10.0
npm
vercel/ai
The AI Toolkit for TypeScript. From the creators of Next.js, the AI SDK is a free open-source library for building AI-powered applications and agents
Licencia propia5 ago 2026 · métricas 2.10.0
PyPI · crates.io
ai-dynamo/aiconfigurator
Offline optimization of your disaggregated Dynamo graph
Apache-2.026 jul 2026 · métricas 2.10.0
crates.io
ai-dynamo/modelexpress
Model Express is a Rust-based component meant to be placed next to existing model inference systems to speed up their startup times and improve overall performance.
Apache-2.01 ago 2026 · métricas 2.10.0
PyPI
gpustack/gpustack
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
Apache-2.02 ago 2026 · métricas 2.10.0
npm
huggingface/huggingface.js
Use Hugging Face with JavaScript
MIT27 ago 2026 · métricas 2.10.0
PyPI
huggingface/optimum
🚀 Accelerate inference and training of 🤗 Transformers, Diffusers, TIMM and Sentence Transformers with easy to use hardware optimization tools
Apache-2.021 jul 2026 · métricas 2.10.0
crates.io · npm · PyPI
pykeio/ort
Fast ML inference & training for ONNX models in Rust
Apache-2.027 ago 2026 · métricas 2.10.0
npm · RubyGems
vega/vega
A visualization grammar.
BSD-3-Clause27 ago 2026 · métricas 2.10.0
PyPI · npm · RubyGems
mlc-ai/xgrammar
Fast, Flexible and Portable Structured Generation
Apache-2.027 ago 2026 · métricas 2.10.0
PyPI
pytorch/TensorRT
PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT
BSD-3-Clause30 jul 2026 · métricas 2.10.0
PyPI
raullenchai/Rapid-MLX
The fastest local AI engine for Apple Silicon. 4.2x faster than Ollama, 0.08s cached TTFT, 100% tool calling. 17 tool parsers, prompt cache, reasoning separation, cloud routing. Drop-in OpenAI replacement. Works with Claude Code, Cursor, Aider.
Apache-2.02 ago 2026 · métricas 2.10.0
Go
run-ai/karta
Translation layer that maps any Kubernetes framework's Custom Resource Definitions (CRDs) into a standardized, generic structure.
Apache-2.05 ago 2026 · métricas 2.10.0
crates.io · PyPI
lightseekorg/smg
Engine-agnostic LLM gateway in Rust. Full OpenAI & Anthropic API compatibility across vLLM, TRT-LLM, TokenSpeed, SGLang, OpenAI, Gemini & more. Industry-first gRPC pipeline, KV cache-aware routing, chat history, tokenization caching, Responses API, embeddings, WASM plugins, MCP, and multi-tenant auth.
Apache-2.016 jul 2026 · métricas 2.10.0
PyPI · npm
roboflow/inference
Turn any computer or edge device into a command center for your computer vision projects.
Licencia propia5 sept 2026 · métricas 2.10.0
PyPI
triton-inference-server/model_analyzer
Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.
Apache-2.029 jul 2026 · métricas 2.10.0
Go
utkuozdemir/nvidia_gpu_exporter
Nvidia GPU exporter for prometheus using nvidia-smi binary
MIT22 jul 2026 · métricas 2.10.0
—
NVIDIA/nvcf
Platform for deploying and routing GPU-accelerated inference, streaming, and batch workloads at scale.
Apache-2.017 jul 2026 · métricas 2.10.0
PyPI
Tencent/ncnn
ncnn is a high-performance neural network inference framework optimized for the mobile platform
Licencia propia12 ago 2026 · métricas 2.10.0
npm
colinhacks/zod
TypeScript-first schema validation with static type inference
MIT4 ago 2026 · métricas 2.10.0