PyPI · crates.io
ai-dynamo/dynamo
A Datacenter Scale Distributed Inference Serving Framework
Custom licenseAug 28, 2026 · metrics 2.10.0
All tags
Catalogue tag
#llm-inference
Every repository in the public record carrying this tag — from its GitHub topics or the keywords its package registries publish. Health is measured under the same versioned methodology as the rest of the record.
51 records
Maven · PyPI
ray-project/ray
Ray is an AI compute engine. Ray consists of a core distributed runtime and a set of AI Libraries for accelerating ML workloads.
Apache-2.0Sep 16, 2026 · metrics 2.10.0
Go · PyPI
kserve/kserve
Standardized Distributed Generative and Predictive AI Inference Platform for Scalable, Multi-Framework Deployment on Kubernetes
Apache-2.0Sep 18, 2026 · metrics 2.10.0
PyPI
kvcache-ai/Mooncake
Mooncake is the serving platform for Kimi, a leading LLM service provided by Moonshot AI.
Apache-2.0Sep 17, 2026 · metrics 2.10.0
PyPI
flashinfer-ai/flashinfer
FlashInfer: Kernel Library for LLM Serving
Apache-2.0Aug 27, 2026 · metrics 2.10.0
npm
software-mansion/react-native-executorch
High-performance, privacy-first on-device AI inference library for React Native, powered by PyTorch's ExecuTorch runtime
Custom licenseOct 17, 2026 · metrics 2.10.0
PyPI
InternLM/lmdeploy
LMDeploy is a toolkit for compressing, deploying, and serving LLMs.
Apache-2.0Sep 20, 2026 · metrics 2.10.0
PyPI
bentoml/BentoML
The easiest way to serve AI apps and models - Build Model Inference APIs, Job queues, LLM apps, Multi-model pipelines, and more!
Apache-2.0Sep 17, 2026 · metrics 2.10.0
PyPI
gpustack/gpustack
A GPU cluster manager for high-performance AI model serving (vLLM, SGLang) and on-demand SSH-accessible GPU instances.
Apache-2.0Sep 15, 2026 · metrics 2.10.0
crates.io
spiceai/spiceai
Add a real-time analytics node to your operational database. Spice is a portable, accelerated SQL query, search, and LLM-inference engine in Rust for data-grounded AI apps and agents.
Apache-2.0Sep 20, 2026 · metrics 2.10.0
npm
tetherto/qvac
Open-source local AI SDK - run AI on-device with no cloud, no API keys. Supports GGUF, RAG, image, music, and video generation, speech-to-text, P2P inference, and more. Cross-platform: Linux, macOS, Windows, Android, iOS.
Apache-2.0Sep 10, 2026 · metrics 2.10.0
PyPI
lemonade-sdk/lemonade
Lemonade helps users discover and run local AI apps by serving optimized LLMs right from their own GPUs and NPUs. Join our discord: https://discord.gg/5xXzkMu8Zk
Apache-2.0Aug 28, 2026 · metrics 2.10.0
PyPI
monocle2ai/monocle
Monocle is a framework for tracing GenAI app code. This repo contains implementation of Monocle for GenAI apps written in Python.
Apache-2.0Aug 28, 2026 · metrics 2.10.0
Packagist · npm
neuron-core/neuron-ai
The Agentic Framework of the PHP ecosystem to build production-ready AI driven applications. Connect components (LLMs, Tools, vector DBs, memory) to agents that interact with your data and UI.
MITOct 22, 2026 · metrics 2.10.0
PyPI
Lightning-AI/litgpt
20+ high-performance LLMs with recipes to pretrain, finetune and deploy at scale.
Apache-2.0Aug 27, 2026 · metrics 2.10.0
npm
Nano-Collective/nanocoder
An open coding agent for your terminal, built by a community collective rather than a company. Bring your own model, keep your code on your machine, and owe nothing to anyone.
Custom licenseAug 25, 2026 · metrics 2.10.0
npm · crates.io
ruvnet/RuVector
RuVector provides High Performance, Real-Time decisions and agent memory , Self-Learning Ai, Vector GNN DB built in Rust.
MITOct 20, 2026 · metrics 2.10.0
Go · npm
matrixhub-ai/matrixhub
An Open-source, self-hosted AI model hub with Hugging Face compatibility, accelerating vLLM/SGLang performance.
Apache-2.0Sep 10, 2026 · metrics 2.10.0
Go · npm
ome-projects/ome
Open Model Engine (OME) — Kubernetes operator for LLM serving, GPU scheduling, and model lifecycle management. Works with SGLang, vLLM, TensorRT-LLM, and Triton
Apache-2.0Sep 12, 2026 · metrics 2.10.0
PyPI · npm
rajveer43/VeloxQuant-MLX
Fast KV-cache quantization for Apple Silicon (MLX) — 43 research-adapted compression methods with Metal kernels
MITOct 21, 2026 · metrics 2.10.0
PyPI
Nayjest/lm-proxy
OpenAI-compatible HTTP LLM proxy / gateway for multi-provider inference (Google, Anthropic, OpenAI, PyTorch). Lightweight, extensible Python/FastAPI—use as library or standalone service.
MITAug 26, 2026 · metrics 2.10.0
PyPI
algorithmicsuperintelligence/openevolve
Open-source implementation of AlphaEvolve
Apache-2.0Aug 28, 2026 · metrics 2.10.0
PyPI
Sahil170595/Chimeraforge
PyPI capacity-planning CLI for LLM deployment. pip install chimeraforge.
MITOct 20, 2026 · metrics 2.10.0
Go
gitleaks/gitleaks
Find secrets with Gitleaks 🔑
MITOct 18, 2026 · metrics 2.10.0
Go · npm
lordmathis/llamactl
Unified management and routing for llama.cpp, MLX and vLLM models with web dashboard.
MITSep 20, 2026 · metrics 2.10.0
npm
requestyai/ai-sdk-requesty
The Requesty integration package for the Vercel AI SDK.
Apache-2.0Sep 12, 2026 · metrics 2.10.0
crates.io
Muvon/octolib
One Rust API to 25+ AI providers — chat, vision, tool calls, structured output, embeddings & reranking with built-in cost tracking
Apache-2.0Sep 18, 2026 · metrics 2.10.0
npm
webgptorg/promptbook
Turn your company's scattered knowledge into AI ready Books ✨
Custom licenseOct 22, 2026 · metrics 2.10.0
Maven · npm
qxoticai/qxotic
AI sovereignty for the world's most trusted runtime.
Apache-2.0Oct 19, 2026 · metrics 2.10.0
PyPI
ARahim3/mlx-dspark
Up to 4× faster LLM decoding on Apple Silicon, lossless. Native MLX port of DeepSeek's DSpark & z-lab's DFlash speculative decoding — Gemma-4, Qwen3.8, Muse-Glimmer, Nemotron, LFM2.5, Ornith-1.0, ternary Bonsai-27B.
MITOct 19, 2026 · metrics 2.10.0