PyPI
google/jetstream
JetStream is a throughput and memory optimized engine for LLM inference on XLA devices, starting with TPUs (and GPUs in future -- PRs welcome).
Apache-2.0Jul 18, 2026 · metrics 2.10.0
All tags
Catalogue tag
#llm-inference
Every repository in the public record carrying this tag — from its GitHub topics or the keywords its package registries publish. Health is measured under the same versioned methodology as the rest of the record.
35 records
PyPI
ARahim3/mlx-dspark
Up to 4× faster LLM decoding on Apple Silicon, lossless. Native MLX port of DeepSeek's DSpark & z-lab's DFlash speculative decoding — Gemma-4, Qwen3.8, Muse-Glimmer, Nemotron, Ornith-1.0, ternary Bonsai-27B.
MITAug 19, 2026 · metrics 2.10.0
npm
empirical-run/empirical
Test and evaluate LLMs and model configurations, across all the scenarios that matter for your application
MITJul 15, 2026 · metrics 2.10.0
—
MekayelAnik/vllm-cpu
Wheels & Docker images for running vLLM on CPU-only systems, optimized for different CPU instruction sets
GPL-3.0Aug 10, 2026 · metrics 2.10.0
PyPI
openvinotoolkit/openvino
OpenVINO™ is an open source toolkit for optimizing and deploying AI inference
Apache-2.0Aug 12, 2026 · metrics 2.10.0