#nvidia

Frontier CoreML audio models in your apps — text-to-speech, speech-to-text, voice activity detection, and speaker diarization. In Swift, powered by SOTA open source.

GPUd automates monitoring, diagnostics, and issue identification for GPUs

Notebook images for ODH

Triton Model Analyzer is a CLI tool to help with better understanding of the compute and memory requirements of the Triton Inference Server models.

Nvidia GPU exporter for prometheus using nvidia-smi binary

Platform for deploying and routing GPU-accelerated inference, streaming, and batch workloads at scale.

📊 Simple package for monitoring and control your NVIDIA Jetson [Orin, Xavier, Nano, TX] series

Kubernetes operator for self-hosted LLM inference across a heterogeneous GPU fleet: NVIDIA CUDA, AMD Vulkan, and Apple Silicon Metal. Runtimes: llama.cpp, vLLM, TGI, mlx-server. Multi-GPU sharding, model caching, OpenAI-compatible endpoints. Apache-2.0, run across homelab and on-prem fleets, actively developed.

NVIDIA® TensorRT™ is an SDK for high-performance deep learning inference on NVIDIA GPUs. This repository contains the open source components of TensorRT.

cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

Tensor Fusion is a state-of-the-art GPU virtualization and pooling solution designed to optimize GPU cluster utilization to its fullest potential.

Crater is a cloud-native AI training & inference platform.

Optimized primitives for collective multi-GPU communication

Tensor Fusion is a state-of-the-art GPU virtualization and pooling solution designed to optimize GPU cluster utilization to its fullest potential.

Find, benchmark and install in CLI 170+ FREE coding LLM models across 15+ providers in real time