#nvidia

A Datacenter Scale Distributed Inference Serving Framework

NeMo Guardrails is an open-source toolkit for easily adding programmable guardrails to LLM-based conversational systems.

Ongoing research training transformer models at scale

Run agents like Hermes, LangChain Deep Agents, and OpenClaw more securely inside NVIDIA OpenShell with managed inference

TensorRT LLM provides users with an easy-to-use Python API to define Large Language Models (LLMs) and supports state-of-the-art optimizations to perform inference efficiently on NVIDIA GPUs. TensorRT LLM also contains components to create Python and C++ runtimes that orchestrate the inference execution in a performant way.

CUDA Core Compute Libraries

NVIDIA cuML: GPU-Accelerated Machine Learning

A Python framework for GPU-accelerated simulation, robotics, and machine learning.

OpenShell is the safe, private runtime for autonomous AI agents.

FlashInfer: Kernel Library for LLM Serving

🎨 NeMo Data Designer: Generate high-quality synthetic data from scratch or from seed data.

cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

Evaluate and improve models and agents using environments

Multi-language agent runtime and library for execution scope management, lifecycle events, and middleware on tool and LLM calls.

Platform for deploying and routing GPU-accelerated inference, streaming, and batch workloads at scale.

Offline optimization of your disaggregated Dynamo graph

Model Express is a Rust-based component meant to be placed next to existing model inference systems to speed up their startup times and improve overall performance.

Open-source local AI SDK - run AI on-device with no cloud, no API keys. Supports GGUF, RAG, image, music, and video generation, speech-to-text, P2P inference, and more. Cross-platform: Linux, macOS, Windows, Android, iOS.

The NVIDIA® Tools Extension SDK (NVTX) is a C-based Application Programming Interface (API) for annotating events, code ranges, and resources in your applications.

CUDA Templates and Python DSLs for High-Performance Linear Algebra

NVIDIA Design System and UI Agent Harness for AI/ML Factories, Robotics, and Autonomous Vehicles

PyTorch/TorchScript/FX compiler for NVIDIA GPUs using TensorRT

Nvidia GPU exporter for prometheus using nvidia-smi binary OR using NVML