#cuda

World's fastest and most compact embedded vector database: exact by default, multimodal, local-first, and GPU-accelerated

Consumer-aware compression for embedding indexes and LLM KV caches — compress by the metric the downstream consumer actually uses. PCA-Matryoshka + TurboQuant (27x @ 99.8% recall@10), asymmetric K/V, CUDA/Triton kernels, vLLM plugin, replayable CI-gated claims. MIT.

Speaker diarization in Rust with pyannote-level accuracy. 900–1940x realtime on NVIDIA GPUs, 430–910x on Apple Silicon.

Safe rust wrapper around CUDA toolkit

Rust JPEG 2000 / HTJ2K codec with CUDA and Metal GPU paths.

Prioritized inverse kinematics for many robot bodies: a C++ solver with Python bindings, collision and balance constraints, and an experimental CUDA batch whole-body solver.

Fast, GPU-ready atmospheric radiative transfer in Julia: the RTE solver with RRTMGP correlated-k gas optics.

Go with your own intelligence - Write Go applications that directly integrate llama.cpp for local inference using hardware acceleration on Linux, macOS, Windows, & WebAssembly.

AI sovereignty for the world's most trusted runtime.

GPU-accelerated transparent compression S3-compatible storage gateway. Drop-in replacement for AWS S3 endpoints; cuts your S3 bill 50-80% with no app changes (Rust, nvCOMP, zstd).

Train and serve LLMs at extreme speed and massive throughput.

.NET library for on-demand local AI model inference — zero bundled models, lazy loading, hardware-aware GPU/CPU selection, 10 task types including embeddings, generation, vision, and audio.

CUDA driver API in pure Go, no cgo: loads libcuda at runtime, embeds PTX, JIT-compiles through the driver

Search code and feature records by meaning through the command line or Model Context Protocol (MCP). Inference runs on your GPU.

The fastest .NET tensor library. Beats MathNet (6x), NumSharp (3200x), matches TorchSharp CPU - pure managed C# with hand-tuned AVX2/FMA SIMD kernels. Optional CUDA/OpenCL GPU acceleration.

Train and serve MoE models that do not fit in VRAM: fused 4-bit experts, QLoRA, CPU/NVMe offload, and fast inference on consumer NVIDIA GPUs.

Make GPUs do branchy stuff fast.

An another Node binding of whisper.cpp to make same API with whisper.rn as much as possible.

Deep Learning framework in Go with Tensors, AutoGrad, and GPU acceleration

CUDA grammar for tree-sitter

TurboQuant+ KV cache compression for vLLM. 3.8x smaller KV cache, same conversation quality. Fused CUDA kernels with automatic PyTorch fallback.

JPEG XR decoder in Rust with optional Metal and CUDA backends

Triton kernels for 4-bit MoE inference: grouped NF4/MXFP4 GEMM, INT4 GEMV, FP8 paged attention, and CPU/NVMe expert streaming.

CUDA dependency management and process isolation for ComfyUI custom nodes

ArrayFire: a general purpose GPU library.