#local-llm

A harness optimized to smaller LLMs

Keine Repository-Beschreibung veröffentlicht.

A coding agent that runs entirely on your own machine: open-weight models, up to 256K context, NVIDIA GPU or Apple Silicon Mac, via Ollama.

Local agent infrastructure in one stdlib-only Go binary

Distribute GGUF and safetensors weights through any OCI registry, with signatures verified before a byte moves. Daemonless, runs offline, one static binary.

Terminal coding agent for local LLMs (llama.cpp, Ollama, vLLM) and any OpenAI-compatible API. OS-sandboxed autonomy, MCP, sessions. Go.

Confidential local RAG for your coding agents.

Write once. Generate everything. A modular, open-source ecosystem for building durable, observable, multimodal AI systems. Text, voice, image, video, music — one unified interface, any provider, any model, local or cloud.

Node.js binding of llama.cpp

CLI, SDK, and local Studio dashboard to automate i18n translation across i18next, vue-i18n, next-intl, ngx-translate, XLIFF, YAML, Flutter ARB, and Java properties, using OpenAI, Anthropic, Gemini, DeepL, or an openai-compatible model.

Self-hosted family AI gateway with parental controls. Runs on Linux, macOS, and Android (Termux) — Raspberry Pi, mini PC, old laptop, homelab server, even a phone. Telegram, Discord, web. Privacy-first, works with any LLM (local or cloud), OPA content filtering, MCP skill scanning.

Structured planning, bounded memory, TDD, and Git guardrails for local coding models in Pi Code (I don't know TypeScript at all; this is mostly a local-model experiment, with occasional help from Claude Code)

Dulus Ai — Agentic AI, 100% cache hit rate, Dulus is made to save you money [Gui, Web, Cli, Telegram, 2,000 MCP, 100K Skills . LiteLLM (100+ providers), local models via Ollama, /lang in 34 languages, Mesa Redonda, I create the first utility coin that can be used 100% as AI quota or Fuel, is called $Dulus

Agent-first CLI for Proton Mail. Local AI inbox classification, search, and Q&A via Proton Mail Bridge and Ollama. Nothing leaves your machine.

Up to 4× faster LLM decoding on Apple Silicon, lossless. Native MLX port of DeepSeek's DSpark & z-lab's DFlash speculative decoding — Gemma-4, Qwen3.8, Muse-Glimmer, Nemotron, LFM2.5, Ornith-1.0, ternary Bonsai-27B.

Autonomous AI pentesting engine across web, cloud, identity, CI/CD, IaC, databases, Active Directory, Kubernetes, IoT firmware and AI/LLM endpoints (OWASP LLM Top 10). Real exploits with proof for every finding. Privacy gateway: the LLM never sees your real IPs, hosts or creds; nothing leaves your perimeter.

Native K-quant support for MLX, with a quantization and fine-tuning toolchain for Apple Silicon

One CLI for all your robots. Connect them, command them, and let them work together, each with an LLM for a brain (Claude, OpenAI, Gemini, Grok, or local via Ollama or vLLM), VLAs for arms and decision LLMs (Jev, Laya, Kev). Drives Microduck, Open Duck Mini, LeRobot, XLeRobot, AlohaMini, ToddlerBot, or any ROS 2 base from a laptop, offboard.

Run a 110B on a 2016 PC with 16 GB RAM. Know your tok/s before you download. Placement beats budget: predicts speed + memory fit for any GGUF on your exact hardware, self-calibrates, emits the exact llama.cpp command — or 'quantprobe auto' does it all. Falsification-tested laws; misses published at full size. pip install quantprobe

Agentic Runtime

Local-first AI coding agent runtime — your models, fail-closed safety, evidence-gated self-improvement

Flywheel runs AI tasks with a model you choose and keeps a receipt anyone can recheck offline.

Delegate summarize/classify/extract/triage to a FREE local Gemma-4 cascade via llama.cpp. Go CLI + MCP server; never calls a cloud model. Includes a turnkey setup skill for Claude Code.

Air-gap-native agentic coding for closed environments — a hardened OpenCode TUI driven by a local model, with rtmx as the intent layer. Zero egress by construction.

Pure-Go, no-cgo local LLM inference — run Gemma, Qwen, Llama and friends from safetensors or GGUF in a single static binary.