A tool to extract plain (unformatted) multilingual / language-agnostic text, redirects, links and categories from wikipedia backups (dumps). Designed to prepare clean training data for AI Training / Machine Learning software.
A torch.compile backend that pays attention: custom FX fusion passes, a Triton LayerNorm kernel, and a persistent compile cache. Apache-2.0 free core of the g2n platform.