Catalogue tag

#crawlers

Every repository in the public record carrying this tag — from its GitHub topics or the keywords its package registries publish. Health is measured under the same versioned methodology as the rest of the record.

4 records

Tagged “crawlers”Ranked by health index

npm

99Exceptionalhealth index

apify/crawlee

Crawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.

TypeScript · MDX★ 26.2K↓ 6.8M/moOct 19, 2026

Apache-2.0Oct 19, 2026 · metrics 2.10.0

npm

86Excellenthealth index

omrilotan/isbot

🤖/👨‍🦰 Detect bots/crawlers/spiders using the user agent string

TypeScript · JavaScript★ 1,160↓ 98.8M/moSep 16, 2026

UnlicenseSep 16, 2026 · metrics 2.10.0

npm · Packagist · Go +1

80Excellenthealth index

monperrus/crawler-user-agents

Syntactic patterns of HTTP user-agents used by bots / robots / crawlers / scrapers / spiders. pull-request welcome :star:

Go · Python★ 1,403↓ 277.6K/moSep 15, 2026

MITSep 15, 2026 · metrics 2.10.0

Maven

77Goodhealth index

Norconex/crawler

Norconex Crawlers (or spiders) are flexible web and filesystem crawlers for collecting, parsing, and manipulating data from the web or filesystem to various data repositories such as search engines.

Java★ 205Oct 21, 2026

Apache-2.0Oct 21, 2026 · metrics 2.10.0