apify/crawleeCrawlee—A web scraping and browser automation library for Node.js to build reliable crawlers. In JavaScript and TypeScript. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Puppeteer, Playwright, Cheerio, JSDOM, and raw HTTP. Both headful and headless mode. With proxy rotation.
TypeScript · MDX★ 26.2K↓ 6.8M/月2026年10月19日
apify/crawlee-pythonCrawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
Python · MDX★ 9,535↓ 975.8K/月2026年9月17日
apify/apify-client-pythonApify API client for Python—Programmatically run Actors, manage and stream data from storages (datasets, key-value stores, request queues), schedule and monitor runs, and access the full Apify platform API. Sync and async interfaces with automatic retries and pagination.
Python★ 97↓ 2.8M/月2026年8月27日
apify/apify-sdk-pythonApify SDK for Python—The official library for building Apify Actors: serverless cloud programs for web scraping, browser automation, data processing, and AI agents. Manages the Actor lifecycle, storages (datasets, key-value stores, request queues), events, proxies, and pay-per-event monetization. Built on top of the the Apify API Client.
Python · MDX★ 176↓ 232.6K/月2026年9月11日
PHP★ 3,296↓ 147.9K/月2026年9月12日
D4Vinci/Scrapling🕷️ An adaptive Web Scraping framework that handles everything from a single request to a full-scale crawl! Don't be shy, join here: https://discord.gg/EMgGbDceNQ
Python★ 81.4K2026年9月16日
Python★ 31K2026年9月16日
TypeScript · MDX★ 24.3K↓ 6.4M/月2026年9月16日
PyPI · Packagist · crates.io +294卓越健康指数
TypeScript · Python★ 181.1K↓ 5.8M/月2026年9月16日
lexiforest/curl_cffiPython binding for curl-impersonate fork via cffi. A http client that can impersonate browser tls/ja3/http2 fingerprints.
Python★ 6,5152026年9月20日
pinchtab/pinchtabHigh-performance browser automation bridge and multi-instance orchestrator with advanced stealth injection and real-time dashboard.
Go · Shell★ 10.1K↓ 7,120/月2026年8月28日
adbar/trafilaturaPython & Command-line tool to gather text and metadata on the Web: Crawling, scraping, extraction, output as CSV, JSON, HTML, MD, TXT, XML
Python★ 6,718↓ 14M/月2026年8月28日
crates.io · npm · PyPI91优秀健康指数
fastcrw/crwFast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.
Rust★ 1,095↓ 10.6K/月2026年10月2日
Python★ 13K↓ 3M/月2026年8月27日
crates.io · npm · PyPI91优秀健康指数
us/crwFast, lightweight Firecrawl/Tavily alternative in Rust. Web scraper, crawler & search API with MCP server for AI agents. Drop-in Firecrawl-compatible API (/scrape, /crawl, /search). 2.3x faster than Tavily, 1.5x faster than Firecrawl in 1K-URL benchmarks. 6 MB RAM, single binary. Self-host or use managed cloud.
Rust★ 1,032↓ 10.7K/月2026年9月16日
feder-cr/invisible_playwrightPlaywright that anti-bots cannot see, so no captchas: same API, anti-detect stealth headless Firefox, undetected fingerprint, bypass bot detection, Python web scraping and browser automation.
Python★ 2,499↓ 32.3K/月2026年10月21日
Python★ 480↓ 5,294/月2026年9月11日
jhy/jsoupjsoup: the Java HTML parser, built for HTML editing, cleaning, scraping, and XSS safety.
Java · HTML★ 11.4K2026年8月27日
tox-dev/turbohtmlA fast, fully typed HTML toolkit for Python. Escape, unescape, parse, select, and serialize HTML from one library, powered by a C-accelerated core.
C · Python★ 20↓ 30.3K/月2026年10月1日
yusufkaraaslan/Skill_SeekersConvert documentation websites, GitHub repositories, and PDFs into Claude AI skills with automatic conflict detection
Python★ 15K↓ 22.1K/月2026年9月19日
zoharbabin/web-researcher-mcpGive your AI assistant real web search, full-page reading, and multi-source research with citations that are never fabricated.
Go★ 572026年9月11日
TypeScript · JavaScript★ 7,332↓ 492.7K/月2026年8月28日
microlinkhq/browserlessThe headless Chrome/Chromium driver on top of Puppeteer. Take screenshots, generate PDFs, extract text and HTML with a production-ready API.
JavaScript★ 1,834↓ 71.9K/月2026年9月11日
sqdshguy/wreq-jsHTTP client for Node.js with browser TLS fingerprint impersonation
TypeScript · Rust★ 402↓ 498.7K/月2026年9月23日
MODSetter/SurfSenseOpen-source NotebookLM alternative. Research the open web with live data(Reddit, YT, IG, TikTok, Indeed, Google Search, Maps etc) through one platform, API or MCP server. Join our Discord: https://discord.gg/ejRNvftDp9
Python · TypeScript★ 16.2K2026年9月16日
Python★ 182↓ 12.1K/月2026年10月3日
Rust · Python★ 1,4412026年9月17日
0xMassi/webclawFast, local-first web content extraction for LLMs. Scrape, crawl, extract structured data — all from Rust. CLI, REST API, and MCP server.
Rust★ 2,343↓ 1,301/月2026年9月13日
getmaxun/maxun🔥 The open-source no-code platform for web scraping, crawling, search and AI data extraction • Turn websites into structured APIs in minutes 🔥
TypeScript★ 17.5K2026年9月16日
jo-inc/camofox-browserStealth headless browser for AI agents — bypass Cloudflare, bot detection, and anti-scraping. Drop-in Puppeteer/Playwright replacement.
JavaScript★ 8,924↓ 243.1K/月2026年8月28日
