Crawlee—A web scraping and browser automation library for Python to build reliable crawlers. Extract data for AI, LLMs, RAG, or GPTs. Download HTML, PDF, JPG, PNG, and other files from websites. Works with Parsel, BeautifulSoup, Playwright, and raw HTTP. Both headful and headless mode. With proxy rotation.
Python · MDXApache-2.0★ 9,535 stars⑂ 807 forkssince Jan 2024View on GitHub ↗
apify/crawlee-python holds a health index of 99 out of 100, placing it in the Exceptional band. It scores highest on Vitality (100/100) and lowest on AI Readiness (78/100). It was last updated 1 day ago. 2 contributors account for most of its recent work.
99
overall / 100
Exceptional
Software health index
Metrics are grouped into weighted categories on one standardized 1–100 scale. Overall starts as their weighted mean, calibrated against the distribution of the public record so bands carry percentile meaning; when public evidence triggers the High-Risk Jurisdiction Policy, the rating is adjusted and receives an At Risk ceiling of 34.
99
Exceptional93-100The record's top tier (≈ top 5%); essentially all checked criteria met
Excellent80-92Strong across the board; minor gaps
Good65-79Healthy; gaps are limited and manageable
Moderate50-64Acceptable with notable gaps; review recommended
Weak35-49Material weaknesses across several areas
At Risk20-34Significant weaknesses; adoption warrants caution
Critical1-19Severe problems (abandoned, single-maintainer, no hygiene)
Score profile
Each axis is a category. The shape matters more than the average — a healthy subject fills the whole shape, while a spike-and-crater profile means strength in one dimension is masking risk in another.
The weighted overall 89 is calibrated to 99 on the published index scale (record calibration 2026-08-02).
Direct dependencies free of known advisories — no direct dependency carries a known advisory
25/25
Indirect dependencies free of known advisories — no indirect dependency carries a known advisory
0/40
No advisories left outstanding — no advisory carries a publication date
Inputs used
source
osv
advisories
0
affected_packages
0
assessed_packages
25
unassessed_packages
0
affected_by_severity
none
direct_affected_packages
0
Excluded from scoring (no data or not applicable): No advisories left outstanding. Remaining weights renormalized. Matched the pypi:crawlee@1.10.1 runtime dependency closure — what installing the published package pulls in — 25 packages. Reachability is not analyzed.
How well is the repo equipped to be developed and maintained with AI coding agents? Carries a deliberately small weight (4%): agent tooling is a real maintenance signal, but a repository with none can still reach 100/100.
API schema (OpenAPI/GraphQL/proto) — not applicable to this kind of software
0/20
MCP server — not applicable to this kind of software
40/40
Runnable examples — examples
Inputs used
example_dirs
examples
has_mcp_signal
no
api_schema_files
—
interfaces_expected_of
—
Excluded from scoring (no data or not applicable): API schema (OpenAPI/GraphQL/proto), MCP server. Remaining weights renormalized.
Key facts
9,535GitHub stars
48contributors
765commits, last 12 months
1days since last push
76releases
2bus factor
87open issues
npm, PyPIpackage ecosystems
Data collection warnings
Star history unavailable: GitHub GraphQL error: Resource not accessible by personal access token
First-time contributor figures cover 12 of 13 authors (cap 12)
More detail
Star and fork history 0 ★ / 807 ⇿
0Stars
807Forks
72Releases
When each star and fork was added, collected from GitHub and bucketed by day. Cumulative growth sits directly above the daily additions it is made of, so the two read against each other: steady organic accretion looks nothing like an abrupt, short-lived burst. Where that difference is measurable, it is reported as growth authenticity.
Major 1Minor 16Patch 55
Each point covers 3 days.
OpenSSF Scorecard 7.2 / 10
7.2aggregate
Independent, tool-agnostic security assessment from the open-source OpenSSF Scorecard. Each check rewards a security practice, not a specific vendor's tool. Checks Scorecard could not determine are marked n/a and excluded from the security score (never counted as zero).Scorecard v5.5.0 · 2026-09-17 18:36 UTC
Full resolved dependency set from the GitHub dependency graph: 12 direct and 212 indirect (transitive) packages. The transitive closure is complete when the repository commits a lockfile.
Registry
Package
Version
Relation
PyPI
async-timeout
5.0.1
direct
PyPI
cachetools
7.1.8
direct
PyPI
colorama
0.4.6
direct
PyPI
impit
0.14.1
direct
PyPI
more-itertools
11.1.0
direct
PyPI
protego
0.6.2
direct
PyPI
psutil
7.2.2
direct
PyPI
pydantic
2.13.5
direct
PyPI
pydantic-settings
2.15.0
direct
PyPI
tldextract
5.3.2
direct
PyPI
typing-extensions
4.16.0
direct
PyPI
yarl
1.24.5
direct
npm
@pnpm/exe.android-arm64
12.4.1
indirect
npm
@pnpm/exe.android-x64
12.4.1
indirect
npm
@pnpm/exe.darwin-arm64
12.4.1
indirect
npm
@pnpm/exe.darwin-x64
12.4.1
indirect
npm
@pnpm/exe.freebsd-x64
12.4.1
indirect
npm
@pnpm/exe.linux-arm64
12.4.1
indirect
npm
@pnpm/exe.linux-arm64-musl
12.4.1
indirect
npm
@pnpm/exe.linux-ppc64
12.4.1
indirect
npm
@pnpm/exe.linux-riscv64
12.4.1
indirect
npm
@pnpm/exe.linux-s390x
12.4.1
indirect
npm
@pnpm/exe.linux-x64
12.4.1
indirect
npm
@pnpm/exe.linux-x64-musl
12.4.1
indirect
npm
@pnpm/exe.win32-arm64
12.4.1
indirect
npm
@pnpm/exe.win32-x64
12.4.1
indirect
npm
loader-utils
^3.2.1
indirect
npm
pnpm
12.4.1
indirect
PyPI
aiomysql
0.3.2
indirect
PyPI
aiosqlite
0.22.1
indirect
PyPI
annotated-doc
0.0.5
indirect
PyPI
annotated-types
0.8.0
indirect
PyPI
ansicon
1.89.0
indirect
PyPI
anyio
4.15.1
indirect
PyPI
apify
—
indirect
PyPI
apify-client
2.5.1
indirect
PyPI
apify-fingerprint-datapoints
0.15.0
indirect
PyPI
apify-shared
2.3.0
indirect
PyPI
arrow
1.4.0
indirect
PyPI
asyncpg
0.31.0
indirect
PyPI
backports-asyncio-runner
1.2.0
indirect
PyPI
backports-zstd
1.7.0
indirect
PyPI
beautifulsoup4
4.15.0
indirect
PyPI
binaryornot
0.6.0
indirect
PyPI
black
26.5.1
indirect
PyPI
blessed
1.49.0
indirect
PyPI
brotli
1.2.0
indirect
PyPI
brotlicffi
1.2.0.2
indirect
PyPI
browserforge
1.2.4
indirect
PyPI
build
1.6.1
indirect
PyPI
camoufox
—
indirect
PyPI
certifi
2026.7.22
indirect
PyPI
cffi
2.1.1
indirect
PyPI
cfgv
3.5.0
indirect
PyPI
charset-normalizer
3.5.1
indirect
PyPI
click
8.5.0
indirect
PyPI
cloudpickle
3.1.2
indirect
PyPI
cookiecutter
2.7.1
indirect
PyPI
coverage
7.16.0
indirect
PyPI
crawlee
1.10.1
indirect
PyPI
cryptography
50.0.1
indirect
PyPI
cssselect
1.5.0
indirect
PyPI
curl-cffi
0.16.3
indirect
PyPI
databind
4.5.5
indirect
PyPI
databind-core
4.5.5
indirect
PyPI
databind-json
4.5.5
indirect
PyPI
deprecated
1.3.1
indirect
PyPI
distlib
0.4.3
indirect
PyPI
distro
1.9.0
indirect
PyPI
docspec
2.2.1
indirect
PyPI
docspec-python
2.2.2
indirect
PyPI
docstring-parser
0.11
indirect
PyPI
editor
1.8.0
indirect
PyPI
exceptiongroup
1.3.1
indirect
PyPI
execnet
2.1.2
indirect
PyPI
fakeredis
2.38.0
indirect
PyPI
filelock
3.32.6
indirect
PyPI
genai-prices
0.1.6
indirect
PyPI
googleapis-common-protos
1.75.3
indirect
PyPI
greenlet
3.5.5
indirect
PyPI
griffelib
2.3.0
indirect
PyPI
grpcio
1.83.1
indirect
PyPI
h11
0.16.0
indirect
PyPI
h2
4.4.1
indirect
PyPI
hiredis
3.4.1
indirect
PyPI
hpack
4.2.0
indirect
PyPI
html5lib
1.1
indirect
PyPI
httpcore
1.0.9
indirect
PyPI
httpcore2
2.12.0
indirect
PyPI
httptools
0.8.0
indirect
PyPI
httpx
0.28.1
indirect
PyPI
httpx2
2.12.0
indirect
PyPI
httpx2-jsfetch
1.0
indirect
PyPI
hyperframe
6.1.0
indirect
PyPI
identify
2.6.19
indirect
PyPI
idna
3.19
indirect
PyPI
importlib-metadata
9.0.1
indirect
PyPI
iniconfig
2.3.0
indirect
PyPI
inquirer
3.4.1
indirect
PyPI
jaro-winkler
2.0.3
indirect
PyPI
jinja2
3.1.6
indirect
PyPI
jinxed
2.1.0
indirect
PyPI
jiter
0.17.0
indirect
PyPI
jmespath
1.1.0
indirect
PyPI
joblib
1.6.0
indirect
PyPI
jsonpath-ng
1.8.0
indirect
PyPI
logfire-api
5.1.0
indirect
PyPI
lupa
2.8
indirect
PyPI
lxml
6.1.3
indirect
PyPI
lxml-html-clean
0.4.5
indirect
PyPI
markdown-it-py
4.2.0
indirect
PyPI
markupsafe
3.0.3
indirect
PyPI
mdurl
0.1.2
indirect
PyPI
multidict
6.8.0
indirect
PyPI
mypy-extensions
1.1.0
indirect
PyPI
narwhals
2.26.0
indirect
PyPI
nodeenv
1.10.0
indirect
PyPI
nr-date
2.1.0
indirect
PyPI
nr-stream
1.1.5
indirect
PyPI
nr-util
0.8.12
indirect
PyPI
numpy
2.2.6
indirect
PyPI
numpy
2.4.6
indirect
PyPI
numpy
2.5.3
indirect
PyPI
openai
3.13.0
indirect
PyPI
opentelemetry-api
1.44.0
indirect
PyPI
opentelemetry-distro
0.65b0
indirect
PyPI
opentelemetry-exporter-otlp
1.44.0
indirect
PyPI
opentelemetry-exporter-otlp-proto-common
1.44.0
indirect
PyPI
opentelemetry-exporter-otlp-proto-grpc
1.44.0
indirect
PyPI
opentelemetry-exporter-otlp-proto-http
1.44.0
indirect
PyPI
opentelemetry-instrumentation
0.65b0
indirect
PyPI
opentelemetry-instrumentation-httpx
0.65b0
indirect
PyPI
opentelemetry-proto
1.44.0
indirect
PyPI
opentelemetry-sdk
1.44.0
indirect
PyPI
opentelemetry-semantic-conventions
0.65b0
indirect
PyPI
opentelemetry-util-http
0.65b0
indirect
PyPI
packaging
26.3
indirect
PyPI
parsel
1.11.0
indirect
PyPI
pastel
0.2.1
indirect
PyPI
pathspec
1.1.1
indirect
PyPI
platformdirs
4.11.8
indirect
PyPI
playwright
1.62.0
indirect
PyPI
pluggy
1.6.0
indirect
PyPI
poethepoet
0.48.0
indirect
PyPI
pre-commit
4.6.2
indirect
PyPI
propcache
0.5.2
indirect
PyPI
protobuf
7.36.1
indirect
PyPI
proxy-py
2.4.10
indirect
PyPI
pycparser
3.0
indirect
PyPI
pydantic-ai-slim
2.43.0
indirect
PyPI
pydantic-core
2.46.5
indirect
PyPI
pydantic-graph
2.43.0
indirect
PyPI
pydoc-markdown
4.8.2
indirect
PyPI
pyee
13.0.1
indirect
PyPI
pygments
2.21.0
indirect
PyPI
pymysql
1.2.0
indirect
PyPI
pyprobables
0.7.0
indirect
PyPI
pyproject-hooks
1.2.0
indirect
PyPI
pytest
9.1.1
indirect
PyPI
pytest-asyncio
1.4.0
indirect
PyPI
pytest-cov
7.1.0
indirect
PyPI
pytest-rerunfailures
16.6.1
indirect
PyPI
pytest-timeout
2.4.0
indirect
PyPI
pytest-xdist
3.8.0
indirect
PyPI
python-dateutil
2.9.0.post0
indirect
PyPI
python-discovery
1.6.0
indirect
PyPI
python-dotenv
1.2.3
indirect
PyPI
python-slugify
9.0.0
indirect
PyPI
pytokens
0.4.1
indirect
PyPI
pyyaml
6.0.3
indirect
PyPI
readchar
4.2.2
indirect
PyPI
redis
8.1.0
indirect
PyPI
regex
2026.9.10
indirect
PyPI
requests
2.34.2
indirect
PyPI
requests-file
3.0.1
indirect
PyPI
rich
15.0.0
indirect
PyPI
ruff
0.16.7
indirect
PyPI
runs
1.3.0
indirect
PyPI
scikit-learn
1.7.2
indirect
PyPI
scikit-learn
1.9.1
indirect
PyPI
scipy
1.15.3
indirect
PyPI
scipy
1.17.1
indirect
PyPI
scipy
1.18.1
indirect
PyPI
setuptools
84.0.0
indirect
PyPI
shellingham
1.5.4
indirect
PyPI
six
1.17.0
indirect
PyPI
sniffio
1.3.1
indirect
PyPI
sortedcontainers
2.4.0
indirect
PyPI
soupsieve
2.9.2
indirect
PyPI
sqlalchemy
2.0.52
indirect
PyPI
stagehand
3.23.0
indirect
PyPI
text-unidecode
1.3
indirect
PyPI
threadpoolctl
3.6.0
indirect
PyPI
tiktoken
0.14.0
indirect
PyPI
tomli
2.4.1
indirect
PyPI
tomli-w
1.2.0
indirect
PyPI
truststore
0.10.4
indirect
PyPI
ty
0.0.80
indirect
PyPI
typeapi
2.3.0
indirect
PyPI
typer
0.27.2
indirect
PyPI
types-beautifulsoup4
4.12.0.20250516
indirect
PyPI
types-cachetools
7.0.0.20260713
indirect
PyPI
types-colorama
0.4.15.20260508
indirect
PyPI
types-html5lib
1.1.11.20260518
indirect
PyPI
types-psutil
7.2.2.20260906
indirect
PyPI
types-python-dateutil
2.9.0.20260807
indirect
PyPI
types-webencodings
0.6.0.20260907
indirect
PyPI
typing-inspection
0.4.4
indirect
PyPI
tzdata
2026.4
indirect
PyPI
urllib3
2.7.0
indirect
PyPI
uvicorn
0.52.4
indirect
PyPI
uvloop
0.22.1
indirect
PyPI
virtualenv
21.7.9
indirect
PyPI
w3lib
2.4.1
indirect
PyPI
watchdog
6.0.0
indirect
PyPI
watchfiles
1.2.0
indirect
PyPI
wcwidth
0.8.3
indirect
PyPI
webencodings
0.6.1
indirect
PyPI
websockets
16.1.1
indirect
PyPI
websockets
17.1
indirect
PyPI
wrapt
2.4.1
indirect
PyPI
xmod
1.10.0
indirect
PyPI
yapf
0.43.0
indirect
PyPI
zipp
4.1.0
indirect
Dependency advisories 0
Installing pypi:crawlee@1.10.1 pulls in 25 packages, direct and transitive: 0 carry known advisories, of which 0 are direct dependencies.
No known advisories affect the assessed dependencies.
An advisory means the version recorded in the dependency graph falls inside an advisory’s affected range. Reachability is not analysed, and the graph includes development and test pins — a finding may concern tooling rather than shipped software.
Raw JSON report machine-readable
Feedback
Spotted something off in this report, or have thoughts to share? Wrong measurements, missed tooling, ideas, questions — anything is welcome. Every message is read and gets a response.
Related records
Inspected repositories sharing catalogue tags or ecosystems with apify/crawlee-python. Read them side by side →
Scores are signals, not warranties. They reflect publicly visible practices on GitHub — not a code audit, and not a security guarantee.
Missing data is excluded and weights renormalized, never scored as zero. Methodology is versioned and open: metrics v2.10.0, schema v0.34.0 — full methodology · metrics wiki.