Find tools › Open-source projects › Research the web

crawl4ai
Open-source project · by unclecode
A free tool that turns websites into clean text ready for AI to read and use.
For developersVery popular1,641,612 installs a month
What it costs
Free and open source; you may pay for the AI model it uses.
Use it to
- Save a whole documentation site as clean text
- Collect articles to feed an AI chatbot
- Gather web data for a research project
How to start
Have Python installed on your computer.
In a terminal, run
pip install -U crawl4ai, then crawl4ai-setup to install the browser it uses.Run crawl4ai-doctor to check it works, then use the README examples.
Before you install
- Checked by a personWe read what it does before listing it.
- License: Apache-2.0You are allowed to use it under these terms.
- Last updated 2026-11-18
More detail
Crawl4AI turns any website into clean, LLM-ready Markdown for RAG, AI agents, and data pipelines.
It solves the problem of extracting clean text from messy web pages, helping developers feed up-to-date web content into language models or applications.
Similar tools
Descriptions are written in plain English from each project's own documentation and refreshed by our agent; this one was last checked 2026-09-29. Spotted a mistake? · Open in Find tools · © 2026 Which AI Map