crawl4ai-mcp-server
If you are the rightful owner of crawl4ai-mcp-server and would like to certify it and/or have it hosted online, please leave a comment on the right or send an email to henry@mcphub.com.
Crawl4ai MCP Server provides web crawling capabilities using crawl4ai with markdown output for LLM.
The Crawl4ai MCP Server is designed to facilitate web crawling tasks by leveraging the capabilities of the crawl4ai platform. It outputs the crawled data in markdown format, making it suitable for integration with language learning models (LLM). The server is built on Node.js and requires access to a crawl4ai instance. It supports crawling multiple URLs and returns the content with proper citations, formatted in markdown. The server is configurable through environment variables, allowing for customization of the API URL and authentication token if needed. It also includes error handling mechanisms to manage common issues such as invalid URLs, authentication errors, and network connectivity problems.
Features
- Web Crawling: Crawl web pages and retrieve content in markdown format with citations.
- Markdown Output: Outputs crawled data in markdown format, suitable for LLM integration.
- Error Handling: Includes mechanisms to handle common errors such as invalid URLs and network issues.
- Authentication Support: Supports optional authentication for secure access to the crawl4ai API.
- Development Mode: Offers a development mode with auto-rebuild for easier testing and debugging.
Tools
crawl_urls
Crawl web pages and get markdown content with citations. Requires a list of URLs.
