BjornMelin_crawl4ai-mcp-server

BjornMelin_crawl4ai-mcp-server

3.2

If you are the rightful owner of BjornMelin_crawl4ai-mcp-server and would like to certify it and/or have it hosted online, please leave a comment on the right or send an email to henry@mcphub.com.

The Crawl4AI MCP Server is a high-performance server designed to enable AI assistants to perform web scraping, crawling, and deep research using the Model Context Protocol.

The Crawl4AI MCP Server is a cutting-edge solution for AI-driven web data acquisition, offering seamless integration with the Crawl4AI library. Deployed on CloudFlare Workers, this server provides AI assistants like Claude with the ability to access powerful web scraping and crawling capabilities. It is designed to be faster and more efficient than existing solutions like FireCrawl. The server supports a multi-tenant architecture, ensuring flexibility with cloud providers and secure access through OAuth authentication. With a focus on high performance, the server is optimized for speed and efficiency, making it an ideal choice for AI applications requiring comprehensive web data extraction and processing.

Features

  • Single Webpage Scraping: Extract content from individual webpages efficiently.
  • Web Crawling: Perform comprehensive website crawling with configurable depth and page limits.
  • Asynchronous Crawling: Enhance efficiency by crawling entire websites asynchronously.
  • Structured Data Extraction: Use CSS selectors or LLM-based extraction for precise data retrieval.
  • OAuth Authentication: Secure access with OAuth or API key (Bearer token) options.

Tools

  1. crawl

    Crawl web pages from a starting URL.

  2. getCrawl

    Retrieve crawl data by ID.

  3. listCrawls

    List all crawls or filter by domain.

  4. search

    Search indexed documents by query.

  5. extract

    Extract structured content from a URL.