jmagar/crawlerr
If you are the rightful owner of crawlerr and would like to certify it and/or have it hosted online, please leave a comment on the right or send an email to henry@mcphub.com.
Crawlerr is an advanced RAG-enabled web crawling MCP server that integrates semantic search capabilities with web crawling, built using FastMCP 2.0, Crawl4AI 0.7.0, Qdrant vector database, and Qwen3-Embedding-0.6B.
Tools
Functions exposed to the LLM to take actions
scrape
Single page crawling with advanced extraction capabilities.
crawl
Comprehensive site crawling via sitemap or recursive strategies.
crawl_repo
GitHub repository cloning and analysis.
crawl_dir
Local directory processing with file type detection.
rag_query
Semantic search across all crawled content.
list_sources
Enumerate and filter all crawled sources.
Prompts
Interactive templates invoked by user choice
No prompts
Resources
Contextual data attached and managed by the client