Valerio357/mcp_server1
3.1
If you are the rightful owner of mcp_server1 and would like to certify it and/or have it hosted online, please leave a comment on the right or send an email to dayong@mcphub.com.
The Web Crawler MCP Server is a Smithery-compatible FastMCP server designed to crawl web pages, extract text, detect titles, and discover outbound links.
Web Crawler MCP Server
This repository contains a Smithery-compatible FastMCP server that crawls web pages and returns a cleaned text preview alongside discovered hyperlinks.
Features
- Single URL crawl with text extraction, title detection, and outbound link discovery
- Batch crawl helper that executes requests concurrently
- Session-level configuration for user agent, timeouts, and output limits
Running Locally
- Install dependencies with your preferred package manager such as uv sync.
- Start the Smithery playground to interact with the server using: uv run playground
Container Deployment
- Build the container:
docker build -t web-crawler-mcp . - Run locally:
docker run -p 8081:8081 web-crawler-mcp - The container entrypoint binds to
0.0.0.0and will respect thePORTenvironment variable (default 8081).
Configuration
The server exposes a configurable schema through Smithery so you can tweak the user agent, HTTP timeout, and payload limits per session.