damameet14/web-research-mcp-server

Web Research MCP Server

A completely self-contained Model Context Protocol (MCP) server for web research, designed for integration with AI assistants like Cline, Cursor, and Claude.

Overview

This MCP server enables AI assistants to perform real-time web research without relying on third-party APIs or paid services. It includes a fully integrated Searxng search engine that is automatically set up and managed, providing completely free and private web search capabilities directly from your development environment.

Features

• Private Web Search: Integrated Searxng instance for completely private searching
• Content Extraction: Visits web pages and extracts clean, readable content
• Visual Documentation: Takes screenshots of web pages for visual reference
• Research Sessions: Tracks search queries, visited pages, and extracted content
• Agentic Research: Guided research prompt for deep topic exploration
• No API Keys Required: Completely self-hosted solution
• Containerized Deployment: Ready for Docker deployment
• Automatic Setup: One-command setup and start

Prerequisites

• Node.js >= 18
• Docker (for containerized deployment)
• Docker Compose (for containerized deployment)
• AI Assistant with MCP support (Cline, Cursor, Claude Desktop, etc.)

Quick Start (Containerized - Recommended)

1. Clone the Repository:

   git clone https://github.com/damameet14/web-research-mcp-server.git
   cd web-research-mcp-server
   

2. Start with Docker Compose:

   docker-compose up
   

3. Configure Your AI Assistant:

For Cline/Cursor, add this to your MCP settings:

{
  "mcpServers": {
    "webresearch": {
      "command": "docker",
      "args": ["run", "--rm", "-i", "--network", "host", "mcp-webresearch"]
    }
  }
}

Manual Installation (Without Docker)

1. Clone the Repository:

   git clone https://github.com/damameet14/web-research-mcp-server.git
   cd web-research-mcp-server
   

2. Install Dependencies:

   npm install
   

3. Setup and Start Everything:

   npm run start-all
   

Usage

Once configured, your AI assistant can use the following tools:

Tools

1. search_web

   • Performs web searches using the integrated Searxng instance
   • Arguments: { query: string }

2. visit_page

   • Visits a webpage and extracts its content in clean Markdown format
   • Arguments: { url: string, takeScreenshot?: boolean }

3. take_screenshot

   • Takes a screenshot of the current page
   • No arguments required

Prompts

agentic-research

A guided research prompt that helps your AI assistant conduct thorough web research. Simply ask your assistant to "perform agentic research on [topic]" to start a guided research session.

How It Works

1. Setup: Automatically clones and configures Searxng on first run
2. Search: Uses the integrated Searxng instance to perform web searches
3. Extract: Visits web pages and converts content to clean, readable Markdown
4. Document: Optionally captures screenshots for visual reference
5. Track: Maintains a research session with all queries, pages, and content
6. Cleanup: Automatically stops all services when the session ends

Benefits

• Completely Free: No API keys or paid services required
• Privacy Focused: All search and browsing happens locally
• Self-Contained: Everything included and automatically managed
• Integrated: Seamlessly works with popular AI development tools
• Extensible: Easy to modify and extend with additional features
• Containerized: Ready for production deployment with Docker

Project Structure

├── index.js              # Main MCP server implementation
├── setup-searxng.sh      # Script to setup Searxng
├── start-searxng.sh      # Script to start Searxng
├── stop-searxng.sh       # Script to stop Searxng
├── start.sh              # Main start script
├── Dockerfile            # Docker configuration
├── docker-compose.yml    # Multi-container configuration
└── searxng-config/       # Searxng configuration files

Requirements

• Node.js >= 18
• Playwright (automatically installed as a dependency)
• Python 3 (for Searxng)
• Git (for cloning Searxng)

License

MIT

Created and maintained by damameet14