mcp-fetch

maxyychen/mcp-fetch

3.2

If you are the rightful owner of mcp-fetch and would like to certify it and/or have it hosted online, please leave a comment on the right or send an email to dayong@mcphub.com.

The MCP HTTP Fetch Server is a robust and secure server designed to fetch data from HTTP/HTTPS APIs using the Model Context Protocol (MCP) with streamable HTTP transport.

Tools
1
Resources
0
Prompts
0

MCP HTTP Fetch Server

A Model Context Protocol (MCP) server with Streamable HTTP transport for fetching data from HTTP/HTTPS APIs with comprehensive security controls.

Features

  • ✅ MCP Streamable HTTP - Full MCP specification compliance (2024-11-05)
  • ✅ JSON-RPC 2.0 - Complete JSON-RPC protocol implementation
  • ✅ Stateful Sessions - Session management with automatic cleanup
  • ✅ SSE Support - Server-Sent Events for bidirectional communication
  • ✅ SSRF Protection - Blocks private IPs, localhost, and link-local addresses
  • ✅ Rate Limiting - Per-session rate limiting to prevent abuse
  • ✅ Security First - URL validation, header sanitization, size limits
  • ✅ Docker Ready - Multi-stage build, production-optimized
  • ✅ Type Safe - Full Python type hints and Pydantic models

Quick Start

Option 1: Local Development

# Create virtual environment
python -m venv venv
source venv/bin/activate  # Windows: venv\Scripts\activate

# Install dependencies
pip install -r requirements.txt

# Run server
uvicorn src.server:app --reload --port 8080

Option 2: Docker Compose (Recommended)

# Start services
docker compose up -d

# View logs
docker compose logs -f

# Stop services
docker compose down

Option 3: Docker

# Build image
docker build -t mcp-http-fetch:latest .

# Run container
docker run -d -p 8080:8080 \
  -v $(pwd)/config:/app/config:ro \
  -v $(pwd)/logs:/app/logs \
  --name mcp-fetch \
  mcp-http-fetch:latest

Testing the Server

1. Health Check

curl http://localhost:8080/health

2. Initialize MCP Session

curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -d '{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "initialize",
    "params": {
      "protocolVersion": "2024-11-05",
      "capabilities": {},
      "clientInfo": {"name": "curl-client", "version": "1.0"}
    }
  }'

Response:

{
  "jsonrpc": "2.0",
  "id": 1,
  "result": {
    "protocolVersion": "2024-11-05",
    "capabilities": {
      "tools": {"listChanged": false},
      "logging": {}
    },
    "serverInfo": {
      "name": "mcp-http-fetch-server",
      "version": "1.0.0"
    }
  }
}

Important: Save the Mcp-Session-Id from response headers!

3. List Available Tools

curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -H "Mcp-Session-Id: YOUR_SESSION_ID_HERE" \
  -d '{
    "jsonrpc": "2.0",
    "id": 2,
    "method": "tools/list",
    "params": {}
  }'

4. Fetch HTTP API (GET Request)

curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -H "Mcp-Session-Id: YOUR_SESSION_ID_HERE" \
  -d '{
    "jsonrpc": "2.0",
    "id": 3,
    "method": "tools/call",
    "params": {
      "name": "http_fetch",
      "arguments": {
        "url": "https://api.github.com/users/octocat",
        "method": "GET",
        "headers": {
          "Accept": "application/json"
        }
      }
    }
  }'

5. Fetch HTTP API (POST Request with Body)

curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -H "Mcp-Session-Id: YOUR_SESSION_ID_HERE" \
  -d '{
    "jsonrpc": "2.0",
    "id": 4,
    "method": "tools/call",
    "params": {
      "name": "http_fetch",
      "arguments": {
        "url": "https://httpbin.org/post",
        "method": "POST",
        "headers": {
          "Content-Type": "application/json"
        },
        "body": {
          "name": "test",
          "value": 123
        }
      }
    }
  }'

6. Open SSE Stream

# Open SSE stream for server notifications
curl -N http://localhost:8080/mcp \
  -H "Mcp-Session-Id: YOUR_SESSION_ID_HERE"

HTTP Fetch Tool

Parameters

ParameterTypeRequiredDefaultDescription
urlstringYes-Target URL (HTTP/HTTPS only)
methodstringNoGETHTTP method (GET, POST, PUT, DELETE, PATCH, HEAD, OPTIONS)
headersobjectNo{}HTTP headers as key-value pairs
bodystring/objectNonullRequest body (string or JSON object)
timeoutnumberNo30Request timeout in seconds
follow_redirectsbooleanNotrueWhether to follow HTTP redirects

Response Format

{
  "status_code": 200,
  "headers": {
    "content-type": "application/json",
    "content-length": "1234"
  },
  "body": "response content",
  "content_type": "application/json",
  "is_binary": false,
  "size_bytes": 1234,
  "elapsed_ms": 123.45,
  "url": "https://final.url.after.redirects/"
}

Security Features

SSRF Protection

The server blocks requests to:

  • Private IP ranges (10.0.0.0/8, 172.16.0.0/12, 192.168.0.0/16)
  • Localhost (127.0.0.0/8)
  • Link-local addresses (169.254.0.0/16) - AWS metadata endpoint
  • Reserved and multicast IPs

URL Validation

  • Only HTTP and HTTPS schemes allowed
  • DNS resolution before request
  • Domain allowlist/blocklist support
  • Redirect validation

Rate Limiting

  • Per-session rate limiting (default: 60 requests/minute)
  • Configurable limits via environment variables
  • Token bucket algorithm

Header Sanitization

  • Dangerous headers blocked (Host, Connection, etc.)
  • Only safe headers allowed
  • Header value validation

Response Handling

  • Maximum response size (default: 10MB)
  • Binary content detection
  • Sensitive header filtering (Authorization, Cookie)

Configuration

Environment Variables

Create a .env file (see .env.example):

# Server
HOST=0.0.0.0
PORT=8080
LOG_LEVEL=INFO

# HTTP Fetch
HTTP_TIMEOUT=30
HTTP_MAX_REDIRECTS=5
HTTP_MAX_RESPONSE_SIZE=10485760
HTTP_USER_AGENT=MCP-HTTP-Fetch/1.0
HTTP_FOLLOW_REDIRECTS=true

# Security
SECURITY_BLOCK_PRIVATE_IPS=true
ALLOWED_DOMAINS=api.github.com,httpbin.org
BLOCKED_DOMAINS=malicious.com
RATE_LIMIT_PER_MINUTE=60

# Session
SESSION_TIMEOUT_MINUTES=30

YAML Configuration

Edit config/config.yaml:

server:
  host: "0.0.0.0"
  port: 8080
  log_level: "INFO"

http_fetch:
  timeout: 30
  max_redirects: 5
  max_response_size: 10485760

security:
  block_private_ips: true
  allowed_domains: []
  blocked_domains: []
  rate_limit_per_minute: 60

Project Structure

mcp-fetch/
├── src/
│   ├── server.py              # FastAPI application
│   ├── mcp_handler.py         # Tool registration and execution
│   ├── mcp_transport.py       # MCP Streamable HTTP transport
│   ├── mcp_session.py         # Session management
│   ├── jsonrpc/
│   │   ├── models.py          # JSON-RPC models
│   │   └── handler.py         # JSON-RPC handler
│   ├── tools/
│   │   └── http_fetch.py      # HTTP fetch tool
│   └── utils/
│       ├── errors.py          # Custom exceptions
│       ├── validation.py      # Input validation
│       └── security.py        # SSRF protection
├── config/
│   └── config.yaml            # Configuration
├── tests/                     # Test suite (future)
├── logs/                      # Application logs
├── Dockerfile
├── docker-compose.yml
├── requirements.txt
└── README.md

API Endpoints

MethodEndpointDescription
POST/mcpMCP requests with session management
GET/mcpOpen SSE stream for server notifications
GET/healthHealth check
GET/Service information

MCP Protocol Flow

1. Client → POST /mcp (initialize)
   Server → Returns session ID in Mcp-Session-Id header

2. Client → GET /mcp (with session ID)
   Server → Opens SSE stream

3. Client → POST /mcp (tools/list, with session ID)
   Server → Returns available tools

4. Client → POST /mcp (tools/call http_fetch, with session ID)
   Server → Executes HTTP request, returns result

5. Client → POST /mcp (ping, with session ID)
   Server → Keepalive response

Development

Install Development Dependencies

pip install -r requirements.txt

Run Tests

pytest tests/ -v

Code Formatting

black src/
ruff check src/
mypy src/

Example Use Cases

1. Fetch JSON API

curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -H "Mcp-Session-Id: YOUR_SESSION_ID" \
  -d '{
    "jsonrpc": "2.0",
    "id": 1,
    "method": "tools/call",
    "params": {
      "name": "http_fetch",
      "arguments": {
        "url": "https://api.github.com/repos/python/cpython",
        "method": "GET"
      }
    }
  }'

2. POST JSON Data

curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -H "Mcp-Session-Id: YOUR_SESSION_ID" \
  -d '{
    "jsonrpc": "2.0",
    "id": 2,
    "method": "tools/call",
    "params": {
      "name": "http_fetch",
      "arguments": {
        "url": "https://httpbin.org/post",
        "method": "POST",
        "body": {"key": "value"}
      }
    }
  }'

3. Custom Headers

curl -X POST http://localhost:8080/mcp \
  -H "Content-Type: application/json" \
  -H "Mcp-Session-Id: YOUR_SESSION_ID" \
  -d '{
    "jsonrpc": "2.0",
    "id": 3,
    "method": "tools/call",
    "params": {
      "name": "http_fetch",
      "arguments": {
        "url": "https://api.github.com/user",
        "method": "GET",
        "headers": {
          "Authorization": "Bearer YOUR_TOKEN",
          "Accept": "application/vnd.github.v3+json"
        }
      }
    }
  }'

Error Handling

The server returns standard JSON-RPC error responses:

{
  "jsonrpc": "2.0",
  "id": 1,
  "error": {
    "code": -32004,
    "message": "Cannot fetch from private IP: example.local resolves to 192.168.1.1",
    "data": null
  }
}

Error Codes

CodeDescription
-32700Parse error
-32600Invalid request
-32601Method not found
-32602Invalid params
-32603Internal error
-32001Tool not found
-32002Tool execution error
-32003HTTP fetch error
-32004Security error
-32005Rate limit error

Troubleshooting

SSRF Protection Blocking Valid Requests

If you need to fetch from internal services (e.g., in development):

# Disable SSRF protection (NOT recommended for production)
export SECURITY_BLOCK_PRIVATE_IPS=false

Rate Limit Issues

Increase rate limit:

export RATE_LIMIT_PER_MINUTE=120

Connection Issues

Check logs:

# Docker
docker compose logs -f

# Local
tail -f logs/server.log

License

MIT License

Contributing

  1. Fork the repository
  2. Create a feature branch
  3. Commit your changes
  4. Push to the branch
  5. Create a Pull Request

References

Support

For issues and questions, please open an issue on GitHub.