web-content-mcp
If you are the rightful owner of web-content-mcp and would like to certify it and/or have it hosted online, please leave a comment on the right or send an email to henry@mcphub.com.
Web Content MCP is a server designed to efficiently retrieve web content through search-based and direct URL access.
Web Content MCP
This MCP server is called web-content-mcp and provides two tools:
web-search: Searches Google to get the top-most results, and scrapes data from those pages by visiting them firsturl: Directly visits the provided URL(s) and scrapes content without searching Google
Features
- Fast and efficient web content retrieval
- Support for both search-based and direct URL access
- Compatible with Claude Desktop, VSCode, and any agent that supports MCP servers
Prerequisites
- Node.js and pnpm/npm installed
- TypeScript support
- A
.envfile with required API keys (see Configuration section)
Installation
- Clone this repository
git clone https://github.com/yourusername/web-content-mcp.git
cd web-content-mcp
- Install dependencies
pnpm install
- Configure your
.envfile (see Configuration section)
Configuration
Create a .env file in the root directory with the following environment variables:
GOOGLE_API_KEY=your_google_api_key
GOOGLE_SEARCH_ENGINE_ID=your_google_search_engine_id
Setup in Claude Desktop
Add the following configuration to your Claude Desktop configuration file:
{
"mcpServers": {
"web-content-mcp": {
"command": "pnpx",
"args": [
"tsx",
"--env-file=/path/to/your/.env",
"/path/to/your/web-content-mcp/src/index.ts"
]
}
}
}
For Windows users, use something like:
{
"mcpServers": {
"web-content-mcp": {
"command": "pnpx",
"args": [
"tsx",
"--env-file=C:\\Users\\username\\Projects\\web-content-mcp\\.env",
"C:\\Users\\username\\Projects\\web-content-mcp\\src\\index.ts"
]
}
}
}
Setup in VSCode
Add the following to your VSCode's settings.json (User Settings preferred):
{
"mcp": {
"servers": {
"web-content-mcp": {
"type": "stdio",
"command": "pnpx",
"args": [
"tsx",
"--env-file=/path/to/your/.env",
"/path/to/your/web-content-mcp/src/index.ts"
]
}
}
}
}
For Windows users, make sure to escape backslashes:
{
"mcp": {
"servers": {
"web-content-mcp": {
"type": "stdio",
"command": "pnpx",
"args": [
"tsx",
"--env-file=C:\\\\Users\\\\username\\\\Projects\\\\web-content-mcp\\\\.env",
"C:\\\\Users\\\\username\\\\Projects\\\\web-content-mcp\\\\src\\\\index.ts"
]
}
}
}
}
Usage
Once configured, the MCP server can be accessed in Claude prompts:
Using web-search
Search for information about [search query]
Using direct URL access
Visit and summarize information from [URL]
Troubleshooting
- Connection errors: Make sure the server is running and the path to your files is correct
- API errors: Verify that your API keys in the
.envfile are valid - Path issues: Double-check the paths in your configuration files
Contributing
Contributions are welcome! Please feel free to submit a Pull Request.
