mcpnow-io/mcp-host-evals

3.2

If you are the rightful owner of mcp-host-evals and would like to certify it and/or have it hosted online, please leave a comment on the right or send an email to henry@mcphub.com.

MCP Host Evals is a server designed to evaluate a host's support for the MCP protocol specification.

MCP Host Evals (WIP)

A MCP Server that helps to evaluate host’s support for the MCP protocol specification.

Get Start

install

pnpm i mcp-host-evals -g

Streamable HTTP Server

start http server:

mcp-host-evals http

config file:

{
  "servers": {
    "mcp-host-evals": {
      "url": "http://localhost:3001/mcp",
      "type": "http"
    }
  },
  "inputs": []
}

Stdio

{
  "servers": {
    "mcp-host-evals": {
      "command": "mcp-host-evals",
      "args": ["stdio"]
    }
  }
}

Running

If you see the following output, it means you have successfully started and configured the mcp-host-evals mcp:

Usage

To have your LLM start evaluating the current host, please follow these steps:

Enter the following instruction to your LLM:
```
help me to evals this host
```
The LLM will guide you through the evaluation process step by step. For each step, follow the LLM's instructions or reply as prompted until the evaluation is complete.
Example result: