cordlesssteve/token-analyzer-mcp

Token Analyzer MCP

A tool for independent verification of token counts and context volume in Model Context Protocol (MCP) server configurations. Provides accurate token consumption analysis to validate and optimize your Claude Code setup.

Purpose

This tool serves as an independent verification system for token usage in MCP configurations. Rather than relying on estimates or built-in metrics, it provides precise token counting and context volume analysis to:

• Verify actual token consumption vs. expected usage
• Validate context window utilization against the 200k token limit
• Identify servers consuming excessive tokens
• Provide independent confirmation of token optimization efforts

Features

• Independent Token Verification - Accurate token counting separate from MCP server self-reporting
• Context Volume Analysis - Precise measurement of total context consumption
• MCP Configuration Scanning - Automatic discovery and analysis of configured servers
• Overhead Calculation - Breakdown of token usage by server and tool definitions
• Verification Reports - Detailed analysis for validating token usage claims
• Quick Estimation Mode - Fast overhead checking without full server connections

Installation

npm install -g token-analyzer-mcp

Usage

Basic Verification

# Perform independent token count verification
token-analyzer-mcp analyze

# Quick context overhead verification
token-analyzer-mcp quick

# Validate MCP configuration setup
token-analyzer-mcp config

# Verify analyzer dependencies
token-analyzer-mcp doctor

Verification Reports

# Generate detailed verification report
token-analyzer-mcp analyze \
  --output verification.json \
  --report token-analysis.txt \
  --csv usage-breakdown.csv

Configuration Discovery

The analyzer automatically discovers MCP configurations from standard locations:

• ~/.claude/claude_desktop_config.json
• ~/.config/claude-desktop/claude_desktop_config.json
• Windows: %APPDATA%/Claude/claude_desktop_config.json
• macOS: ~/Library/Application Support/Claude/claude_desktop_config.json

Supported Configuration Format

{
  "mcpServers": {
    "server-name": {
      "command": "node",
      "args": ["path/to/server.js"],
      "disabled": false
    }
  }
}

Sample Verification Output

Token Analyzer - Independent Verification v1.0.0
Verifying MCP server token consumption...

Phase 1: Configuration Discovery
✓ Found 6 servers in ~/.claude/claude_desktop_config.json
✓ 4 active servers to verify

Phase 2: Schema Extraction  
✓ document-organizer: 12 tools extracted
✓ conversation-search: 15 tools extracted
✓ claude-telemetry: 8 tools extracted

Phase 3: Independent Token Verification
✓ Token verification complete

Verification Results:
Built-in Tools:     2,250 tokens (1.13%)
MCP Servers:        8,540 tokens (4.27%)
Total Overhead:    10,790 tokens (5.40%)
Available Context: 189,210 tokens (94.60%)

Status: ✅ Context usage within acceptable limits

Commands

• analyze - Full independent verification with detailed breakdown
• quick - Fast context overhead verification without server connections
• config - Validate MCP configuration accessibility and format
• doctor - Verify analyzer dependencies and permissions

Verification Accuracy

• Connection Timeout: 10 seconds per server to ensure accurate results
• Retry Logic: Up to 2 retry attempts for reliable token counting
• Independent Counting: Separate token calculation engine from MCP servers
• Cross-Validation: Multiple counting methods for accuracy verification

Troubleshooting

Configuration Not Found

# Verify configuration locations
token-analyzer-mcp doctor

# Check file permissions and accessibility
ls -la ~/.claude/claude_desktop_config.json

Server Connection Issues

# Test individual server accessibility
node /path/to/server/index.js

# Run verification with debug output
token-analyzer-mcp analyze --debug

Token Count Discrepancies

# Generate detailed verification report
token-analyzer-mcp analyze --summary

# Compare with previous verification results
token-analyzer-mcp analyze --report comparison.txt

Development

# Clone repository
git clone https://github.com/cordlesssteve/token-analyzer-mcp.git
cd token-analyzer-mcp

# Install dependencies
npm install

# Run tests
npm test

# Run analyzer locally
node src/index.js analyze

API Reference

TokenVerificationEngine

• performIndependentVerification() - Execute comprehensive token verification
• verifyServerTokens(servers) - Independently verify token consumption
• validateContextUsage(tokens) - Check context volume against limits
• generateVerificationReport(analysis) - Create verification documentation

ConfigurationValidator

• discoverConfigurations() - Find and validate MCP configuration files
• verifyServerAccessibility(server) - Test server connection and response
• validateConfigurationFormat(config) - Check configuration file structure
• calculateBaselineOverhead() - Measure built-in tool token consumption

Contributing

1. Fork the repository
2. Create a feature branch: git checkout -b feature-name
3. Add verification improvements and tests
4. Run the test suite: npm test
5. Submit a pull request

License

MIT License - see file for details.

Support

• Issues
• Discussions
• Documentation