Splunk-MCP-Server

Abhishek-Aditya-bs/Splunk-MCP-Server

3.2

If you are the rightful owner of Splunk-MCP-Server and would like to certify it and/or have it hosted online, please leave a comment on the right or send an email to henry@mcphub.com.

The Splunk MCP Server is a secure and efficient solution for executing Splunk queries with environment-specific configurations and robust credential management.

Tools
5
Resources
0
Prompts
0

Splunk MCP Server

A secure Model Context Protocol (MCP) server for executing Splunk queries with environment-specific index mappings and enterprise-grade credential management.

Features

  • Secure Credential Management: Machine-specific encryption ensures passwords can only be decrypted on the authorized machine
  • Environment-Specific Indexes: UAT/PROD environments with different index mappings
  • AI-Optimized Output: JSON responses formatted for optimal consumption by MCP clients and AI assistants
  • Pagination Support: Handle large result sets with automatic pagination guidance
  • Automatic Retry Logic: Built-in connection retry with exponential backoff
  • Dynamic Discovery: Automatically discover available sourcetypes and indexes

Installation

Prerequisites

  • Python 3.8 or higher
  • Access to Splunk Enterprise (UAT and/or PROD)
  • Valid Splunk credentials

Setup Steps

  1. Clone the repository

    git clone <repository-url>

  2. Install dependencies

    pip install -r requirements.txt

  3. Update configuration file

    Edit config.yml with your actual Splunk server hostname:

    • Replace splunk.yourcompany.com with your actual Splunk server
    • Update index mappings for UAT and PROD environments

  4. Encrypt your passwords

    python encrypt_password.py

    Follow the prompts to:

    • Enter your Splunk username
    • Enter your Splunk password
    • Copy the encrypted output to your config.yml

  5. Test the connection

    python tests/test_connection.py

    This will verify:

    • Configuration is loaded correctly
    • Credentials can be decrypted
    • Connection to Splunk is successful
    • Test query executes properly

Configuration

Configuration Structure (config.yml)

splunk:
  host: "splunk.yourcompany.com"  # Your Splunk server
  port: 8089
  username: "your_username"  # Your Splunk username
  password_encrypted: "xxx"  # From encrypt_password.py
  password_salt: "xxx"       # From encrypt_password.py
  machine_hash: "xxx"        # From encrypt_password.py

indexes:
  uat: "your_app_uat_index"   # UAT environment index
  prod: "your_app_prod_index"  # PROD environment index

query_settings:
  default_earliest_time: "-30d"  # Default: last 30 days
  default_latest_time: "now"
  max_results: 10000
  page_size: 1000

Usage

Starting the Server

python splunk_mcp.py

Available Tools

1. get_index_for_environment

Get the index name for UAT or PROD environment. Use this first to determine which index to use.

Parameters:

  • environment: "uat" or "prod"

Example:

{
  "tool": "get_index_for_environment",
  "arguments": {
    "environment": "prod"
  }
}

Returns: {"index": "your_app_prod_index"}

2. check_connection

Verify connectivity to Splunk server.

Parameters: None

Example:

{
  "tool": "check_connection",
  "arguments": {}
}
3. execute_query

Execute SPL queries. The 'search' command is automatically prepended when needed, so you can start queries directly with 'index='. Use the index from get_index_for_environment. To filter by sourcetype, first use get_sourcetypes to discover available sourcetypes.

Parameters:

  • query: SPL query string (can start with 'index=' directly)
  • earliest_time (optional): Start time (default: "-30d")
  • latest_time (optional): End time (default: "now")
  • max_results (optional): Maximum results

Examples:

// Simple search - 'search' is auto-prepended
{
  "tool": "execute_query",
  "arguments": {
    "query": "index=main \"Error message\"",
    "earliest_time": "-2d"
  }
}

// With sourcetype filter
{
  "tool": "execute_query",
  "arguments": {
    "query": "index=main sourcetype=application_logs status=error",
    "earliest_time": "-7d"
  }
}

// Pipe command (no 'search' needed)
{
  "tool": "execute_query",
  "arguments": {
    "query": "| metadata type=sourcetypes index=main"
  }
}
4. get_available_indexes

List all available indexes in Splunk.

Parameters: None

5. get_sourcetypes

List available sourcetypes from Splunk, optionally filtered by index. Use this to discover what sourcetypes are available, then include sourcetype=your_sourcetype in your queries.

Parameters:

  • index (optional): Filter by specific index

Example:

{
  "tool": "get_sourcetypes",
  "arguments": {
    "index": "main"
  }
}

Workflow Example

Finding Failed Transactions in Production

  1. Get PROD index:

    Tool: get_index_for_environment
Input: {"environment": "prod"}
Output: {"index": "your_app_prod_index"}

  2. Get available sourcetypes (optional):

    Tool: get_sourcetypes
Input: {"index": "your_app_prod_index"}
Output: {"sourcetypes": ["application_logs", "system_logs", ...]}

  3. Execute search query:

    Tool: execute_query
Input: {
  "query": "index=your_app_prod_index sourcetype=application_logs transaction_id=TXN123 | head 100",
  "earliest_time": "-2d"
}

    Note: The 'search' command is automatically prepended when needed

  4. AI analyzes the JSON response to identify issues

Cross-Platform Support

The password encryption utility works across all major operating systems:

  • Windows: Uses USERNAME environment variable and Windows paths
  • macOS: Uses USER environment variable and Unix paths
  • Linux: Uses USER environment variable and Unix paths

The machine identifier combines:

  • MAC address (network hardware ID)
  • Username (OS-agnostic)
  • Home directory path (cross-platform)
  • Platform info (OS and architecture)

Example SPL Queries

Basic Searches

# Find all errors in the last 24 hours
index=main error

# Count events by host
index=main | stats count by host

# Search for specific error patterns
index=app_logs "NullPointerException" | head 100

# Time-based analysis
index=main | timechart span=1h count by status

Application Monitoring

# Find slow API responses
index=app_logs response_time>1000 
| stats avg(response_time) as avg_time, max(response_time) as max_time by endpoint

# Error rate by service
index=app_logs 
| stats count(eval(status>=400)) as errors, count as total by service 
| eval error_rate=round((errors/total)*100, 2)

# Find failed login attempts
index=security action=login result=failure 
| stats count by user, src_ip 
| sort -count

Log Analysis

# Extract and analyze log levels
index=app_logs 
| rex field=_raw "(?<log_level>ERROR|WARN|INFO|DEBUG)" 
| stats count by log_level

# Find unique error messages
index=app_logs ERROR 
| rex field=_raw "ERROR.*?(?<error_message>[^\\n]+)" 
| stats count by error_message 
| sort -count

# Trace specific request IDs
index=app_logs request_id="abc-123-def" 
| sort _time

Performance Monitoring

# Database query performance
index=db_logs 
| stats avg(query_time) as avg_time, p95(query_time) as p95_time by query_type

# Memory usage trends
index=metrics metric_name=memory_usage 
| timechart span=5m avg(value) as avg_memory by host

# CPU spikes
index=metrics metric_name=cpu_usage value>80 
| table _time, host, value

Handling Large Result Sets

When queries return more than the configured page_size (default: 1000), the server provides:

  1. Preview of first 100 results
  2. Pagination metadata showing total pages needed
  3. Guidance on refining queries

Best Practices for Large Datasets

  1. Use time constraints

    index=main earliest=-1h latest=now

  2. Add specific filters

    index=main host=prod-server-01 status=error

  3. Use statistical commands

    index=main | stats count by field

  4. Limit results explicitly

    index=main | head 1000

  5. Use summary indexes for historical data

    index=summary_daily

Security Considerations

  1. Password Encryption: Passwords are encrypted using machine-specific identifiers (MAC address, username, home directory)
  2. No Plain Text Storage: Never store plain text passwords in configuration
  3. Machine Binding: Encrypted passwords only work on the machine where they were encrypted
  4. Secure Your Config: Add config.yaml to .gitignore and never commit it

Session Management

Connection Lifetime

  • Initial Authentication: Uses username/password from config
  • Connection Reuse: Connections are reused for up to 1 hour
  • Automatic Refresh: After 1 hour, a new connection is automatically created
  • Session Testing: Each query tests the connection and reconnects if needed
  • No Token Expiry: Using username/password auth, there's no token to expire

Query Guidelines for MCP Clients

When using with MCP clients and AI assistants:

  1. No Need to Add 'search' Prefix:

    # The server automatically adds 'search' when needed
index=main "Error message"     # Works correctly
search index=main "Error"      # Also works
| stats count                   # Pipe commands work directly

  2. Quotes in Queries Are Handled Automatically:

    # Complex queries with quotes are automatically fixed
index=main "Database connection failed with error code '1045' ('Access denied')"

  3. Use Proper Field Searches for Complex Strings:

    # For exact phrase matching:
index=main "ERROR com.example.app.services.UserService"

# For field-based search (recommended for class names):
index=main logger="com.example.app.services.UserService" level=ERROR

  4. Count Occurrences and Statistics:

    index=main "Error while loading configuration" 
| stats count

# Find last occurrence
index=main "error message" | sort - _time | head 1

Troubleshooting

Common Issues

  1. "Cannot decrypt password: Different machine"

    • Re-run encrypt_password.py on the current machine
    • Update config.yaml with new encrypted values

  2. Connection timeouts

    • Check network connectivity to Splunk servers
    • Verify firewall rules allow port 8089
    • Increase timeout in configuration

  3. Authentication failures

    • Verify username is correct
    • Re-encrypt password using encrypt_password.py
    • Check account permissions in Splunk

  4. Query timeouts

    • Reduce time range
    • Add more specific filters
    • Increase max_execution_time in config

Debug Mode

Enable debug logging in config.yaml:

logging:
  level: "DEBUG"
  log_file: "splunk_mcp.log"

MCP Client Integration

Configuration for MCP Clients

To integrate this server with any MCP client, add the following configuration to your MCP client settings:

{
  "mcpServers": {
    "splunk": {
      "command": "python",
      "args": ["/path/to/your/splunk_mcp.py"],
      "transport": "stdio"
    }
  }
}

Replace /path/to/your/splunk_mcp.py with the absolute path to the splunk_mcp.py file.

Features for MCP Clients

This MCP server is optimized for use with MCP clients and AI assistants:

  1. Structured JSON responses for easy parsing
  2. Field summaries to understand data structure
  3. Event summaries with top values and distributions
  4. Clear error messages with troubleshooting tips
  5. Pagination guidance for large result sets

Contributing

Feel free to submit issues or pull requests to improve the server.

License

This project is for internal enterprise use. Please follow your organization's guidelines for code sharing and distribution.

Support

For issues or questions:

  1. Check the troubleshooting section
  2. Review example queries
  3. Verify configuration settings
  4. Enable debug logging for detailed information

Note: This server is designed for enterprise Splunk deployments with username/password authentication. Token-based authentication can be added if your organization enables it.