robkisk/dbx_mcp_server_demo

Databricks MCP Server

A production-ready Model Context Protocol (MCP) server for seamless Databricks integration. This server provides comprehensive tools for SQL execution, job management, and Delta Live Table pipeline operations, designed for easy deployment on Databricks Apps.

🚀 Features

SQL Operations

• execute_sql: Execute SQL queries against Databricks warehouses
• get_workspace_info: Retrieve current workspace and user information

Job Management

• list_jobs: List all Databricks jobs in the workspace
• get_job_status: Get detailed job status and recent run history
• run_job: Trigger job execution

Pipeline Management

• list_pipelines: List all Delta Live Table pipelines
• get_pipeline_status: Get pipeline status and recent update history
• start_pipeline: Start pipeline updates

Resources

• databricks://jobs/{job_id}: Access detailed job information as MCP resources

📁 Project Structure

databricks-mcp-server/
├── src/
│   └── mcp_server/
│       ├── __init__.py
│       ├── main.py           # Entry point for uvicorn
│       ├── app.py            # FastAPI + FastMCP server
│       ├── config.py         # Configuration management
│       ├── exceptions.py     # Custom exceptions
│       └── static/
│           └── index.html    # Web documentation
├── app.yaml                  # Databricks Apps configuration
├── pyproject.toml           # Project configuration
└── README.md               # This file

🛠️ Prerequisites

1. Python 3.11+: Required for this project
2. UV Package Manager: Install from https://docs.astral.sh/uv/getting-started/installation/
3. Databricks CLI: Install from https://docs.databricks.com/dev-tools/cli/install.html
4. Databricks Workspace Access: With appropriate permissions for resources you want to manage

📦 Local Development Setup

1. Install Dependencies

# Clone or navigate to project directory
cd databricks-mcp-server

# Install all dependencies
uv sync

2. Configure Environment

Create a .env file in the project root:

# Required environment variables
DATABRICKS_HOST=https://your-workspace.cloud.databricks.com
DATABRICKS_TOKEN=your_personal_access_token
DATABRICKS_WAREHOUSE_ID=your_warehouse_id  # Optional, can be provided per query

3. Local Testing

Test the server locally:

# Run the MCP server locally
uv run mcp-server

The server will start on http://localhost:8000. You can visit this URL to see the documentation page.

Test with MCP Client:

Add to your Claude Code configuration (~/Library/Application Support/Claude Code/config.json on macOS):

{
  "mcpServers": {
    "databricks-local": {
      "command": "uv",
      "args": ["run", "mcp-server"],
      "cwd": "/absolute/path/to/your/databricks-mcp-server"
    }
  }
}

🚀 Databricks Apps Deployment

Step 1: Authenticate with Databricks

# Set up authentication profile
export DATABRICKS_CONFIG_PROFILE=your-profile-name
databricks auth login --profile "$DATABRICKS_CONFIG_PROFILE"

# Verify authentication
databricks current-user me

Step 2: Create Databricks App

# Create a new Databricks App
databricks apps create databricks-mcp-server

Step 3: Sync Code to Workspace

# Get your username
DATABRICKS_USERNAME=$(databricks current-user me | jq -r .userName)

# Sync project files to workspace
databricks sync . "/Users/$DATABRICKS_USERNAME/databricks-mcp-server"

Step 4: Deploy the App

# Deploy the app
databricks apps deploy databricks-mcp-server \
  --source-code-path "/Workspace/Users/$DATABRICKS_USERNAME/databricks-mcp-server"

Step 5: Get App URL

# List apps to get the URL
databricks apps list | grep databricks-mcp-server

The output will show your app URL, typically in the format:

https://your-app-name.apps.your-workspace.databricksapps.com

🔗 Connecting to Deployed Server

For Claude Code

Add the deployed server to your MCP configuration:

{
  "mcpServers": {
    "databricks-deployed": {
      "transport": "streamableHttp",
      "url": "https://your-app-url.databricksapps.com/",
      "headers": {
        "Authorization": "Bearer your-databricks-token"
      }
    }
  }
}

For Cursor

Add to your Cursor settings:

{
  "mcpServers": {
    "databricks-deployed": {
      "transport": "streamableHttp",
      "url": "https://your-app-url.databricksapps.com/",
      "headers": {
        "Authorization": "Bearer your-databricks-token"
      }
    }
  }
}

🧪 Testing Your Deployment

1. Test the Web Interface

Visit your app URL in a browser to see the documentation page and verify the server is running.

2. Test MCP Functionality

Use your AI client (Claude Code, Cursor) to test the tools:

# Example prompts to test
"List all jobs in my Databricks workspace"
"Execute this SQL query: SELECT 1 as test"
"Show me information about job ID 123"
"List all Delta Live Table pipelines"

⚙️ Configuration Options

Environment Variables

Server Configuration

The server runs on port 8000 by default. You can modify the configuration in src/mcp_server/main.py:

def main():
    uvicorn.run(
        "mcp_server.app:app",
        host="0.0.0.0",
        port=8000,
        reload=True,  # Set to False for production
    )

🔍 Troubleshooting

Common Issues

1. Authentication Errors

• Verify your DATABRICKS_HOST and DATABRICKS_TOKEN are correct
• Ensure your token has appropriate permissions
• Check that your workspace URL format is correct

2. Deployment Issues

• Ensure you have the correct Databricks CLI version
• Verify your authentication profile is active
• Check that the source code path exists in the workspace

3. Connection Issues

• Verify the app URL is correct and accessible
• Check that the MCP endpoint ends with /
• Ensure authentication headers are properly configured

Debugging Commands

# Check app status
databricks apps list

# View app logs (if available)
databricks apps logs databricks-mcp-server

# Test endpoint directly
curl -H "Authorization: Bearer your-token" \
  https://your-app-url.databricksapps.com/

# Verify local environment
uv run python -c "from mcp_server.config import DatabricksConfig; print(DatabricksConfig.from_environment().is_configured())"

🔧 Development

Adding New Tools

1. Add your tool function to src/mcp_server/app.py:

@mcp.tool()
def your_new_tool(param: str) -> Dict[str, Any]:
    """Description of your tool"""
    # Your implementation here
    return {"status": "success", "result": "your_result"}

2. Update the documentation in src/mcp_server/static/index.html

3. Test locally, then redeploy:

# Test locally
uv run mcp-server

# Redeploy to Databricks Apps
databricks sync . "/Users/$DATABRICKS_USERNAME/databricks-mcp-server"
databricks apps deploy databricks-mcp-server \
  --source-code-path "/Workspace/Users/$DATABRICKS_USERNAME/databricks-mcp-server"

Project Scripts

Available scripts defined in pyproject.toml:

# Run MCP server
uv run mcp-server

# Other available scripts
uv run add-asset      # Add project assets
uv run test           # Run tests
uv run validate-setup # Validate environment setup

📚 Additional Resources

• Databricks MCP Documentation
• Model Context Protocol Specification
• Databricks Apps Documentation
• FastMCP Framework

🤝 Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Add tests if applicable
5. Submit a pull request

📄 License

This project is licensed under the MIT License - see the LICENSE file for details.

Note: This server demonstrates how to build production-ready MCP servers for Databricks Apps. Customize the tools and configuration based on your specific use case and security requirements.