Cogni-DAO/heygen-mcp

Runway MCP Server

A Model Context Protocol (MCP) server for integrating with Runway AI's API to generate images and videos using their Gen4 models.

Features

Image Generation

• Basic Text-to-Image: Generate images from text prompts using Gen4 Image
• Reference-based Image Generation: Generate images with reference images and @ syntax for precise control

Video Generation

• Text-to-Video: Generate videos from text descriptions using Gen4 Turbo
• Image-to-Video: Animate existing images into videos
• Video-to-Video: Transform existing videos with reference images (supports "restyled first frame" workflows)
• Video Upscaling: Enhance video quality and resolution

Task Management

• Task Status Monitoring: Check the progress of generation tasks
• Task Listing: View recent generation history
• Task Cancellation: Stop running tasks when needed

Setup

1. Install Dependencies

   npm install
   

2. Get Runway API Key

   • Sign up at Runway Developer Portal
   • Create an API key from your dashboard

3. Configure Environment Create a .env file in the project root:

   RUNWAY_API_KEY=your_runway_api_key_here
   

4. Test the Server

   # List available tools
   npm run list-tools
   
   # Run in stdio mode (for MCP clients)
   node mcpServer.js
   
   # Run in SSE mode (for web interfaces)
   node mcpServer.js --sse
   

Usage Examples

Generate Image from Text

{
  "name": "GenerateImage",
  "arguments": {
    "promptText": "A beautiful sunset over a calm ocean with vibrant colors",
    "ratio": "1920:1080"
  }
}

Generate Image with References

{
  "name": "GenerateImageWithReferences",
  "arguments": {
    "promptText": "@EiffelTower painted in the style of @StarryNight",
    "referenceImages": [
      {
        "uri": "https://example.com/eiffel-tower.jpg",
        "tag": "EiffelTower"
      },
      {
        "uri": "https://example.com/starry-night.jpg", 
        "tag": "StarryNight"
      }
    ],
    "ratio": "1920:1080"
  }
}

Generate Video from Image

{
  "name": "GenerateVideoFromImage",
  "arguments": {
    "promptImage": "https://example.com/my-image.jpg",
    "promptText": "The waves gently move and the clouds drift slowly",
    "duration": 5,
    "ratio": "1280:720"
  }
}

Generate Video from Video (Restyled First Frame)

{
  "name": "GenerateVideoFromVideo",
  "arguments": {
    "promptVideo": "https://example.com/original-video.mp4",
    "promptText": "Transform the first frame to look like @styledFrame while maintaining the original motion",
    "referenceImages": [
      {
        "uri": "https://example.com/restyled-first-frame.jpg",
        "tag": "styledFrame"
      }
    ],
    "duration": 5,
    "ratio": "1280:720"
  }
}

Check Task Status

{
  "name": "GetTaskStatus",
  "arguments": {
    "taskId": "your-task-id-here"
  }
}

Available Tools

Supported Models

• gen4_image: Advanced image generation with reference support
• gen4_turbo: Fast video generation from text or images
• upscale_video: Video quality enhancement

Supported Formats

Images

• JPEG, PNG, WebP
• URLs or base64 data URIs
• Max size: 10MB per image

Videos

• MP4, MOV, WebM
• URLs or base64 data URIs
• Max size: 100MB per video

Error Handling

The server includes comprehensive error handling for:

• Invalid API keys
• Network connectivity issues
• Task failures and timeouts
• Invalid input parameters
• Rate limiting

Development Notes

API Integration

• Uses official @runwayml/sdk v2.4.0
• Implements async task polling with proper error handling
• Supports both URL and data URI inputs for media files

Contributing

1. Fork the repository
2. Create a feature branch
3. Make your changes
4. Test with npm run list-tools
5. Submit a pull request

License

MIT License - see LICENSE file for details.

Support

For issues with this MCP server, please create an issue on GitHub. For Runway API questions, visit the Runway Developer Documentation.