elevenlabs-agents by taskcrew - MCP Server

ElevenLabs Agents Integration

A complete toolkit for integrating ElevenLabs Conversational AI agents with Claude Code and other AI assistants. This repository contains both a Claude Code skill and an MCP server for seamless voice AI interactions.

Overview

This repository provides two complementary components:

Claude Code Skill (skill/) - Pre-built skill definition for Claude Code with usage examples and API reference
MCP Server (mcp-server/) - Model Context Protocol server that runs on Vercel as a serverless function

Together, these enable AI assistants to interact with ElevenLabs voice agents, initiate conversations, make outbound phone calls, and retrieve conversation transcripts.

Quick Start

For Claude Code Users

Install the skill in your Claude Code configuration:

# Copy skill to Claude Code skills directory
cp -r skill ~/.claude/skills/elevenlabs-agents

Then add the MCP server to your Claude Code MCP configuration:

{
  "mcpServers": {
    "elevenlabs-agents": {
      "url": "https://your-app.vercel.app/mcp",
      "transport": "http"
    }
  }
}

For Developers

Deploy the MCP server to Vercel:

cd mcp-server
npm install
vercel --prod

Important: Set ELEVENLABS_API_KEY as a Vercel environment variable in your project settings (not in .env files).

Features

Agent Management

List Agents: Get all available ElevenLabs agents in your account
Get Agent Details: Retrieve detailed information about specific agents

Conversation Capabilities

Initiate Agent Calls: Start conversations with agents and wait for completion (default 50s timeout, configurable up to 300s)
Outbound Phone Calls: Make phone calls to any number using an agent
Status Tracking: Monitor call progress and completion
Transcript Retrieval: Get full conversation transcripts with metadata

Call Outcomes

The system handles multiple call scenarios:

Completed: Successful conversations with full transcripts
Failed: Calls that encountered errors
No Answer: Calls that were initiated but never answered
In Progress: Active or processing calls

Repository Structure

elevenlabs-agents/
├── README.md                    # This file
├── CLAUDE.md                    # Project instructions for Claude Code
├── skill/                       # Claude Code skill files
│   ├── SKILL.md                 # Skill definition and usage guide
│   ├── examples.md              # Comprehensive usage examples
│   └── reference.md             # API reference documentation
└── mcp-server/                  # MCP server implementation
    ├── README.md                # Detailed MCP server documentation
    ├── api/                     # Vercel serverless functions
    ├── lib/                     # Core implementation
    │   ├── elevenlabs/          # ElevenLabs API client
    │   └── mcp/                 # MCP server logic
    ├── test/                    # Test scripts
    ├── package.json
    ├── tsconfig.json
    └── vercel.json              # Vercel configuration

Prerequisites

ElevenLabs API Key: Get one from ElevenLabs
Vercel Account: For deploying the MCP server
Node.js 18+: For local development
(Optional) Phone Numbers: For outbound calling features

Setup Guide

1. Get Your ElevenLabs API Key

Sign up at ElevenLabs
Navigate to your API settings
Generate or copy your API key

2. Deploy MCP Server to Vercel

cd mcp-server
npm install

# Install Vercel CLI if needed
npm i -g vercel

# Login to Vercel
vercel login

# Deploy to production
vercel --prod

3. Configure Environment Variables in Vercel

Critical: Set environment variables in Vercel dashboard, not in .env files:

Go to your Vercel project settings
Navigate to "Environment Variables"
Add: ELEVENLABS_API_KEY = your_elevenlabs_api_key_here
Apply to Production, Preview, and Development environments

Why Vercel Environment Variables?

Secure: Never committed to version control
Environment-specific: Different keys for dev/staging/prod
Automatic: Available to all serverless functions
Standard practice: Vercel's recommended approach

4. Add to Claude Code

Once deployed, add the MCP server URL to your Claude Code configuration:

{
  "mcpServers": {
    "elevenlabs-agents": {
      "url": "https://your-app.vercel.app/mcp",
      "transport": "http"
    }
  }
}

Available MCP Tools

Agent Management

list_agents - Get all agents in your account
get_agent - Get details for a specific agent

Conversations

initiate_agent_call - Start a conversation and wait for completion
- Default timeout: 50 seconds (configurable up to 300s)
- Returns full transcript on completion

Outbound Calling

list_phone_numbers - Get available phone numbers for outbound calls
initiate_outbound_call - Make a phone call to any number
- Phone format: E.164 (e.g., +15551234567)
- Returns conversation_id for tracking

Status & Transcripts

get_call_status - Check current status of a conversation
- Statuses: initiated, in-progress, processing, done, failed
get_call_transcript - Retrieve full transcript of any finalized call
- Works for completed, failed, and no-answer calls
- Includes conversation turns, timestamps, costs, and metadata

Usage Examples

Quick Agent Test

// List agents
const agents = await useMCP('elevenlabs-agents', 'list_agents');

// Test an agent
const result = await useMCP('elevenlabs-agents', 'initiate_agent_call', {
  agent_id: 'agent_xxx',
  initial_message: 'Hello, can you tell me about your services?'
});

console.log(result.transcript);

Outbound Phone Call

// Check available phone numbers
const numbers = await useMCP('elevenlabs-agents', 'list_phone_numbers');

// Initiate call
const call = await useMCP('elevenlabs-agents', 'initiate_outbound_call', {
  agent_id: 'agent_xxx',
  to_number: '+15551234567'
});

// Poll for status
const status = await useMCP('elevenlabs-agents', 'get_call_status', {
  conversation_id: call.conversation_id
});

// Get transcript when done
if (status.call_outcome === 'completed') {
  const transcript = await useMCP('elevenlabs-agents', 'get_call_transcript', {
    conversation_id: call.conversation_id
  });
}

Common Workflows

Development Workflow

Make changes to mcp-server/ code
Test locally: cd mcp-server && npm run dev
Deploy preview: vercel
Test in Claude Code with preview URL
Deploy production: vercel --prod

Testing Agent Changes

Update agent configuration in ElevenLabs dashboard
Use list_agents to verify changes
Test with initiate_agent_call for quick validation
Monitor transcripts for quality assurance

Architecture

Serverless Design

HTTP Transport: Runs as Vercel serverless function
Stateless: No persistent connections or state
Timeout-Aware: Default 50s, max 300s (Vercel limit)
Retry Logic: Automatic retry with exponential backoff for API calls

Polling Strategy

Starts at 2-second intervals
Exponential backoff (1.5x multiplier)
Max interval: 10 seconds
Continues polling on transient errors

Error Handling

Network errors: Retry up to 3 times
API errors: Formatted with status code and message
Tool errors: Wrapped in MCP error format

Local Development

Run MCP Server Locally

cd mcp-server

# Set environment variable
export ELEVENLABS_API_KEY=your_api_key

# Run with stdio transport (for testing)
node lib/mcp/server.ts

# Or run with Vercel dev server (HTTP transport)
npm run dev

Then test at http://localhost:3000/api/mcp

Run Tests

cd mcp-server
npm test

Troubleshooting

"Authentication failed" errors

Verify ELEVENLABS_API_KEY is set in Vercel environment variables (not .env)
Check API key is valid in ElevenLabs dashboard
Ensure key has proper permissions

Conversation timeouts

Default timeout is 50s; increase via timeout parameter if needed
Maximum timeout is 300s (Vercel serverless limit)
For longer conversations, use outbound calling with status polling
Check Vercel function logs for errors

"No phone numbers available"

Verify phone numbers are configured in ElevenLabs account
Check phone number permissions and status
Use list_phone_numbers to see available numbers

Outbound calls not connecting

Ensure phone number is in E.164 format (+15551234567)
Check recipient phone can receive calls
Verify Twilio integration is active in ElevenLabs

Security Best Practices

Never commit API keys: Use Vercel environment variables only
CORS configured: Properly set up for MCP client access
Environment isolation: Separate keys for dev/staging/prod
Regular rotation: Update API keys periodically

Documentation

Skill Documentation

- Skill definition and usage guide
- Comprehensive usage examples
- API reference documentation

MCP Server Documentation

- Detailed server documentation
- Project instructions for Claude Code

External Resources

Contributing

Contributions welcome! Please ensure:

TypeScript types are properly defined
Error handling is comprehensive
Tests pass: cd mcp-server && npm test
Documentation is updated
Environment variables are documented

Limitations

Timeout: Max 300 seconds (5 minutes) per request due to Vercel limits
Stateless: No background job processing or persistent state
Synchronous: All operations complete within single request lifecycle

For conversations exceeding 5 minutes, use the outbound calling pattern with status polling.

License

MIT

Support

Issues: GitHub Issues
ElevenLabs: ElevenLabs Documentation
MCP Protocol: MCP Documentation
Vercel: Vercel Documentation