champi-ai/champi

Champi Voice Assistant 🎤

An intelligent MCP (Model Context Protocol) server that provides advanced voice interaction capabilities with real-time visual feedback through a modern floating UI overlay.

Features

🎯 Core Voice Capabilities

• Real-time Speech-to-Text: Powered by WhisperLive for continuous audio transcription
• Text-to-Speech: High-quality voice synthesis using Kokoro TTS models
• Voice Activity Detection: Smart silence detection for natural conversation flow
• Multiple Voice Models: Support for various TTS voices (Adam, etc.)

🖥️ Visual Interface

• Floating Avatar UI: Clean, modern overlay window using imgui-bundle
• Real-time Status Indicators: Animated visual feedback for different modes
  • 🔵 Idle: Blue pulsing circle
  • 🟠 Listening: Orange animated indicator
  • 🟡 Processing: Yellow processing animation
  • 🟢 Speaking: Green active voice indicator
• Transparent Background: Non-intrusive overlay that works with any application
• Right-click Context Menu: Quick access to test controls and settings

🔧 Technical Architecture

• MCP Server Integration: Full Model Context Protocol compliance
• Multi-threaded Processing: Separate threads for audio, UI, and IPC communication
• Socket-based IPC: Real-time communication between voice engine and UI
• Cross-platform Support: Works on Windows, macOS, and Linux

Installation

Prerequisites

# Python 3.13+ required
uv python pin 3.13

# Install system dependencies (Ubuntu/Debian)
sudo apt-get update
sudo apt-get install portaudio19-dev python3-dev

# For macOS
brew install portaudio

# For Windows
# No additional system packages required

Setup

# Clone the repository
git clone <repository-url>
cd champi

# Install Python and dependencies
uv python pin 3.13
uv sync

Usage

Starting the Voice Assistant

# Run the complete voice assistant
uv run python -m mcp_champi

# Or start individual components
uv run python -m mcp_champi.server  # MCP server only
uv run python -m mcp_champi.ui_svc.voice_ui_bundle  # UI overlay only

Voice UI Controls

• Visual Status: The floating avatar shows current voice activity
• Right-click Menu: Access test controls and settings
• IPC Communication: Send commands via JSON messages to port 9876

MCP Integration

# Example client usage
import mcp_champi

# Initialize voice conversation
conversation = mcp_champi.VoiceConversation()
await conversation.start_listening()

# Get transcribed text
text = await conversation.get_transcription()
print(f"User said: {text}")

# Respond with voice
await conversation.speak("Hello! I heard you say: " + text)

Configuration

Environment Variables

Champi can be configured using environment variables with the CHAMPI_ prefix:

# Core Settings
export CHAMPI_BASE_DIR="~/champi"
export CHAMPI_DEBUG="true"
export CHAMPI_SAVE_ALL="false"

# Audio Configuration
export CHAMPI_SAMPLE_RATE="16000"
export CHAMPI_AUDIO_FORMAT="pcm"
export CHAMPI_DISABLE_SILENCE_DETECTION="false"

# TTS Configuration
export CHAMPI_TTS_VOICES="am_adam,ef_dora,ff_siwis"
export CHAMPI_TTS_AUDIO_FORMAT="pcm"
export CHAMPI_AUTO_START_KOKORO="false"

# STT Configuration
export CHAMPI_STT_AUDIO_FORMAT="wav"
export CHAMPI_SILENCE_THRESHOLD_MS="800"
export CHAMPI_VAD_AGGRESSIVENESS="2.0"

# Streaming Configuration
export CHAMPI_STREAMING_ENABLED="true"
export CHAMPI_STREAM_CHUNK_SIZE="4096"

Voice Models

# Available TTS voices
VOICES = {
    "adam": "am_adam",  # Default male voice
    "dora": "ef_dora",  # Spanish female voice
    "siwis": "ff_siwis", # French female voice
    # Add more as needed
}

Audio Settings

# Default audio configuration
AUDIO_CONFIG = {
    "sample_rate": 16000,
    "channels": 1,
    "audio_format": "pcm",
    "silence_threshold_ms": 800,
    "vad_aggressiveness": 2.0
}

API Reference

MCP Tools Available

Voice Conversation Tools

• start_voice_conversation: Begin interactive voice session
• stop_voice_conversation: End voice session
• get_conversation_status: Check current conversation state
• set_voice_model: Change TTS voice

Audio Processing Tools

• transcribe_audio: Convert audio file to text
• synthesize_speech: Generate audio from text
• detect_voice_activity: Check for speech in audio

Device Management Tools

• list_audio_devices: Show available audio input/output devices
• set_audio_device: Configure audio device settings
• test_audio_setup: Verify audio configuration

IPC Message Format

{
  "type": "voice_start",
  "text": "Hello, world!",
  "voice": "am_adam"
}

{
  "type": "listening_start"
}

{
  "type": "listening_stop",
  "text": "Transcribed speech..."
}

{
  "type": "voice_stop"
}

Development

Project Structure

mcp_champi/
├── core.py                 # Main MCP server implementation
├── server.py              # Server entry point
├── streaming.py           # Audio streaming utilities
├── config.py              # Configuration management
├── whisper_svc/           # Speech-to-text services
├── kokoro_svc/            # Text-to-speech services
├── ui_svc/                # User interface components
├── tools/                 # MCP tool implementations
├── resources/             # MCP resources
└── prompts/               # System prompts

Contributing

1. Fork the repository
2. Create a feature branch: git checkout -b feature-name
3. Make your changes with tests
4. Submit a pull request

Testing

# Run the test suite
uv run pytest tests/

# Test voice functionality
uv run python -m mcp_champi.ui_svc.voice_ui_bundle test

# Manual testing with specific components
uv run python -m mcp_champi.tools.simple_duplex

Troubleshooting

Common Issues

Audio Device Not Found

# List available devices
uv run python -c "import sounddevice; print(sounddevice.query_devices())"

# Test audio setup
uv run python -m mcp_champi.tools.diagnostics

UI Not Displaying

• Ensure imgui-bundle is installed: uv add imgui-bundle
• Check graphics drivers are up to date
• Try running without transparency: set TRANSPARENT_FRAMEBUFFER=False

Voice Models Not Loading

• Verify internet connection for model downloads
• Check available disk space for model storage
• Ensure proper permissions for model cache directory

Performance Optimization

• Adjust chunk_size for lower latency
• Use smaller voice models for faster processing
• Disable UI for headless operation
• Configure silence detection sensitivity

License

This project is licensed under the MIT License - see the LICENSE file for details.

Acknowledgments

• WhisperLive: Real-time speech recognition
• Kokoro TTS: High-quality text-to-speech
• imgui-bundle: Modern UI framework
• MCP Protocol: Model Context Protocol specification

Made with ❤️ for seamless voice interaction