ismaArce/ai-knowledge-base

AI Knowledge Base MCP Server

A unified AI knowledge base that uses OpenAI embeddings and vector search to help you find relevant information across prompt engineering and agent development resources.

🎯 Purpose

This MCP server helps you:

• Search through prompt engineering AND agent development books
• Find cross-domain insights between prompting and agent building
• Discover specific techniques and examples across both domains
• Compare how concepts apply differently in prompting vs agents
• Build a comprehensive searchable knowledge base of your AI expertise

🏗️ Architecture

• MCP Server: FastMCP-based server with search tools
• Vector Database: Qdrant for storing embeddings
• Embeddings: OpenAI text-embedding-3-small
• Content Processing: Support for PDF, EPUB, DOCX, TXT, MD files

🚀 Quick Start

1. Setup Environment

# Clone and enter the project
cd prompt-knowledge-base

# Set your OpenAI API key
touch .env
# add your OPENAI_API_KEY

2. Add Your Content

# Add your books organized by domain
cp prompt_engineering_book.pdf books/prompting/
cp agent_development_book.pdf books/agents/
cp general_ai_book.pdf books/general/

# Add any existing prompt files (optional)
cp your_prompts.txt prompts/

3. Start Services

# Start the services
docker-compose up -d

# Wait for services to be ready
docker-compose logs -f

4. Index Your Content

# Run the indexing script to process your books
docker-compose exec prompt-kb python scripts/index_content.py

# Test the search functionality
docker-compose exec prompt-kb python scripts/test_search.py

🛠️ MCP Tools Available

search_prompts

Search for prompts and techniques using semantic similarity.

search_prompts("debugging python code", limit=10, source_type="book")

search_by_technique

Find examples of specific prompt engineering techniques.

search_by_technique("chain of thought", limit=5)

search_by_use_case

Search for prompts relevant to specific job tasks.

search_by_use_case("code review", limit=8)

get_context

Get full context around a specific search result.

get_context("chunk_id_here")

list_sources

List all indexed books and prompt files.

list_sources()

📁 Directory Structure

prompt-knowledge-base/
├── books/              # Your prompt engineering books (PDF, EPUB, etc.)
├── prompts/            # Your existing prompt files (TXT, MD)
├── data/qdrant/        # Vector database storage
├── src/                # Source code
├── scripts/            # Utility scripts
└── docker-compose.yml  # Services configuration

🔧 Configuration

Environment variables in .env:

OPENAI_API_KEY=your_api_key_here
EMBEDDING_MODEL=text-embedding-3-small
CHUNK_SIZE=500
CHUNK_OVERLAP=50

📚 Supported File Formats

• PDF: Books, papers, documents
• EPUB: E-books
• DOCX: Word documents
• TXT: Plain text files
• MD: Markdown files

🔍 Usage Examples

# Search for debugging techniques
await search_prompts("step by step debugging approach")

# Find chain of thought examples
await search_by_technique("chain of thought")

# Get prompts for code reviews
await search_by_use_case("code review feedback")

# List what's in your knowledge base
await list_sources()

🚨 Troubleshooting

No results found

• Check if content was indexed: docker-compose exec prompt-kb python scripts/test_search.py
• Verify files are in books/ or prompts/ directories
• Re-run indexing: docker-compose exec prompt-kb python scripts/index_content.py

OpenAI API errors

• Verify your API key is set correctly in .env
• Check your OpenAI account has sufficient credits

Qdrant connection issues

• Ensure services are running: docker-compose ps
• Check logs: docker-compose logs qdrant

📈 Next Steps

1. Add your prompt engineering books to the books/ directory
2. Run the indexing script to build your knowledge base
3. Start using the MCP tools in your AI assistant
4. Refine search queries based on your specific use cases

🤝 Contributing

Feel free to extend this knowledge base with additional features:

• More file format support
• Advanced search filters
• Prompt rating and feedback systems
• Export functionality