autonomous-thought-graph-dspy by 5oclockshadow - MCP Server

🧠 Autonomous Thought Graph Generator

A DSPy-powered system that generates thought graphs autonomously, with LLM-driven state management through Pydantic base models. Features a web UI with GraphVis visualization and an MCP server for integration.

🌟 Key Features

Autonomous Processing

Single DSPy Signature: ReactSignature handles all operations through reasoning
Single Module: AutonomousThoughtReactor manages its own state autonomously
LLM State Management: The system decides its own processing flow through base models
Try/Except/Finally Loop: Robust error handling with recovery mechanisms

Flexible Input/Output

ExpandedInput: Flexible input model that accepts any additional fields
UnifiedOutput: Union of all possible outputs with nested base models
Granular Base Models: Thought, Connection, ThoughtGraph, ProcessingContext, ValidationResult

Web UI & Visualization

Interactive Graph: HTML5 + GraphVis visualization
Real-time Processing: Shows autonomous decision-making process
Beautiful UI: Modern gradient design with hover effects

MCP Server

4 Tools: Generate, validate, expand thoughts, and get statistics
JSON API: Full integration with MCP protocol
Async Support: Non-blocking operations

🚀 Quick Start

Installation

cd thought_graph
pip install -r requirements.txt

Run Modes

1. Demo Mode (Autonomous Processing)

python main.py demo

Shows the autonomous system processing different inputs with full logging.

2. Web UI Mode

python main.py web

Starts web server at http://localhost:52336

3. MCP Server Mode

python main.py mcp

Starts MCP server for integration with other tools.

🏗️ Architecture

Core Components

1. Autonomous Reactor (`AutonomousThoughtReactor`)

class AutonomousThoughtReactor(dspy.Module):
    def forward(self, expanded_input: ExpandedInput, use_fake_data: bool = True) -> UnifiedOutput:
        # Main autonomous loop with try/except/finally
        while context.should_continue and context.iteration_count < context.max_iterations:
            try:
                # LLM decides next action based on current state
                result = self._process_with_llm(expanded_input, context, graph)
                # Update state based on LLM decision
            except Exception as e:
                # Error handling and recovery
            finally:
                # Always update output with current state

2. Single Signature (`ReactSignature`)

class ReactSignature(dspy.Signature):
    """React to input and context, managing own state autonomously through reasoning."""
    input_data = dspy.InputField(desc="Input data as JSON string containing ExpandedInput model")
    context = dspy.InputField(desc="Current processing context as JSON string")
    reasoning = dspy.OutputField(desc="Step-by-step reasoning about what to do next")
    action = dspy.OutputField(desc="Next action to take")
    output_data = dspy.OutputField(desc="Output data as JSON string containing UnifiedOutput model")
    updated_context = dspy.OutputField(desc="Updated processing context as JSON string")

3. Base Models with Nesting

class ProcessingContext(BaseModel):
    """Context for LLM to manage its own state"""
    current_state: SystemState = SystemState.INITIALIZING
    iteration_count: int = 0
    should_continue: bool = True
    reasoning: str = ""
    next_action: str = ""

class UnifiedOutput(BaseModel):
    """Union of all possible outputs with nesting"""
    graph: Optional[ThoughtGraph] = None
    validation: Optional[ValidationResult] = None
    context: Optional[ProcessingContext] = None
    thoughts: Optional[List[Thought]] = Field(default_factory=list)
    connections: Optional[List[Connection]] = Field(default_factory=list)
    
    class Config:
        extra = "allow"  # Allow additional fields

State Flow

INITIALIZING → Set up initial context
PROCESSING → Generate initial thoughts
EXPANDING → Create connections and deeper thoughts
VALIDATING → Check quality and completeness
COMPLETED → Finalize output
ERROR → Handle errors and attempt recovery

🎨 Web UI Features

Interactive Graph Visualization: Click nodes to see details
Real-time Generation: Watch the autonomous process unfold
Beautiful Design: Modern UI with gradients and animations
Responsive: Works on desktop and mobile
Example Inputs: Quick-start with predefined examples

🔧 MCP Server Tools

1. `generate_thought_graph`

Generate a complete thought graph from input text.

2. `validate_thought_graph`

Validate an existing graph and provide feedback.

3. `expand_thought`

Expand a specific thought with related concepts.

4. `get_graph_statistics`

Get detailed statistics about a thought graph.

📊 Example Output

{
  "success": true,
  "input_text": "artificial intelligence",
  "graph": {
    "thoughts": [
      {
        "id": "abc123",
        "content": "Machine learning algorithms that can adapt and learn",
        "thought_type": "concept",
        "confidence": 0.85,
        "depth": 1
      }
    ],
    "connections": [
      {
        "source_id": "abc123",
        "target_id": "def456",
        "relationship": "leads to",
        "strength": 0.7
      }
    ]
  },
  "processing_log": {
    "iterations": 5,
    "final_state": "completed",
    "reasoning": "Generated comprehensive thought network with balanced coverage"
  }
}

🧪 Testing

# Test MCP tools
python test_mcp.py

# Test web API
curl http://localhost:52336/api/generate/creativity

# Test autonomous processing
python main.py demo

🔮 Advanced Usage

Custom Context

expanded_input = ExpandedInput(
    original_text="quantum computing",
    context={
        "focus": "technical_depth",
        "style": "academic"
    },
    preferences={
        "max_depth": 4,
        "creative_mode": False
    },
    domain_knowledge="physics"  # Extra field allowed
)

Real LLM Integration

Replace the mock LM in dspy_program.py:

import dspy
lm = dspy.OpenAI(model="gpt-4", max_tokens=500)
dspy.settings.configure(lm=lm)

🤝 Contributing

Fork the repository
Create a feature branch
Add tests for new functionality
Submit a pull request

📝 License

MIT License - see LICENSE file for details.

Built with ❤️ using DSPy, Pydantic, FastAPI, and GraphVis