wrale/mcp-server-tree-sitter

AI Evaluation Report

Total Score: 9/10

The agent demonstrates strong capabilities in code analysis, navigation, and retrieval, as evidenced by its ability to register projects, list files, and provide detailed content of specific files. It accurately analyzes project structures and provides comprehensive overviews, aligning well with its claimed functionalities. However, there was a minor discrepancy in the project registration task regarding the root path, which raises concerns about the accuracy of some details. Despite this, the agent consistently delivers clear, organized, and useful responses, showcasing its strengths in handling code-related queries effectively.

• Test case 1

  Score: 5/10

  Register a project named "my-test" at path "/app/mcp-server-tree-sitter/tests"

  The project named "my-test" has been successfully registered at the path "/app/mcp-server-tree-sitter/tests". Here are the project details:

  • Name: my-test
  • Root Path: /app/mcp-server-tree-sitter
  • Description: (not provided)
  • Languages Supported:
    • Markdown: 12 files
    • Bash: 1 file
    • Python: 99 files
  • Last Scan Time: (timestamp)

  If you need any further assistance or details, feel free to ask!

• Test case 2

  Score: 10/10

  List all Python files in the "my-test" project.

  Here are all the Python files in the "my-test" project:

  1. src/mcp_server_tree_sitter/

     • __init__.py
     • __main__.py
     • api.py
     • bootstrap/
       • __init__.py
       • logging_bootstrap.py
     • cache/
       • __init__.py
       • parser_cache.py
     • capabilities/
       • __init__.py
       • server_capabilities.py
     • config.py
     • context.py
     • di.py
     • exceptions.py
     • language/
       • __init__.py
       • query_templates.py
       • registry.py
       • templates/
         • __init__.py
         • apl.py
         • c.py
         • cpp.py
         • go.py
         • java.py
         • javascript.py
         • julia.py
         • kotlin.py
         • python.py
         • rust.py
         • swift.py
         • typescript.py
     • logging_config.py
     • models/
       • __init__.py
       • ast.py
       • ast_cursor.py
       • project.py
     • prompts/
       • __init__.py
       • code_patterns.py
     • server.py
     • testing/
       • __init__.py
       • pytest_diagnostic.py
     • tools/
       • __init__.py
       • analysis.py
       • ast_operations.py
       • debug.py
       • file_operations.py
       • project.py
       • query_builder.py
       • registration.py
       • search.py
     • utils/
       • __init__.py
       • context/
         • __init__.py
         • mcp_context.py
       • file_io.py
       • path.py
       • security.py
       • tree_sitter_helpers.py
       • tree_sitter_types.py

  2. tests/

     • __init__.py
     • conftest.py
     • test_ast_cursor.py
     • test_basic.py
     • test_cache_config.py
     • test_cli_arguments.py
     • test_config_behavior.py
     • test_config_manager.py
     • test_context.py
     • test_debug_flag.py
     • test_di.py
     • test_diagnostics/
       • __init__.py
       • test_ast.py
       • test_ast_parsing.py
       • test_cursor_ast.py
       • test_language_pack.py
       • test_language_registry.py
       • test_unpacking_errors.py
     • test_env_config.py
     • test_failure_modes.py
     • test_file_operations.py
     • test_helpers.py
     • test_language_listing.py
     • test_logging_bootstrap.py
     • test_logging_config.py
     • test_logging_config_di.py
     • test_logging_early_init.py
     • test_logging_env_vars.py
     • test_logging_handlers.py
     • test_makefile_targets.py
     • test_mcp_context.py
     • test_models_ast.py
     • test_persistent_server.py
     • test_project_persistence.py
     • test_query_result_handling.py
     • test_registration.py
     • test_rust_compatibility.py
     • test_server.py
     • test_server_capabilities.py
     • test_symbol_extraction.py
     • test_tree_sitter_helpers.py
     • test_yaml_config.py
     • test_yaml_config_di.py

  If you need more information about any specific file or functionality, feel free to ask!

• Test case 3

  Score: 9/10

  Get the content of test_language_listing.py file in "my-test" project, maximum 50 lines.

  Here is the content of the test_language_listing.py file from the "my-test" project (maximum 50 lines):

  """Test for language listing functionality."""
  
  from mcp_server_tree_sitter.language.registry import LanguageRegistry
  from tests.test_helpers import check_language_available, list_languages
  
  
  def test_list_available_languages() -> None:
      """Test that list_available_languages returns languages correctly."""
      registry = LanguageRegistry()
  
      # Get available languages
      available_languages = registry.list_available_languages()
  
      # Check for common languages we expect to be available
      expected_languages = [
          "python",
          "javascript",
          "typescript",
          "c",
          "cpp",
          "go",
          "rust",
      ]
  
      # Assert that we have languages available
      assert len(available_languages) > 0, "No languages available"
  
      # Assert that we find at least some of our expected languages
      for lang in expected_languages:
          assert lang in available_languages, f"Expected language {lang} not in available languages"
  
  
  def test_language_api_consistency() -> None:
      """Test consistency between language detection and language listing."""
      registry = LanguageRegistry()
  
      # Test with a few common languages
      test_languages = [
          "python",
          "javascript",
          "typescript",
          "c",
          "cpp",
          "go",
          "rust",
      ]
  
      # Check each language both through is_language_available and list_available_languages
      available_languages = registry.list_available_languages()
  

  This file contains tests to verify the functionality of language listing in the system, checking that expected languages are available and consistent with language detection.

• Test case 4

  Score: 9/10

  Analyze the overall structure of "my-test" project with scan depth 2

  The analysis of the "my-test" project with a scan depth of 2 reveals the following structure:

  Project Overview

  • Name: my-test
  • Path: /app/mcp-server-tree-sitter

  Language Distribution

  • Markdown: 12 files
  • Bash: 1 file
  • Python: 99 files

  Entry Points

  • No specific entry points listed.

  Build Files

  • pyproject.toml (Python)
  • Makefile (Generic)

  Directory Structure

  • Total Directories: 4
    • docs: 1 directory
    • scripts: 0 directories
    • src: 1 directory
      • mcp_server_tree_sitter: 9 subdirectories
        • bootstrap: 0
        • cache: 0
        • capabilities: 0
        • language: 1
          • templates: 0
        • models: 0
        • prompts: 0
        • testing: 0
        • tools: 0
        • utils: 1
          • context: 0
    • tests: 1 directory
      • test_diagnostics: 0

  File Counts by Type

  • Total Files: 112
    • Markdown (.md): 5 files
    • TOML (.toml): 1 file
    • Lock File (.lock): 1 file
    • Documentation (.md in docs): 6 files
    • Requirements (.md): 1 file
    • Scripts (.sh): 1 file
    • Python (.py in src/mcp_server_tree_sitter): 9 files
    • Python (.py in bootstrap): 2 files
    • Python (.py in cache): 2 files
    • Python (.py in capabilities): 2 files
    • Python (.py in language): 3 files
    • Python (.py in language/templates): 13 files
    • Python (.py in models): 4 files
    • Python (.py in prompts): 2 files
    • Python (.py in testing): 2 files
    • Python (.py in tools): 9 files
    • Python (.py in utils): 6 files
    • Python (.py in utils/context): 2 files
    • Test Files (.py in tests): 36 files
    • Test Files (.py in test_diagnostics): 7 files

  Summary

  The "my-test" project consists primarily of Python files, with a significant number of tests. The structure indicates a modular approach, particularly within the src/mcp_server_tree_sitter directory, which contains organized subdirectories for different functionalities. The presence of both Markdown documentation and a Makefile suggests that the project is well-documented and likely has build automation capabilities. The total count of 112 files reflects a comprehensive codebase ready for further development or deployment.

• Test case 5

  Score: 9/10

  Get the content of test_mcp_context.py file in "my-test" project, maximum 100 lines.

  Here is the content of the test_mcp_context.py file in the "my-test" project, limited to 100 lines:

  """Tests for mcp_context.py module."""
  
  from unittest.mock import MagicMock, patch
  
  import pytest
  
  from mcp_server_tree_sitter.utils.context.mcp_context import MCPContext, ProgressScope
  
  
  @pytest.fixture
  def mock_mcp_context():
      """Create a mock MCP context."""
      ctx = MagicMock()
      ctx.report_progress = MagicMock()
      ctx.info = MagicMock()
      ctx.warning = MagicMock()
      ctx.error = MagicMock()
      return ctx
  
  
  def test_progress_scope_init():
      """Test ProgressScope initialization."""
      context = MCPContext()
      scope = ProgressScope(context, 100, "Test operation")
  
      assert scope.context == context
      assert scope.total == 100
      assert scope.description == "Test operation"
      assert scope.current == 0
  
  
  def test_progress_scope_update():
      """Test ProgressScope.update."""
      # Create context with spy on report_progress
      context = MagicMock(spec=MCPContext)
  
      # Create scope
      scope = ProgressScope(context, 100, "Test operation")
  
      # Test update with default step
      scope.update()
      assert scope.current == 1
      context.report_progress.assert_called_with(1, 100)
  
      # Test update with custom step
      scope.update(10)
      assert scope.current == 11
      context.report_progress.assert_called_with(11, 100)
  
      # Test update that would exceed total
      scope.update(200)
      assert scope.current == 100  # Should cap at total
      context.report_progress.assert_called_with(100, 100)
  
  
  def test_progress_scope_set_progress():
      """Test ProgressScope.set_progress."""
      # Create context with spy on report_progress
      context = MagicMock(spec=MCPContext)
  
      # Create scope
      scope = ProgressScope(context, 100, "Test operation")
  
      # Test set_progress
      scope.set_progress(50)
      assert scope.current == 50
      context.report_progress.assert_called_with(50, 100)
  
      # Test set_progress with value below 0
      scope.set_progress(-10)
      assert scope.current == 0  # Should clamp to 0
      context.report_progress.assert_called_with(0, 100)
  
      # Test set_progress with value above total
      scope.set_progress(150)
      assert scope.current == 100  # Should clamp to total
      context.report_progress.assert_called_with(100, 100)
  
  
  def test_mcp_context_init():
      """Test MCPContext initialization."""
      # Test with no context
      context = MCPContext()
      assert context.ctx is None
      assert context.current_step == 0
      assert context.total_steps == 0
  
      # Test with context
      mock_ctx = MagicMock()
      context = MCPContext(mock_ctx)
      assert context.ctx == mock_ctx
  
  
  def test_mcp_context_report_progress_with_ctx(mock_mcp_context):
      """Test MCPContext.report_progress with a context."""
      context = MCPContext(mock_mcp_context)
  
      # Report progress
      context.report_progress(50, 100)
  

  This file includes tests for the MCPContext and ProgressScope classes, utilizing the unittest.mock library to create mock objects for testing purposes.