iamsnh01/test-automator

🤖 Test Automator

A comprehensive, intelligent, and extensible testing automation platform powered by Large Language Models. Test Automator streamlines the process of generating, executing, and analyzing various types of software tests (unit, integration, end-to-end, API) for both web UI and backend logic.

✨ Features

🧠 LLM-Powered Test Generation

• Intelligent Analysis: Automatically analyzes your codebase to understand structure and dependencies
• Smart Test Cases: Generates comprehensive test scenarios including edge cases and error conditions
• Context-Aware: Understands your code patterns and generates idiomatic tests
• Multi-Language Support: Optimized for Python with extensible architecture

🔧 Comprehensive Test Types

🔬 Unit Testing

• Analyzes individual functions, methods, and classes
• Generates pytest test functions with proper fixtures
• Includes positive, negative, and edge case scenarios
• Handles both sync and async code patterns

🔗 Integration Testing

• Tests interactions between different modules and services
• Simulates real component interactions
• Uses appropriate mocking strategies
• Tests configuration and initialization flows

🌐 End-to-End (E2E) Testing

• Browser automation using Playwright and browser-use
• Simulates real user interactions
• Tests complete user workflows
• Captures screenshots and generates visual reports

🚀 API Testing

• Comprehensive HTTP endpoint testing
• Request/response validation
• Authentication and authorization testing
• Performance and timeout testing

📊 Intelligent Reporting

• LLM-Enhanced Analysis: AI-powered insights from test results
• Multi-Format Support: XML, JSON, and HTML report parsing
• Actionable Recommendations: Specific suggestions for improvement
• Risk Assessment: Identifies critical areas needing attention

🛠️ Installation

Prerequisites

• Python 3.11+
• Google API Key (for Gemini LLM)
• Claude Code or Cursor with MCP support
• Git

Quick Install

Windows (Native or WSL)

# Clone the repository
git clone https://github.com/your-repo/test-automator.git
cd test-automator

# Create virtual environment
python -m venv .venv
.venv\\Scripts\\activate  # Windows
# or
source .venv/bin/activate  # WSL/Linux

# Install dependencies
pip install -e .

# Install Playwright browsers
playwright install

Linux/macOS

# Clone and setup
git clone https://github.com/your-repo/test-automator.git
cd test-automator

# Create virtual environment
python3.11 -m venv .venv
source .venv/bin/activate

# Install dependencies
pip install -e .

# Install Playwright browsers
playwright install

⚙️ Configuration

1. Get Google API Key

1. Visit Google AI Studio
2. Create a new API key
3. Set environment variable:

   # Windows
   set GOOGLE_API_KEY=your_api_key_here
   
   # Linux/macOS/WSL
   export GOOGLE_API_KEY=your_api_key_here
   

2. Configure Claude Code

Option A: CLI Configuration

claude mcp add test-automator "/path/to/test-automator/.venv/bin/test-automator" -e "GOOGLE_API_KEY=your_api_key"

Option B: Manual Configuration

Add to your Claude Code configuration (~/.claude.json):

{
  "projects": {
    "/your/project/path": {
      "mcpServers": {
        "test-automator": {
          "type": "stdio",
          "command": "/path/to/test-automator/.venv/bin/test-automator",
          "env": {
            "GOOGLE_API_KEY": "your_api_key_here"
          }
        }
      }
    }
  }
}

Windows Path Examples

• Windows: C:/path/to/test-automator/.venv/Scripts/test-automator.exe
• WSL: /home/username/test-automator/.venv/bin/test-automator

🚀 Usage

MCP Tools Available

generate_tests(code_path, test_type="all")

Generate intelligent tests for your codebase:

# Generate all test types
generate_tests("/path/to/your/code", "all")

# Generate specific test type
generate_tests("/path/to/your/api.py", "unit")
generate_tests("/path/to/your/project", "integration")
generate_tests("/path/to/your/webapp", "e2e")
generate_tests("/path/to/your/api", "api")

run_tests(test_type="all", target_path="tests/")

Execute generated tests:

# Run all tests
run_tests("all", "/path/to/tests")

# Run specific test type
run_tests("unit", "/path/to/tests")

analyze_test_report(report_path)

Get LLM-powered insights from test results:

analyze_test_report("/path/to/test_results.xml")

Example Workflow

# 1. Generate comprehensive tests
generate_tests("/home/user/my-project", "all")

# 2. Run the tests
run_tests("all", "/home/user/my-project/tests")

# 3. Analyze results
analyze_test_report("/home/user/my-project/tests/results/unit_results.xml")

🎯 Advanced Features

Large Codebase Support (100k-200k lines)

• Modular Analysis: Processes code in manageable chunks
• Incremental Testing: Generates tests incrementally for better performance
• Parallel Execution: Supports pytest-xdist for parallel test runs
• Smart Filtering: Focuses on testable units to avoid overwhelming LLM

Cross-Platform Compatibility

• Windows Native: Full support with proper path handling
• WSL Integration: Seamless Windows Subsystem for Linux support
• Linux/macOS: Native Unix support
• Event Loop Handling: Platform-specific async optimizations

Performance Optimizations

• Async Operations: Non-blocking test execution
• Batch Processing: Efficient handling of multiple test files
• Resource Management: Proper cleanup and memory management
• Timeout Handling: Configurable timeouts for different test types

📁 Project Structure

test-automator/
├── test_automator/
│   ├── __init__.py
│   ├── mcp_server.py          # Main MCP server with tools
│   ├── test_generator.py      # LLM-powered test generation
│   ├── test_runner.py         # Cross-platform test execution
│   └── report_analyzer.py     # AI-enhanced report analysis
├── pyproject.toml             # Package configuration
└── README.md                  # This file

🔧 Troubleshooting

Common Issues

"GOOGLE_API_KEY not found"

# Set the environment variable
export GOOGLE_API_KEY="your_api_key_here"

# Or add to shell profile
echo 'export GOOGLE_API_KEY="your_api_key"' >> ~/.bashrc

"Playwright browsers not found"

# Install browsers
playwright install

# Install system dependencies (Linux)
playwright install-deps

Windows Permission Issues

# Ensure script execution is enabled
Set-ExecutionPolicy -ExecutionPolicy RemoteSigned -Scope CurrentUser

# Check virtual environment activation
.venv\\Scripts\\activate

WSL Display Issues (for E2E tests)

# Install X11 server for Windows
# Add to ~/.bashrc:
export DISPLAY=:0.0

🤝 Contributing

1. Fork the repository
2. Create a feature branch (git checkout -b feature/amazing-feature)
3. Commit your changes (git commit -m 'Add amazing feature')
4. Push to the branch (git push origin feature/amazing-feature)
5. Open a Pull Request

📄 License

This project is licensed under the MIT License - see the file for details.

🙏 Acknowledgments

• Gemini LLM: Google's powerful language model for intelligent test generation
• Playwright: Cross-browser automation framework
• pytest: Robust Python testing framework
• MCP Protocol: Model Context Protocol for seamless AI integration
• Claude Code: AI-powered development environment