mcp-server-automation

Arm63/mcp-server-automation

3.1

If you are the rightful owner of mcp-server-automation and would like to certify it and/or have it hosted online, please leave a comment on the right or send an email to henry@mcphub.com.

AutotestGen MCP is a minimal pipeline that automates the conversion of manual test cases from Testiny into executable automated tests using Anthropic's AI, runnable with Appium/XCUITest for iOS or Playwright.

AutotestGen MCP — Testiny → AI → Automated Tests (MVP)

Minimal pipeline that turns manual test cases from Testiny into executable automated tests, generated by Anthropic and runnable with Appium/XCUITest (iOS).

Flow

  • Testiny stores manual test cases
  • API endpoint fetches a case via TESTINY_* creds
  • Anthropic (CLAUDE_*) generates test code from the prompt
  • Script is saved under output/generated/<target>/...
  • Optional: run immediately (Appium/Pytest)

Requirements

  • Node.js 18+
  • Python 3.13+
  • Xcode + iOS real-device provisioning (for Appium iOS)
  • Appium v2 (we use local, project-scoped install via npx)

Environment

Create .env with:

  • Testiny

    • TESTINY_API_URL (e.g., https://api.testiny.io/v1)
    • TESTINY_API_KEY
    • TESTINY_PROJECT_ID (optional but recommended)

  • Anthropic

    • CLAUDE_API_KEY
    • CLAUDE_MODEL (optional, default claude-3-haiku-20240307)

  • Server

    • PORT (optional, default 3000)

Install & Run the MCP server

npm install
npm run dev
# Server: http://localhost:3000 (health: GET /)

Generate a test from Testiny

Endpoint: GET /api/test-cases/:id/generate

Query params:

  • target: pytest-appium-ios (default) | appium-js
  • save: true|false (default true)
  • run: true|false (default false)
  • force: bypass label check (default false)
  • mock: bypass AI and return a simple heuristic test (default false)
  • allowFallback: use simple test when AI credits are low (default true)
  • raw: include raw Testiny payload and debug (default false)

Example:

curl "http://localhost:3000/api/test-cases/TC-2/generate?target=pytest-appium-ios&save=true&run=false"

Output files:

  • output/generated/pytest-appium-ios/test_TC_<id>_pytest_appium_ios.py

Label gate:

  • Unless force=true, the case must include one of: automation:required or automation:recommended.

iOS (Pytest + Appium + XCUITest)

Python deps (Poetry uses pyproject.toml):

poetry env use 3.13
poetry install

Project-scoped Appium v2 and XCUITest driver:

# Install XCUITest driver locally (stored under ./.appium)
APPIUM_HOME="$PWD/.appium" npx --yes appium driver install xcuitest

# Start Appium on /wd/hub (keep running)
APPIUM_HOME="$PWD/.appium" npx --yes appium --base-path /wd/hub

Real device env (examples):

  • APPIUM_URL (default http://127.0.0.1:4723)
  • IOS_UDID (or DEVICE_UDID)
  • IOS_DEVICE_NAME (or DEVICE_NAME)
  • IOS_PLATFORM_VERSION (or PLATFORM_VERSION)
  • One of: BUNDLE_ID (installed app) OR APP (absolute path to .app/.ipa)
  • Optional for signing: TEAM_ID
  • Test data: TEST_EMAIL (optional)

Run a generated test (example):

APPIUM_URL="http://127.0.0.1:4723" \
IOS_UDID="<your-device-udid>" \
IOS_DEVICE_NAME="iPhone 12" \
IOS_PLATFORM_VERSION="18.0" \
BUNDLE_ID="com.example.app" \
TEAM_ID="<your-apple-team-id>" \
poetry run pytest -q "$(pwd)/output/generated/pytest-appium-ios/test_TC_2_pytest_appium_ios.py" -k test_tc_2 -s

Notes:

  • Tests are written using appium-python-client 5.x and XCUITestOptions API.
  • Capabilities include helpful flags for first-time WDA install: showXcodeLog, useNewWDA.

Smart iOS helpers (no accessibility IDs needed)

Manual QA steps often reference visible text rather than accessibility identifiers. Use src/utils/smart_ios.py to act on elements by human-readable labels:

from src.utils.smart_ios import click_by_text, enter_text_by_label, wait_for_text

click_by_text(driver, "Sign In")
enter_text_by_label(driver, "Email or Phone Number", "qa@example.com")
wait_for_text(driver, "Password")

Under the hood, helpers use iOS predicate and class-chain locators, fuzzy matching, and spatial heuristics to find text fields near labels.

Real-device signing and stability env (recommended)

Provide these env vars to avoid xcodebuild code 70 and WDA issues:

  • TEAM_ID (or XCODE_ORG_ID)
  • XCODE_SIGNING_ID (default Apple Development)
  • UPDATED_WDA_BUNDLE_ID (e.g., com.company.WebDriverAgentRunner)

Extras enabled: showXcodeLog=true, useNewWDA=true, waitForQuiescence=false, WDA startup retries.

Appium base-path handling

Tests automatically fall back between APPIUM_URL with and without /wd/hub when the server returns 404 "unknown command". You can set either:

  • APPIUM_URL="http://127.0.0.1:4723" (no base-path)
  • APPIUM_URL="http://127.0.0.1:4723/wd/hub" (with base-path)

How this works (iOS real device)

  • Start Appium v2 (project-scoped) on /wd/hub:
APPIUM_HOME="$PWD/.appium" npx --yes appium --base-path /wd/hub
  • Start MCP server (in another terminal):
PORT=3001 npm run dev
  • Generate and save a Testiny case as pytest-appium-ios (example TC-4):
curl "http://localhost:3001/api/test-cases/TC-4/generate?target=pytest-appium-ios&save=true&run=false&force=true"
  • Run the generated test on a real device (replace with your values):
PYTHONPATH="$(pwd)" \
APPIUM_URL="http://127.0.0.1:4723/wd/hub" \
IOS_UDID="00008101-001D35A23C0A001E" \
IOS_DEVICE_NAME="iPhone 12" \
IOS_PLATFORM_VERSION="18.5" \
APP="/absolute/path/to/YourApp.ipa" \
BUNDLE_ID="com.example.app" \
TEAM_ID="<your-team-id>" \
UPDATED_WDA_BUNDLE_ID="<your-team-id>.WebDriverAgentRunner" \
TEST_EMAIL="qa@example.com" \
poetry run pytest -q "$(pwd)/output/generated/pytest-appium-ios/test_TC_4_pytest_appium_ios.py" -k test_tc_4 -s

Notes:

  • Use APP to install the .ipa on first run; BUNDLE_ID to attach to an installed app.
  • Signing caps are picked from env; tests use XCUITestOptions and autoAcceptAlerts.
  • Helpers from src/utils/smart_ios.py enable robust text-based actions.

Troubleshooting

  • ModuleNotFoundError: No module named 'appium' → run poetry install, then run tests with poetry run pytest ....
  • xcodebuild failed with code 70 → ensure real device provisioning, valid TEAM_ID, trust developer, correct signing cert, device unlocked.
  • EADDRINUSE: 0.0.0.0:4723 → another Appium is running; stop it or use a different port.
  • Element not found → use robust iOS locators (predicate/class chain); confirm accessibility identifiers in the app.

Project layout (key parts)

  • src/server.ts: Express server, mounts /api
  • src/routes/test-cases.ts: Generate endpoint, AI fallback logic
  • src/services/testiny.service.ts: Testiny fetch + step extraction
  • src/services/ai.service.ts: Anthropic call & code extraction
  • src/services/prompt.service.ts: Prompt templates per target
  • src/services/runner.service.ts: Save file, optional run helper
  • output/generated/: Generated tests

Security & costs

  • Anthropic usage requires funded account/credits.
  • Keep API keys in .env; do not commit.