Integration Tester

Comprehensive integration testing skill for FORGE projects. Tests APIs, databases, external services, webhooks, message queues, and end-to-end user flows with intelligent retry logic and detailed failure analysis.

When to Use

• Pre-deployment validation - Verify all integrations work before shipping
• After dependency updates - Ensure updates don't break integrations
• Debugging production issues - Isolate integration failures
• CI/CD pipelines - Automated integration testing
• New service onboarding - Validate new integrations

Quick Start

# Test API endpoint
/test api --endpoint /api/v1/users --auth bearer

# Test database connection
/test db --url postgresql://user:pass@localhost/db

# Test external service
/test stripe --secret-key sk_test_xxx

# Test webhook endpoint
/test webhook --url https://api.example.com/webhooks --secret whsec_xxx

# Test message queue
/test queue --broker redis://localhost:6379

# Run E2E flow test
/test e2e --flow signup-to-payment

Features

1. API Integration Testing

Test REST/GraphQL endpoints with authentication, rate limits, and response validation.

# Basic endpoint test
/test api --endpoint /api/v1/health

# With authentication
/test api --endpoint /api/v1/users --auth bearer --token $API_TOKEN
/test api --endpoint /api/v1/admin --auth basic --user admin --pass secret
/test api --endpoint /api/v1/webhook --auth api-key --key x-api-key

# Full request test
/test api \
  --endpoint /api/v1/users \
  --method POST \
  --auth bearer \
  --token $TOKEN \
  --body '{"email":"test@example.com"}' \
  --expect-status 201 \
  --expect-field "id"

# Rate limit testing
/test api --endpoint /api/v1/data --rate-limit-test --requests 100

Validation Options:

2. Database Integration Testing

Test database connections, migrations, and query performance.

# Connection test
/test db --url postgresql://user:pass@localhost/mydb

# With migration check
/test db --url $DATABASE_URL --migrations --alembic

# Performance test
/test db --url $DATABASE_URL --query "SELECT * FROM users" --max-time 100

# Full validation
/test db \
  --url postgresql://user:pass@localhost/mydb \
  --migrations \
  --test-queries \
  --connection-pool

Database Support:

• PostgreSQL (with asyncpg)
• MySQL (with aiomysql)
• SQLite
• Redis
• MongoDB

3. External Service Testing

Test third-party integrations (Stripe, OpenAI, AssemblyAI, etc.)

# Stripe
/test stripe --secret-key sk_test_xxx --webhook-secret whsec_xxx
/test stripe --test-mode --test-charge --test-customer

# OpenAI
/test openai --api-key $OPENAI_KEY --model gpt-4 --test-completion

# AssemblyAI
/test assemblyai --api-key $AAI_KEY --test-transcription

# Generic HTTP service
/test service --name "My API" --url https://api.example.com/health

Service-Specific Tests:

4. Webhook Testing

Receive and validate webhooks with signature verification.

# Start webhook listener
/test webhook --port 8080 --secret whsec_xxx --provider stripe

# Test existing endpoint
/test webhook --url https://api.example.com/webhooks \
  --secret whsec_xxx \
  --send-test \
  --provider stripe

# With custom payload
/test webhook --url https://api.example.com/webhooks \
  --payload '{"event":"test"}' \
  --signature-header "X-Signature"

Supported Providers:

• Stripe (signature verification)
• GitHub (signature verification)
• Shopify (HMAC verification)
• Custom (configurable)

5. Message Queue Testing

Test Redis, Celery, and other message queues.

# Redis
/test queue --broker redis://localhost:6379 --backend redis

# Celery with Redis
/test queue --broker redis://localhost:6379 \
  --backend redis://localhost:6379/0 \
  --celery \
  --test-task tasks.add

# Test task execution
/test queue --broker redis://localhost:6379 \
  --send-task test.echo \
  --args '{"message":"hello"}' \
  --expect-result "hello"

Queue Tests:

• Connection establishment
• Message publish/subscribe
• Task enqueue and execution
• Result backend
• Dead letter queue

6. E2E Flow Testing

Test complete user journeys across multiple services.

# Run predefined flow
/test e2e --flow signup-to-payment

# Custom flow from file
/test e2e --file ./flows/onboarding.json

# With specific environment
/test e2e --flow signup-to-payment --env staging

Predefined Flows:

Usage Reference

Command Syntax

/test <type> [options]

Types:
  api         Test API endpoints
  db          Test database connections
  stripe      Test Stripe integration
  openai      Test OpenAI integration
  assemblyai  Test AssemblyAI integration
  webhook     Test webhook endpoints
  queue       Test message queues
  e2e         Run end-to-end flow tests
  all         Run all integration tests

Global Options

Environment Variables

# API Testing
TEST_API_BASE_URL=https://api.example.com
TEST_API_TOKEN=xxx

# Database
TEST_DATABASE_URL=postgresql://...

# External Services
STRIPE_SECRET_KEY=sk_test_xxx
STRIPE_WEBHOOK_SECRET=whsec_xxx
OPENAI_API_KEY=sk-xxx
ASSEMBLYAI_API_KEY=xxx

# Webhook Testing
WEBHOOK_SECRET=whsec_xxx
WEBHOOK_PORT=8080

# Queue Testing
REDIS_URL=redis://localhost:6379
CELERY_BROKER_URL=redis://localhost:6379

Test Result Reporting

Console Output

╔════════════════════════════════════════════════════════════╗
║           INTEGRATION TEST RESULTS                         ║
╠════════════════════════════════════════════════════════════╣
║ Suite: API Integration Tests                               ║
║ Duration: 4.23s                                            ║
║ Timestamp: 2024-01-15T10:30:00Z                            ║
╠════════════════════════════════════════════════════════════╣
║ ✅ Health Endpoint                                         ║
║    GET /api/v1/health                                      ║
║    Status: 200 OK (45ms)                                   ║
║                                                            ║
║ ✅ User List (Authenticated)                               ║
║    GET /api/v1/users                                       ║
║    Status: 200 OK (120ms)                                  ║
║    Response: {"count": 42, "users": [...]}                 ║
║                                                            ║
║ ❌ Create User (Rate Limited)                              ║
║    POST /api/v1/users                                      ║
║    Status: 429 Too Many Requests (523ms)                   ║
║    Error: Rate limit exceeded, retry after 60s             ║
╠════════════════════════════════════════════════════════════╣
║ RESULTS: 2 passed, 1 failed, 0 skipped                     ║
║ Success Rate: 66.7%                                        ║
╚════════════════════════════════════════════════════════════╝

JSON Output

{
  "suite": "API Integration Tests",
  "timestamp": "2024-01-15T10:30:00Z",
  "duration": 4.23,
  "summary": {
    "total": 3,
    "passed": 2,
    "failed": 1,
    "skipped": 0
  },
  "tests": [
    {
      "name": "Health Endpoint",
      "status": "passed",
      "duration": 0.045,
      "request": {
        "method": "GET",
        "url": "/api/v1/health"
      },
      "response": {
        "status": 200,
        "latency_ms": 45
      }
    },
    {
      "name": "Create User",
      "status": "failed",
      "duration": 0.523,
      "error": "Rate limit exceeded",
      "suggestion": "Implement exponential backoff for 429 responses"
    }
  ]
}

Failure Analysis

Automatic Diagnosis

When a test fails, the skill analyzes the failure and suggests fixes:

❌ Database Connection Failed
   Error: connection refused
   
   Diagnosis:
   - Database server not running
   - Incorrect connection string
   - Firewall blocking connection
   
   Suggested Fixes:
   1. Start PostgreSQL: docker-compose up -d db
   2. Check connection string in .env
   3. Verify port 5432 is accessible

Common Issues & Solutions

Retry Logic

Default Retry Policy

retry_policy = {
    "max_attempts": 3,
    "backoff_factor": 2,
    "initial_delay": 1,
    "max_delay": 30,
    "retry_on": [408, 429, 500, 502, 503, 504],
    "retry_exceptions": ["ConnectionError", "TimeoutError"]
}

Custom Retry Configuration

# More aggressive retry
/test api --endpoint /api/v1/data \
  --retry 5 \
  --retry-delay 3 \
  --max-delay 60

# No retry for specific test
/test api --endpoint /api/v1/health --retry 0

Configuration File

Create .integration-test.yml in project root:

# Default settings
defaults:
  retry: 3
  timeout: 30
  verbose: false

# API base URLs by environment
environments:
  development:
    api_base_url: http://localhost:8000
    database_url: postgresql://localhost/dev
  staging:
    api_base_url: https://staging-api.example.com
    database_url: ${STAGING_DATABASE_URL}
  production:
    api_base_url: https://api.example.com

# Predefined test suites
suites:
  smoke:
    - type: api
      endpoint: /api/v1/health
    - type: db
      url: ${DATABASE_URL}
  
  full:
    - type: api
      endpoint: /api/v1/health
    - type: db
      url: ${DATABASE_URL}
      migrations: true
    - type: stripe
      secret_key: ${STRIPE_SECRET_KEY}
    - type: queue
      broker: ${REDIS_URL}

# Custom flows
flows:
  signup-to-payment:
    steps:
      - action: api
        endpoint: /api/v1/auth/signup
        method: POST
        body: '{"email":"test@example.com"}'
      - action: api
        endpoint: /api/v1/payments/checkout
        method: POST
        depends_on: signup

Integration with CI/CD

GitHub Actions

name: Integration Tests

on: [push, pull_request]

jobs:
  test:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      
      - name: Run Integration Tests
        run: |
          /test all --json --output results.json
        env:
          DATABASE_URL: postgresql://test:test@localhost/test
          STRIPE_SECRET_KEY: ${{ secrets.STRIPE_TEST_KEY }}
      
      - name: Upload Results
        uses: actions/upload-artifact@v4
        with:
          name: test-results
          path: results.json

Makefile Integration

.PHONY: test-integration test-api test-db test-e2e

test-integration:
	@test all --fail-fast

test-api:
	@test api --endpoint /api/v1/health

test-db:
	@test db --url ${DATABASE_URL}

test-e2e:
	@test e2e --flow signup-to-payment

Scripts Reference

Prompts Reference

Workflow

1. Pre-test Setup
   • Load configuration from .integration-test.yml or environment
   • Validate all required environment variables
   • Check service availability
2. Test Execution
   • Run tests with retry logic
   • Collect metrics (latency, status codes)
   • Capture errors and stack traces
3. Result Analysis
   • Aggregate test results
   • Generate failure analysis
   • Suggest fixes for failures
4. Reporting
   • Output results (console/JSON/file)
   • Update CI status
   • Notify on failures (optional)

Checklist

• Configuration file created or environment variables set
• All required services are running
• Test credentials/secrets are configured
• Tests pass in development environment
• Tests pass in staging environment
• Failure analysis provides actionable insights
• Results are documented/output properly

Examples

Complete API Test Suite

#!/bin/bash

# Health checks
/test api --endpoint /api/v1/health --expect-status 200
/test api --endpoint /api/v1/ready --expect-status 200

# Auth endpoints
/test api \
  --endpoint /api/v1/auth/login \
  --method POST \
  --body '{"email":"test@test.com","password":"test"}' \
  --expect-status 200 \
  --expect-field "token"

# Protected endpoints
/test api \
  --endpoint /api/v1/users/me \
  --auth bearer \
  --token $TEST_TOKEN \
  --expect-field "id"

# Rate limiting
/test api --endpoint /api/v1/public --rate-limit-test --requests 50

Complete Database Test

#!/bin/bash

/test db \
  --url ${DATABASE_URL} \
  --migrations \
  --connection-pool \
  --test-queries \
  --performance-test

CI/CD Integration Test

#!/bin/bash
set -e

echo "Running integration tests..."

# Smoke tests first
/test api --endpoint /api/v1/health --fail-fast

# Full suite
/test all \
  --retry 3 \
  --json \
  --output integration-results.json

# Check results
if [ $(jq '.summary.failed' integration-results.json) -gt 0 ]; then
  echo "Integration tests failed!"
  jq '.tests[] | select(.status == "failed")' integration-results.json
  exit 1
fi

echo "All integration tests passed!"

Remember: Integration tests verify that your services work together. Run them before deployment and monitor them in production for early issue detection.