Comparison Report Workflow

When to Use This Skill

• User wants to compare multiple test runs side-by-side
• User mentions comparison report, multi-run comparison, or test run comparison
• User has 2+ test runs that have completed the E2E performance testing workflow
• User wants comparison charts (CPU, Memory) across test runs

Reference

This section provides context for humans and capable models. For the step-by-step execution instructions, skip to the Execution section below.

What This Workflow Does

Creates comparison reports that analyze multiple test runs side-by-side, generates comparison bar charts, and publishes the results to Confluence. This workflow is designed to run AFTER the end-to-end performance testing workflow (.cursor/skills/performance-testing-workflow/SKILL.md) has been completed for each individual test run.

The workflow steps:

1. Validate Prerequisites — Verify all test runs have completed the E2E workflow
2. PerfReport MCP — Generate comparison report and comparison charts
3. Confluence MCP (Optional) — Publish comparison report to Confluence
4. AI Revision (Optional) — Enhance the report with AI-generated insights

comparison_id vs run_id

This is a critical distinction:

• comparison_id is a timestamp-based unique identifier generated when create_comparison_report runs (e.g., "2026-01-21-14-27-42")
• comparison_id is NOT the same as any individual run_id
• When publishing to Confluence, use comparison_id as the test_run_id parameter
• report_type: "comparison" tells all tools to look in artifacts/comparisons/{comparison_id}/

Artifact Path Structure

artifacts/comparisons/{comparison_id}/
├── comparison_report_{run_ids}.md               # Generated comparison report
├── comparison_report_{run_ids}_original.md      # Backup (after AI revision)
├── comparison_report_{run_ids}_revised.md       # AI-revised report
├── comparison_metadata_{run_ids}.json           # Metadata
├── comparison_metadata_{run_ids}_original.json  # Backup metadata
├── charts/                                      # Comparison bar charts
│   ├── CPU_PEAK_CORE_COMPARISON_BAR-{resource}.png
│   ├── CPU_AVG_CORE_COMPARISON_BAR-{resource}.png
│   ├── MEMORY_PEAK_USAGE_COMPARISON_BAR-{resource}.png
│   └── MEMORY_AVG_USAGE_COMPARISON_BAR-{resource}.png
└── revisions/                                   # AI revision files
    ├── AI_EXECUTIVE_SUMMARY_v1.md
    ├── AI_KEY_FINDINGS_BULLETS_v1.md
    └── AI_ISSUES_SUMMARY_v1.md

Tool Reference

PerfReport MCP Tools

Confluence MCP Tools

Comparison Chart IDs

Chart configuration:

• CPU charts display in "millicores" or "cores" (configured in chart_schema.yaml)
• Memory charts display in "MB" or "GB" (configured in chart_schema.yaml)
• Comparison charts use the navy-blue gradient color palette from chart_colors.yaml
• Each test run gets a distinct color from light to dark blue

Related Rules

• prerequisites.mdc — test_run_id and artifact structure validation
• skill-execution-rules.mdc — Follow steps in order, collect inputs first, do not skip
• mcp-error-handling.mdc — MCP tool error handling (retry policy, reporting format)

Downstream Workflows

After completing this workflow, the user can proceed to:

• AI HITL Report Revision — .cursor/skills/report-revision-workflow/SKILL.md with report_type: "comparison" and comparison_id as the run_id.

Important Notes

• Prerequisites are critical: Each test run must have completed the full E2E workflow before it can be included in a comparison.
• Recommended: 2-5 test runs for optimal readability. Maximum 10 supported.
• Custom templates: Available via list_templates. Custom templates should include comparison-specific placeholders (see docs/report_template_guidelines.md).
• Unit configuration: CPU in "cores" or "millicores", Memory in "GB" or "MB" (configured in report_config.yaml and chart_schema.yaml).

Execution

Follow these steps exactly, in order. Each step has one or more actions.

Collect Inputs

Ask the user for the following values. Do not proceed until all required values are collected.

REQUIRED:
  run_id_list = [list of 2-5 test run IDs — e.g., ["80593110", "80603131", "80612241"]]

OPTIONAL:
  env_name  = [environment name for chart filtering]
  template  = [custom comparison template name — default: "default_comparison_report_template.md"]

CONDITIONAL (only if publishing to Confluence):
  confluence_mode  = ["cloud" or "onprem"]
  confluence_space = [space name — e.g., "Quality Engineering"]
  parent_page_name = [e.g., "AI Generated Test Reports"]
  parent_page_id   = [optional — e.g., "123456789"]

If the user provides Confluence information, assume they want to publish results.

Step 1 — Validate Prerequisites

Input: run_id_list

Action: For each run_id in the list, verify these files exist:

• artifacts/{run_id}/reports/report_metadata_{run_id}.json
• artifacts/{run_id}/analysis/infrastructure_analysis.json

If any files are missing:

• Report which run_id(s) are missing prerequisites
• Advise the user to run the E2E performance testing workflow for those runs first (.cursor/skills/performance-testing-workflow/SKILL.md)
• Do NOT proceed until all prerequisites are met

Step 2 — Create Comparison Report

Input: run_id_list, template (optional)

Action: Call MCP tool create_comparison_report

create_comparison_report(
  run_id_list = {run_id_list},
  template    = {template},
  format      = "md"
)

Expected response:

{
  "comparison_id": "2026-01-21-14-27-42",
  "report_path": "artifacts/comparisons/{comparison_id}/comparison_report_{run_ids}.md",
  "metadata_path": "artifacts/comparisons/{comparison_id}/comparison_metadata_{run_ids}.json"
}

Save:

• comparison_id = the generated comparison identifier (used as test_run_id in all subsequent steps)
• report_path = path to the generated report

On error: These are Python code executions. Do NOT retry. Report error to user.

Step 3 — Create Comparison Charts

Input: comparison_id (from Step 2), run_id_list, env_name (optional)

Action: First, list available chart types:

list_chart_types()

Look for chart IDs containing "COMPARISON".

Then create all 4 comparison charts:

create_comparison_chart(
  comparison_id = {comparison_id},
  run_id_list   = {run_id_list},
  chart_id      = "CPU_PEAK_CORE_COMPARISON_BAR",
  env_name      = {env_name}
)

create_comparison_chart(
  comparison_id = {comparison_id},
  run_id_list   = {run_id_list},
  chart_id      = "CPU_AVG_CORE_COMPARISON_BAR",
  env_name      = {env_name}
)

create_comparison_chart(
  comparison_id = {comparison_id},
  run_id_list   = {run_id_list},
  chart_id      = "MEMORY_PEAK_USAGE_COMPARISON_BAR",
  env_name      = {env_name}
)

create_comparison_chart(
  comparison_id = {comparison_id},
  run_id_list   = {run_id_list},
  chart_id      = "MEMORY_AVG_USAGE_COMPARISON_BAR",
  env_name      = {env_name}
)

Charts are saved to: artifacts/comparisons/{comparison_id}/charts/

On error: These are Python code executions. Do NOT retry. Report error to user.

Validation

Verify these files exist:

• artifacts/comparisons/{comparison_id}/comparison_report_{run_ids}.md
• artifacts/comparisons/{comparison_id}/comparison_metadata_{run_ids}.json
• artifacts/comparisons/{comparison_id}/charts/CPU_PEAK_CORE_COMPARISON_BAR-*.png
• artifacts/comparisons/{comparison_id}/charts/CPU_AVG_CORE_COMPARISON_BAR-*.png
• artifacts/comparisons/{comparison_id}/charts/MEMORY_PEAK_USAGE_COMPARISON_BAR-*.png
• artifacts/comparisons/{comparison_id}/charts/MEMORY_AVG_USAGE_COMPARISON_BAR-*.png

Step 4 — Confluence Publishing (Optional)

When: Only execute if Confluence details were provided in Collect Inputs.

Prerequisites:

• comparison_id from Step 2
• run_id_list
• confluence_mode, confluence_space, parent_page_name or parent_page_id

4a. Locate Space and Parent Page

list_spaces(
  mode = {confluence_mode}
)

Search for the space matching confluence_space. Extract space_ref.

Locate parent page:

If parent_page_id is provided:

get_page_by_id(
  page_id = {parent_page_id},
  mode    = {confluence_mode}
)

If parent_page_name is provided:

list_pages(
  space_ref = {space_ref},
  mode      = {confluence_mode}
)

Search for the page matching parent_page_name.

Save: parent_id = the parent page ID.

4b. Get Available Reports

get_available_reports(
  test_run_id = {comparison_id}
)

Select the report filename (e.g., comparison_report_{run_ids}.md).

Save: report_filename.

4c. Create Confluence Page

create_page(
  space_ref   = {space_ref},
  test_run_id = {comparison_id},
  filename    = {report_filename},
  mode        = {confluence_mode},
  parent_id   = {parent_id},
  report_type = "comparison",
  title       = {optional custom title}
)

Important: Use comparison_id as test_run_id and report_type: "comparison".

Save: page_ref = returned page ID.

4d. Attach Chart Images

attach_images(
  page_ref    = {page_ref},
  test_run_id = {comparison_id},
  mode        = {confluence_mode},
  report_type = "comparison"
)

Uploads ALL PNG chart images from artifacts/comparisons/{comparison_id}/charts/. Check response for attached/failed counts. Continue even if some fail.

4e. Update Page with Embedded Images

update_page(
  page_ref    = {page_ref},
  test_run_id = {comparison_id},
  mode        = {confluence_mode},
  report_type = "comparison"
)

Replaces {{CHART_PLACEHOLDER: ID}} markers with embedded <ac:image> markup. Check response for placeholders_replaced and placeholders_remaining.

Expected placeholders: CPU_PEAK_CORE_COMPARISON_BAR, CPU_AVG_CORE_COMPARISON_BAR, MEMORY_PEAK_USAGE_COMPARISON_BAR, MEMORY_AVG_USAGE_COMPARISON_BAR.

4f. Validation

Verify:

• Page created successfully (check for page_ref and URL)
• Images attached (check status: "success" or "partial")
• Placeholders replaced (check placeholders_replaced list)

On error: If API calls fail, retry up to 3 times. Wait 5-10 seconds between retries.

Step 5 — Generate Summary

Action: Present a summary to the user.

Comparison Report Details

• Comparison ID: {comparison_id}
• Test Runs Compared: {run_id_list}
• Template Used: {template_name}
• Report Path: artifacts/comparisons/{comparison_id}/comparison_report_{run_ids}.md

Charts Generated

Confluence Publishing (if applicable)

• Page Title: {page_title}
• Page URL: {confluence_url}
• Images Attached: {count}
• Placeholders Replaced: {list}

Next Steps

Ask the user:

• "Would you like to enhance the comparison report with AI-generated insights?" — If yes, follow the skill at .cursor/skills/report-revision-workflow/SKILL.md with report_type: "comparison" and comparison_id as the run_id.

Error Handling

These rules apply to every step:

• PerfReport MCP tools (create_comparison_report, create_comparison_chart): These are Python code executions. Do NOT retry on failure. Report the error with: error message, missing file paths, expected vs. actual structure. Do NOT attempt to fix code or modify files.
• Confluence MCP tools (list_spaces, create_page, attach_images, update_page): These are API calls. Retry up to 3 times. Wait 5-10 seconds between retries.
• Do not proceed to the next step if the current step failed.
• Do NOT write code to fix MCP tool issues.
• Ask the user for next steps on any error.