Requirements Analysis Step

Step skill for the docs-orchestrator pipeline. Follows the step skill contract: parse args → discover → fan out → merge → write output.

This skill uses a two-pass architecture to analyze documentation requirements:

1. Discovery pass — A single requirements-discoverer agent enumerates requirements from JIRA, PRs, and specs, producing a JSON skeleton
2. Deep analysis pass — One requirements-analyst agent per requirement, all running in parallel, each performing thorough analysis with a clean context window
3. Merge — The orchestrator assembles per-requirement JSON results into the standard requirements.md format

Arguments

• $1 — JIRA ticket ID (required)
• --base-path <path> — Base output path (e.g., .agent_workspace/proj-123)
• --pr <url> — PR/MR URL to include in analysis (repeatable)
• --repo <path> — Source code repo path (optional, passed to analyst agents for code verification)

Output

<base-path>/requirements/requirements.md
<base-path>/requirements/step-result.json
<base-path>/requirements/discovered_repos.json     (produced by repo extraction, consumed by resolve_source.py)

Execution

1. Parse arguments

Extract the ticket ID, --base-path, any --pr URLs, and optional --repo from the args string.

Set the output path:

OUTPUT_DIR="${BASE_PATH}/requirements"
OUTPUT_FILE="${OUTPUT_DIR}/requirements.md"
DISCOVERY_FILE="${OUTPUT_DIR}/discovery.json"
mkdir -p "$OUTPUT_DIR"

2. Pass 1 — Discovery

Dispatch one requirements-discoverer agent to enumerate requirements from all sources.

Agent:
  subagent_type: docs-skills:requirements-discoverer
  description: "Discover requirements for <TICKET>"
  prompt: |
    Discover documentation requirements for JIRA ticket <TICKET>.

    PR/MR URLs to include in analysis (merge with any auto-discovered, dedup):
    - <PR_URL_1>
    - <PR_URL_2>

    Save your JSON output to: <DISCOVERY_FILE>

    Follow your standard discovery procedure: JIRA fetch, ticket graph traversal,
    PR listing, spec identification, requirement enumeration.

The PR URL bullet list is conditional — include those bullets only if --pr URLs were provided.

After the agent completes, verify the discovery output before proceeding.

2a. Verify discovery output

Path recovery: Check if $DISCOVERY_FILE exists. If not, search for the output at alternate locations:

if [ ! -f "$DISCOVERY_FILE" ]; then
  # Search OUTPUT_DIR and BASE_PATH for any .json the agent may have written
  FOUND=$(find "$OUTPUT_DIR" "$BASE_PATH" -maxdepth 1 -name "*.json" -newer "$OUTPUT_DIR" -type f 2>/dev/null | head -1)
  if [ -n "$FOUND" ]; then
    cp "$FOUND" "$DISCOVERY_FILE"
    echo "WARNING: Discovery agent wrote to wrong path ($FOUND). Copied to $DISCOVERY_FILE."
  fi
fi

If $DISCOVERY_FILE still does not exist after recovery, STOP with error: "Discovery agent did not produce output. Check agent logs."

Format validation: After reading $DISCOVERY_FILE, verify it matches the expected discovery schema:

python3 -c "
import json, sys
d = json.load(open('$DISCOVERY_FILE'))
if 'comments' in d or 'total_comments' in d:
    print('ERROR: Discovery file contains raw JIRA API output, not structured discovery JSON.', file=sys.stderr)
    sys.exit(1)
if 'requirements' not in d:
    print('ERROR: Discovery file missing \"requirements\" key.', file=sys.stderr)
    sys.exit(1)
print('OK')
"

If validation fails (raw JIRA output or missing requirements key), STOP with error including the validation message.

If the discovery JSON has an error field set, STOP and report the error (likely an access failure).

3. Extract discovered repos

After the discoverer agent completes, extract repo/PR URLs from the JIRA graph data. Skip this step if $OUTPUT_DIR/discovered_repos.json already exists (e.g., from a previous run or pre-flight resolution).

if [ ! -f "$OUTPUT_DIR/discovered_repos.json" ]; then
  JIRA_READER="${CLAUDE_PLUGIN_ROOT}/skills/jira-reader/scripts/jira_reader.py"
  python3 "$JIRA_READER" --graph <TICKET> | \
    python3 ${CLAUDE_SKILL_DIR}/scripts/extract_discovered_repos.py \
      --output-dir "$OUTPUT_DIR" \
      --traverse-links "$JIRA_READER" \
      --repo-discovery "$DISCOVERY_FILE"
fi

The --repo-discovery flag reads PR/repo URLs from discovery.json (the sources_consulted.pull_requests array and per-requirement sources with type: "pr"), ensuring repos referenced in JIRA descriptions and comments — but not formally attached as JIRA remote links — are included in discovered_repos.json.

This produces discovered_repos.json in the output directory, which resolve_source.py reads at Priority 4 for automatic repo discovery. If the script fails, log a warning and continue — repo discovery is optional.

4. Parse discovery output

Extract a lightweight summary from the discovery JSON. Do not read the full discovery.json into context — use a bash command to extract only what the orchestrator needs for fan-out dispatch:

python3 -c "
import json
d = json.load(open('<DISCOVERY_FILE>'))
reqs = d.get('requirements', [])
print(json.dumps({
    'count': len(reqs),
    'has_persisted_sources': 'persisted_sources' in d,
    'items': [{'id': r['id'], 'title': r.get('title','')[:60]} for r in reqs]
}))
"

This prints a compact JSON with requirement count, IDs, and truncated titles — enough to dispatch agents and build file paths. The full skeleton (sources, one_line_summary, related_tickets, release, persisted_sources) remains in discovery.json on disk for agents to read directly.

If count is 0, write a minimal requirements.md noting that no requirements were found, write step-result.json, and exit successfully.

5. Pass 2 — Fan out deep analysis

For each requirement in the discovery output, dispatch one requirements-analyst agent. Launch ALL agents in a single message (parallel execution).

Each agent reads its own requirement skeleton from the discovery file on disk and writes its result to a per-requirement JSON file on disk. This keeps the orchestrator's context lean — agent prompts are compact (~0.3KB each) and agent results are one-line confirmations (~0.1KB each).

For each requirement, use:

Agent:
  subagent_type: docs-skills:requirements-analyst
  description: "Analyze REQ-NNN: <title truncated to 40 chars>"
  prompt: |
    Perform deep analysis of a single documentation requirement.

    Read your requirement skeleton from: <DISCOVERY_FILE>
    Extract the entry with id "<REQ_ID>" from the "requirements" array.
    Also read "related_tickets", "release", and "sources_consulted" from the same file.

    [If persisted_sources is present in discovery JSON:]
    The discovery file contains a "persisted_sources" object with file paths
    to pre-fetched source data (comments, specs, diffs). Read from disk
    instead of re-fetching from APIs.

    [If --repo was provided:]
    REPO_PATH: <repo_path>

    Fetch detailed content from each source, perform web search expansion,
    and produce complete documentation requirements with acceptance criteria.

    OUTPUT_FILE: <OUTPUT_DIR>/req-<NNN>.json
    Write your JSON result to the OUTPUT_FILE path above using the Write tool.
    After writing, print ONLY: Written <OUTPUT_DIR>/req-<NNN>.json

The REPO_PATH line is conditional — include it only if --repo was passed to this step.

The persisted_sources note is conditional — include it only if the discovery JSON has a persisted_sources field (detected in step 4's output).

Important: All Agent calls MUST be in a single message so they run in parallel.

6. Collect results from disk

After all agents complete, verify which per-requirement JSON files were written:

ls <OUTPUT_DIR>/req-*.json 2>/dev/null | wc -l

For each expected requirement (from step 4's item list):

1. Check if <OUTPUT_DIR>/req-<NNN>.json exists
2. If the file exists, it will be read by the merge agent in the next step
3. If the file is missing (agent failed or was skipped), the merge agent will create a fallback entry using skeleton data from discovery.json

Log: "<found_count>/<total_count> requirement analysis files written to disk"

7. Assemble requirements.md via merge agent

Delegate the assembly of requirements.md to a merge subagent. This keeps the full markdown content (~30-60KB) out of the orchestrator's context.

Agent:
  description: "Merge requirements for <TICKET>"
  prompt: |
    Assemble a requirements.md document from per-requirement analysis files.

    DISCOVERY_FILE: <DISCOVERY_FILE>
    OUTPUT_DIR: <OUTPUT_DIR>
    OUTPUT_FILE: <OUTPUT_FILE>
    EXPECTED_REQUIREMENTS: <comma-separated list of REQ IDs from step 4>

    Instructions:
    1. Read <DISCOVERY_FILE> for metadata: ticket_summary, release, sources_consulted, related_tickets
    2. For each expected requirement ID, read <OUTPUT_DIR>/req-<NNN>.json
       - If a file is missing, create a fallback entry from the skeleton in discovery.json:
         {"id": "<REQ-NNN>", "title": "<from skeleton>", "error": "Agent did not return valid JSON",
          "priority": "<from skeleton>", "category": "<from skeleton>",
          "sources": [{"label": "<source.key or source.url>", "url": "<source.url>", "note": "From discovery (deep analysis failed)"}],
          "summary": "<one_line_summary from skeleton>", "user_impact": null, "scope": null,
          "documentation_actions": [], "acceptance_criteria": [], "references": [],
          "web_findings": [], "is_breaking_change": false, "deprecation_version": null,
          "notes": "Deep analysis failed — using skeleton data only"}
    3. Collect all per-requirement results ordered by ID
    4. Assemble <OUTPUT_FILE> using the format contract below
    5. After writing, print ONLY: Written <OUTPUT_FILE>

    FORMAT CONTRACT — the document structure must match this exactly:

    # Documentation Requirements

    **Source**: <ticket_summary from discovery>
    **Date**: <YYYY-MM-DD>
    **Release/Sprint**: <release from discovery>

    ## Summary

    - Total requirements analyzed: <count>
    - New modules needed: <count documentation_actions with action "Create">
    - Existing modules to update: <count documentation_actions with action "Update">
    - Breaking changes requiring docs: <count where is_breaking_change is true>

    ## Requirements by priority

    ### Critical

    #### REQ-001: [title]
    - **Source**: [label](url) | [label](url)
    - **Summary**: [summary]
    - **User impact**: [user_impact]
    - **Documentation action**:
      - [ ] [action] `[file]` ([type]) [note if present]
    - **Acceptance criteria**:
      - [ ] [criterion]
    - **References**:
      - [label](url): [note]

    ### High
    [Same format, requirements with priority "high"]

    ### Medium
    [Same format, requirements with priority "medium"]

    ### Low
    [Same format, requirements with priority "low"]

    ## Documentation scope

    ### New documentation needed

    | Requirement | Scope | References |
    |-------------|-------|------------|
    | REQ-XXX | [From documentation_actions where action is "Create"] | [source labels] |

    ### Existing documentation to update

    | Requirement | What changed | References |
    |-------------|-------------|------------|
    | REQ-XXX | [From documentation_actions where action is "Update"] | [source labels] |

    ## Breaking changes

    [Table of requirements where is_breaking_change is true. Omit section if none.]

    | Change | Migration steps needed | Deprecation notice | References |
    |--------|------------------------|-------------------|------------|

    ## Notes

    [Aggregate any non-null notes from requirements. Omit section if none.]

    ## Related tickets

    [Format related_tickets from discovery output. Omit section if empty.]

    ## Sources consulted

    ### JIRA tickets
    [From sources_consulted.jira_tickets — deduplicated across all requirements]

    ### Pull requests / Merge requests
    [From sources_consulted.pull_requests — deduplicated]

    ### Code files
    [From references with type "code" across all requirements — deduplicated]

    ### Existing documentation
    [From sources_consulted.existing_docs — deduplicated]

    ### External references
    [From references without type "code" that are not JIRA/PR/web_findings — deduplicated]

    ### Web search findings
    [From web_findings across all requirements — deduplicated by URL]

    RULES:
    - Only include priority sections that have requirements (omit empty ### High if none)
    - Requirements with errors: include under their original priority with: **Note:** Deep analysis failed for this requirement. Skeleton data only.
    - Deduplicate sources consulted and references by URL or file path
    - Convert skeleton sources to analyst format: use key (for JIRA) or URL as label, preserve URL, add note "From discovery (deep analysis failed)"

8. Write step-result.json

Do not hand-author the sidecar — a hand-written sidecar drifts from the schema and uses an orchestrator-delayed timestamp instead of a real wall-clock one. Run the script, passing the requirement_count from the discovery output parsed in step 4:

python3 ${CLAUDE_SKILL_DIR}/scripts/write_step_result.py \
  --ticket "<TICKET>" \
  --output-file "<OUTPUT_FILE>" \
  --requirement-count <count from step 4> \
  --sidecar "<OUTPUT_DIR>/step-result.json"

The script extracts the title from <OUTPUT_FILE>, writes the conformant step-result.json with a real wall-clock completed_at. If the script exits non-zero, fix the arguments and re-run; do not substitute a stub.

9. Verify output

Verify that <OUTPUT_FILE> and <OUTPUT_DIR>/step-result.json exist.

Notes

• Two-pass architecture: Pass 1 (discovery) enumerates requirements; pass 2 fans out one agent per requirement for isolated deep analysis
• Disk-based data flow: Agents write per-requirement JSON to disk; the merge agent reads from disk to assemble requirements.md, keeping the orchestrator context lean
• Error isolation: A failed analyst agent does not block others — the merge agent falls back to skeleton data from discovery.json
• Repo discovery: The extraction script produces discovered_repos.json from the JIRA graph for resolve_source.py Priority 4 auto-discovery