CI cost optimizer

Identify actionable opportunities to reduce continuous integration wall-clock duration or compute spend using empirical run history. Preserve full test coverage, platform matrices, and release safeguards, proposing changes only as reviewable diffs after authorization.

Triggers

• CI runtime, queue latency, or compute spend has grown substantially or exceeds team maintenance targets.
• A maintainer requests analyzing CI run records to uncover bottlenecks and propose cost-effective optimizations.
• A proposed pipeline optimization needs an evidence-based comparison of runtime before and after implementation.

Non-triggers

• Removing tests, reducing assertions, skipping failure checks, or lowering test coverage to artificially improve metrics.
• Pruning required operating systems, architectures, or language versions from the platform matrix without explicit stakeholder approval.
• Weakening security scans, compliance gates, linters, release verification, or operational safety checks.
• Directly modifying live CI configuration, credentials, runner instances, or billing limits (treat live CI systems as read-only).
• Estimating savings without representative historical run records.

Process

1. Establish scope and authority. Identify the repository, CI platform (such as GitHub Actions or GitLab CI), target workflows or jobs, date range, and available cost or timing metrics. Inspect workflow files and historical data read-only. Do not expose secrets or alter live settings. Confirm that test coverage and platform matrix requirements remain strictly in scope and cannot be bypassed.
2. Collect representative evidence. Gather recent successful and failed run records across a representative operational window. Record sample sizes, workflow/job durations, queue time versus execution time, runner specifications, platform matrix jobs, and billable compute units. Separate queue delays from active compute time. Account for cache cold/warm states, retries, concurrency limits, and runner variance.
3. Diagnose bottlenecks before proposing changes. Trace long-running jobs to specific steps and dependencies: redundant environment setup, repeated dependency downloads, uncached compilation layers, unnecessary serialization, oversized test shards, or overly broad workflow trigger rules. Validate suspected causes through logs, step timings, or repeated runs. Label unsupported explanations as hypotheses.
4. Rank proposals by estimated daily savings. For each candidate:
   • Identify the specific bottleneck and proposed repository change.
   • Label estimates consistently as elapsed wall-clock time, aggregate compute/runner time, or money, distinguishing reduced elapsed duration from reduced billed compute.
   • Calculate estimated time saved per run and affected runs per day; formulate daily savings (median seconds saved per run × affected runs per day) separately for elapsed wall-clock time and aggregate compute/runner time.
   • Calculate estimated daily monetary savings only when billing units and unit rates are known: saved billable units × unit rate. If unit rates are unknown, state savings in aggregate compute/runner time and mark monetary savings as not estimable—never invent pricing figures.
   • State all assumptions, formulas, confidence bounds, and sample sizes explicitly.
   • Rank proposals only within a stated unit (elapsed wall-clock time, aggregate compute/runner time, or money).
5. Await human authorization. Present the ranked proposals to maintainers and obtain explicit human approval before modifying files or initiating runs. Never apply changes directly to live CI systems.
6. Apply a minimal change. Prepare the smallest reviewable repository diff implementing the approved optimization. Keep changes isolated from unrelated edits; preserve all test assertions, matrix coverage, security gates, fallbacks, and production behaviors.

Evidence and Reporting

Provide a structured report containing:

• Ranked Proposals: Ordered candidate list with clear descriptions, estimated daily savings, and underlying assumptions.
• Evidence: Workflow/job identifiers, date window, sample sizes, observed median and P95 durations, and billable units.
• Savings Formula & Assumptions: Explicit mathematical formulas, unit rates or compute units, and stated caveats.
• Guardrail Confirmation: Explicit confirmation that test coverage, platform matrix, security checks, and operational safeguards remain fully intact.
• Authorization & Status: Record of maintainer approval before any modification is applied.

Verification

• Baseline Recording: Document pre-change baseline timings, runner types, and billable units across comparable workloads and cache conditions.
• Post-Change Measurement: Record post-change runs under matched conditions (same commit workload, runner tier, platform, and cache state). Compare median distributions rather than isolated best-case runs.
• Matrix & Safeguard Check: Verify that all required jobs continue executing and passing across every supported operating system and architecture in the platform matrix.
• Actuals vs. Estimates: Report observed duration and compute differences separately from initial estimates. Note any regressions, anomalies, limitations, or rollback actions.

Example

Trigger: Maintainers request an evidence-based analysis of CI pipeline duration and compute consumption to identify cost-reduction opportunities.

Evidence & Analysis:

• Examined historical run records for .github/workflows/ci.yml across 150 runs over 14 days (hypothetical sample: 150 runs / 14 days = about 10.7 runs/day). The workflow executes 3 parallel matrix jobs (Linux, macOS, Windows) running test suites and linters.
• Observed bottleneck: Dependency installation runs from scratch on every run without caching across all 3 matrix jobs; 20 runs in the 14-day window failed early on linting after downstream matrix jobs had already been scheduled.

Ranked Proposals (Ranked within aggregate runner compute time):

1. Proposal 1: Cache dependency directories across matrix jobs.
   • Estimated aggregate compute saved: ~32.1 runner-minutes/day (hypothetical: cache saves 1 minute in each of 3 matrix jobs per workflow; aggregate runner compute = 10.7 runs/day × 3 matrix jobs × 1 minute saved per job = about 32.1 runner-minutes/day).
   • Estimated elapsed wall-clock saved: ~10.7 wall-clock minutes/day (hypothetical: about 10.7 minutes/day if matrix jobs run in parallel, saving ~1 minute per workflow run).
2. Proposal 2: Fast-fail linting step before running heavy matrix builds.
   • Estimated aggregate compute saved: ~11.4 runner-minutes/day (hypothetical: assume 20 early-failing runs in 14 days avoid 8 downstream runner-minutes per affected run; aggregate runner compute = 20 / 14 × 8 = about 11.4 runner-minutes/day).
   • Estimated elapsed wall-clock saved: Not estimated (scheduling impact is not established). (Monetary savings marked as not estimable due to unknown contractual runner rates; proposals are ranked strictly within aggregate runner compute time.)

Guardrail Confirmation & Next Step:

• Full platform matrix (Linux, macOS, Windows) and all test coverage are preserved. Await maintainer authorization before preparing the workflow cache diff.