ECC Tools Cost Audit

Original work by ECC. Credit belongs to the ECC authors.

Use this skill when ECC Tools may waste tokens or money. It checks for extra pull requests, quota bugs, paid model leaks, repeat jobs, and failed work that still costs money.

This skill is only for the sibling ECC-Tools repo. It is not a full code review or a general billing audit.

Do not add tracking, analytics, telemetry, or new calls to outside services.

Use Other ECC Skills

Use these skills when they fit the task:

• autonomous-loops: Keep a long audit small and bounded.
• agentic-engineering: Split the request path into clear steps.
• customer-billing-ops: Keep code bugs and customer charges separate.
• search-first: Find code that already exists before adding helpers.
• security-review: Check auth, quota gates, access rights, and secrets.
• verification-loop: Prove the fix is safe to run more than once.
• tdd-workflow: Add a test when changing worker, router, or billing code.

If a listed skill is not present, continue without it.

When to Use

Use this skill when:

• The user reports high ECC Tools cost.
• The app creates too many pull requests.
• A user can pass a usage limit.
• A free user can reach a paid model.
• One event starts the same work more than once.
• The app spends tokens but gives no useful result.
• The task covers webhooks, queues, workers, pull requests, usage, or model choice in ECC-Tools.

Do not use this skill for another repo unless the user says to do so.

Safety Rules

• Work only in the sibling ECC-Tools repo.
• Start with read-only checks.
• Change code only when the user asks for a fix.
• Check the current branch and local changes first.
• Keep unrelated user changes.
• Do not change checkout, billing pages, or other UI unless the bug needs it.
• Do not push or deploy unless the user asks.
• Do not expose keys, tokens, private data, or customer data.
• Treat app-made branches and pull requests as a loop risk until proved safe.
• Do not add analytics, telemetry, or outside network calls.

Keep these results separate:

1. The code bug that caused the cost.
2. The billing effect on users.
3. A product rule that needs later work.

Do not guess customer charges from code alone. Use billing records only if the user gave access and asked for that check.

Audit Steps

1. Set the Scope

Go to the sibling ECC-Tools repo.

Check:

• Current branch.
• Local changes.
• The event or user report being checked.
• The time range, account, job ID, pull request, or event ID, if known.

Pick the exact paths to trace:

• Webhook entry.
• Queue sender.
• Queue worker.
• Pull request creation.
• Usage check and usage save.
• Model choice.
• Retry code.

If key details are missing, trace the smallest likely path first. Ask the user only if the missing detail would change the audit in a major way.

2. Trace the Event

Read src/index.* or the real main entry file first.

Map the full path:

GitHub event -> route -> queue job -> worker -> model call -> saved result

For each step, record:

• Event type.
• Job name and unique ID.
• Dedupe rule.
• Quota check.
• Usage hold or charge.
• Model and user tier.
• Side effects, such as a branch or pull request.

Check whether these events can reach the same costly work:

• push
• pull_request
• pull_request.synchronize
• comments
• manual runs
• retries

Do not assume two event types are safe because they use different routes. Follow both routes to the worker.

3. Check the Worker

Find what happens before and after the model call.

Check whether each job can:

• Create a branch.
• Create or update a pull request.
• Change files.
• Call a paid model.
• Hold, add, or refund usage.
• Retry after an error.
• Save a result.

If tokens are spent but no result is saved, label it:

cost-with-no-output

Also check errors between the model call and the saved result. These errors may cause the same costly work to run again.

4. Check the Main Cost Risks

Pull Request Growth

Check:

• Branch names are stable for the same task.
• An open pull request is reused.
• Sync events do not create a new pull request each time.
• App-made branches cannot start the same analysis again.
• Two workers cannot create two pull requests for one job.

An app that reads its own branch or pull request is a top-level loop risk.

Quota Bypass

Find where the quota is checked, held, charged, and refunded.

A check before enqueue is not enough. Two requests may pass before either worker saves usage.

Use one safe rule, such as:

• Hold usage in one atomic step before enqueue.
• Use a database lock or unique row.
• Give each costly job a unique key.
• Release the hold when safe work does not start.

Check what happens when enqueue fails, the worker stops, or the job is canceled.

Paid Model Leak

Check model choice against:

• User plan.
• Remaining quota.
• Job type.
• Feature access.
• Provider keys being present.

A provider key must not grant access by itself. Free or capped users must be blocked or sent to the allowed low-cost path.

Also check fallback code. A cheap model failure must not silently switch a free user to a paid model.

Retry Cost

Check:

• Retry count.
• Delay between retries.
• Which errors may retry.
• Whether a retry uses the same job key.
• Whether a finished job can run again.
• Whether a dead-letter job can be started twice.

Do not retry bad input, denied access, missing files, or other fixed errors without a clear change.

Cost Before Safe Save

Check for paid work before:

• Branch name checks.
• Pull request reuse checks.
• Write access checks.
• File checks.
• Quota holds.
• Dedupe checks.

Move cheap safety checks before costly work when that does not change the product rule.

5. Fix in Cost Order

If the user asked for code changes, use this order:

1. Stop pull request loops and growth.
2. Stop quota bypass.
3. Stop paid model leaks.
4. Stop duplicate jobs and useless retries.
5. Make reruns and updates safe.

Make one to three direct fixes. Change more files only when one root cause spans those files.

Do not clean up nearby code unless the fix needs it.

6. Test the Smallest Useful Case

Run only the tests that prove the changed path.

Prove at least one result:

• The job is blocked before paid work.
• Two matching events become one job.
• An old pull request is reused.
• A free user gets the allowed model.
• A fixed error does not retry.
• A failed enqueue releases its usage hold.
• A rerun does not create a second charge or pull request.

Add a test for race bugs when practical. A single normal request does not prove a quota race is fixed.

If no test exists, use a small local test or a dry run. Do not call a real paid model or create a real pull request just to test.

Concrete Example

User request:

ECC Tools made three pull requests after one push. Find the cause and fix it.

Audit:

1. Check the branch and local changes.
2. Trace the push event from the webhook to the queue.
3. Trace pull request events from app-made branches.
4. Compare their job keys.
5. Check whether the worker looks for an open pull request before creating one.
6. Confirm whether the app's pull request starts another analysis.
7. Fix the first loop point.
8. Add a test that sends the same event twice.
9. Add a test for an event from an app-made branch.
10. Prove that one source change creates at most one open pull request.

Report:

Cause: The app handled its own branch push as a new user push.
Fix: App-made branches are now skipped before enqueue.
Proof: The focused webhook tests pass. Duplicate events create one job.
Status: Changed and verified locally. Not pushed or deployed.

Common Bad Patterns

One Job Type for Every Event

Pushes, pull request syncs, and manual runs all send the same job. The worker always creates a pull request. This can turn each analysis into pull request spam.

Usage Saved Too Late

The route checks quota, but the worker saves usage later. Many requests can pass the first check at the same time.

Free User on a Paid Model

The code sees a paid provider key and uses that model without checking the user's plan.

App Output Starts New Work

A branch push, sync event, or comment on an app-made pull request starts the app again.

Paid Work Before Safety Checks

The app calls a model before it checks branch names, open pull requests, write access, quota, or duplicate jobs.

Endless Fixed-Error Retries

The same bad input or denied request runs again even though waiting cannot fix it.

Missing Refund or Release

A usage hold stays in place after enqueue fails, a job is canceled, or the worker stops before paid work begins.

Final Report

Name exact file paths and code areas.

Report:

• Root cause.
• Cost path.
• Customer effect, if proved.
• Fixes, in cost order.
• Tests or commands run.
• Limits of the proof.
• Any work still needed.

End with one exact status:

• Read-only audit complete
• Changed locally
• Changed and verified locally
• Pushed
• Deployed
• Blocked

Never say the cost bug is fixed until the narrow test passes.