GitHub Actions Debugging

Work out whether the problem is the trigger, the permissions, the runner, or the job itself.

Purpose

Use this when:

• A workflow does not run when you expect it to.
• A job fails with a permissions error, or an OIDC role assumption fails.
• The cache never hits, or builds are much slower than they should be.
• A step passes locally and fails in CI, or the reverse.
• A deployment job hangs, or deploys the wrong thing.

Do not use this when:

• The application code is broken — CI is correctly reporting a real failure.
• The failure is in the infrastructure being deployed — use terraform-troubleshooting or the relevant layer skill.

What this skill assumes: access to the workflow file and the run logs. The gh CLI is used throughout; everything is also available in the web UI.

Operating Procedure

1. Determine whether it ran at all. "Failed" and "never triggered" are different problems with no overlap.
2. Read the log from the failing step, including the setup steps above it. The cause is frequently in a step that reported success.
3. Classify: trigger, permissions, environment, dependency, or the command itself.
4. Reproduce as narrowly as possible. A container that matches the runner, or a debug run.
5. Check what differs from local. Environment variables, secrets, working directory, the checkout's depth, and the runner image contents.
6. Fix and re-run the failed jobs, not the whole workflow, while iterating.

Run logs are Observed. What a workflow file appears to do is an Assumption until you confirm it against the run — reusable workflows, composite actions and repository settings all change effective behaviour.

Investigation

Step 1 - did it run

gh run list --limit 20
gh run list --workflow=<file>.yml --limit 10
gh run view <run-id> --log-failed

If the workflow never appeared, work through these — they cover nearly all cases:

The GITHUB_TOKEN rule is the one that causes the most confusion: a workflow that commits with the default token will not trigger the workflow that watches for commits. Use a PAT or a GitHub App token when chaining is intended.

Step 2 - permissions

permissions:
  contents: read
  id-token: write        # required for OIDC
  packages: write        # required to push to GHCR

The default token permissions depend on a repository or organisation setting and changed to read-only by default for new repositories. A workflow that worked in one repository and fails in another with Resource not accessible by integration is almost always this.

Declare permissions explicitly at the workflow level. Relying on the default makes the workflow's behaviour dependent on a setting nobody reading the file can see.

For OIDC to a cloud provider:

permissions:
  id-token: write
  contents: read

gh run view <run-id> --log | grep -iE 'AssumeRoleWithWebIdentity|sts|audience|sub'

OIDC failures have a short list of causes:

1. id-token: write not granted — the token is never minted.
2. The cloud trust policy's sub condition does not match. The claim is shaped like repo:<org>/<repo>:ref:refs/heads/main or repo:<org>/<repo>:environment:production, and a policy written for a branch will reject a run from an environment or a pull request.
3. The audience does not match what the provider expects.
4. The OIDC provider thumbprint or configuration is missing on the cloud side.

The sub mismatch is the common one, and the error message names the received value — compare it character by character against the trust policy condition.

Step 3 - secrets and forks

gh secret list
gh secret list --env production

Rules that catch people:

• Secrets are not available to pull_request runs from forks. This is deliberate: otherwise anyone could open a PR that exfiltrates them.
• pull_request_target does have secrets, and runs in the context of the base repository. It is genuinely dangerous: checking out and executing PR code under pull_request_target hands your secrets to whoever opened the PR. Use it only without checking out untrusted code.
• Environment secrets require the job to declare environment:. Without it, the secret is simply empty and the failure is confusing.

• Secrets are masked in logs, so an empty secret and a working one look similar. Check length rather than value:

  - run: echo "len=${#TOKEN}"
    env:
      TOKEN: ${{ secrets.MY_TOKEN }}

  A length of 0 means the secret is not reaching the job. Never echo the value itself — masking is best-effort, and a transformed value (base64, substring) is not masked at all.

Step 4 - caching

- uses: actions/cache@v4
  with:
    path: ~/.npm
    key: ${{ runner.os }}-npm-${{ hashFiles('**/package-lock.json') }}
    restore-keys: ${{ runner.os }}-npm-

gh run view <run-id> --log | grep -iE 'cache|Cache restored|Cache saved|cache hit'

Why caches miss:

restore-keys gives a partial-match fallback, which turns a full miss into a near-hit. A cache configuration without restore-keys is all-or-nothing and usually nothing.

For Docker builds, use the registry or GitHub Actions cache backend rather than actions/cache on the Docker directory:

- uses: docker/build-push-action@v6
  with:
    cache-from: type=gha
    cache-to: type=gha,mode=max

Step 5 - works locally, fails in CI

gh run view <run-id> --log | head -50

Differences that account for most of these:

For a stubborn case, reproduce in a container that matches the runner:

docker run --rm -it -v "$PWD":/w -w /w ubuntu:24.04 bash

Or re-run with debug logging:

gh run rerun <run-id> --debug

That sets ACTIONS_STEP_DEBUG, which reveals the inputs and outputs of each action.

Step 6 - concurrency and deployment

concurrency:
  group: deploy-${{ github.ref }}
  cancel-in-progress: false

For deployments, cancel-in-progress: false is almost always correct: cancelling a deployment mid-flight can leave a partial rollout, a half-applied migration, or a Terraform state lock held by a killed process. For pull request checks, cancel-in-progress: true is right — superseded runs are wasted compute.

gh run view <run-id> --json jobs --jq '.jobs[] | {name, status, conclusion, startedAt, completedAt}'

A job that sits in waiting is usually blocked on an environment protection rule — a required reviewer, or a wait timer. That is a configured behaviour, not a failure, and it is worth checking before investigating further.

Decision Tree

Workflow problem
├── Did not run at all
│   ├── Not on the default branch (schedule/dispatch)
│   ├── Path or branch filter excluded it
│   ├── Triggered by GITHUB_TOKEN → does not chain
│   ├── Scheduled workflow auto-disabled after inactivity
│   └── YAML invalid → the file is ignored silently
├── "Resource not accessible by integration" → permissions block
├── OIDC / AssumeRole fails
│   ├── id-token: write missing
│   ├── sub claim does not match the trust policy
│   └── audience mismatch
├── Secret is empty
│   ├── Fork pull_request → no secrets by design
│   └── Environment secret without environment: declared
├── Cache never hits
│   ├── Key varies every run
│   ├── Branch scoping
│   └── Evicted by the 10 GB limit
├── Passes locally, fails in CI
│   ├── fetch-depth: 1 → no git history
│   ├── Case-sensitive filesystem
│   └── Missing tool or environment variable
└── Job hangs → environment approval, an interactive prompt, or a missing timeout

Common Causes

Safety

Read-only

gh run list, gh run view, reading workflow files and logs, gh secret list (names only, never values). These inspect and change nothing.

Log output is a real exfiltration surface. Secrets are masked, but any transformation defeats masking — base64, a substring, or a JSON re-encode all print in the clear. Never add a debug step that prints a secret in any form.

Potentially disruptive

• Re-running a workflow — re-runs whatever it does, including deployments. gh run rerun on a deploy workflow deploys again. Check what the workflow does before re-running it.
• Changing permissions — broadening them increases what a compromised action can do. Grant the narrowest set that works.
• Editing a workflow on the default branch — takes effect for the next run immediately, with no review gate unless one is configured.
• Changing or rotating a secret — every workflow using it is affected from the next run.
• Cancelling a running deployment job — can leave a partial rollout, a half-applied migration, or a Terraform state lock held by the killed process. Prefer letting it finish and then rolling back deliberately.
• Disabling a required check — removes a merge gate for everyone.

Destructive

• gh cache delete --all — removes every cache in the repository. Not data loss, but every subsequent build is a cold build, which on a busy repository is hours of extra runner time. Required first: confirm a targeted deletion would not do.
• A workflow that runs terraform destroy, kubectl delete or a database migration down-step — the destructive operation is in the pipeline, not in this skill, but re-running such a workflow executes it. Required first: read what the workflow actually does; use an environment with required reviewers for anything irreversible.
• pull_request_target that checks out and executes PR code — this is not a destructive command but it is the most serious security mistake available in Actions: it grants an arbitrary contributor's code access to your secrets and a write-scoped token. Treat any workflow doing this as an active incident, and rotate every secret that was exposed.

Verification

Before: deploy workflow fails at the AWS step with AssumeRoleWithWebIdentity denied
After:  3 consecutive runs succeed; the deploy step assumes the role in 2s; cache hit rate 90%

gh run list --workflow=deploy.yml --limit 5 \
  --json conclusion,createdAt,displayTitle --jq '.[] | "\(.conclusion) \(.createdAt) \(.displayTitle)"'

gh run view <run-id> --json jobs \
  --jq '.jobs[] | "\(.name): \(.conclusion) (\((.completedAt|fromdate) - (.startedAt|fromdate))s)"'

Three requirements:

1. Run it more than once. CI failures are frequently intermittent — a flaky test, a rate limit, a cache race. One green run does not distinguish a fix from luck.
2. Verify the outcome, not the exit code. A deployment job that exits 0 has not necessarily deployed. Check the deployed artifact's version at the destination.
3. Check the duration. A fix that works by disabling the cache is a regression that will show up as cost rather than as a failure.

Anti-Patterns

Bad: Add permissions: write-all to fix a permissions error. Better: That grants every scope to every action in the workflow, including third-party ones. Grant the specific scope the failing step needs.

Bad: Echo a secret to debug whether it is set. Better: Masking is defeated by any transformation, and logs are retained and often public. Print the length.

Bad: Use pull_request_target so that fork PRs get secrets. Better: That is the intended behaviour of the security boundary, not a bug. If a fork PR could execute code with your secrets, so could anyone.

Bad: Pin actions to a version tag like @v4. Better: Tags are mutable. A compromised or retagged action runs in your pipeline with your token. Pin to a commit SHA for anything outside your own organisation.

Bad: Re-run the workflow to see if it passes this time. Better: On a deploy workflow that deploys again. Read the log first; on a genuinely flaky test, find the flake rather than retrying until it is green.

Bad: Omit timeout-minutes. Better: The default is 6 hours. A hung job holds a runner and burns the budget for a quarter of a day before anyone notices.

Composition

• container-image-review — for the image the pipeline builds.
• secrets-management-review — how secrets reach the pipeline and how they are rotated.
• terraform-safe-changes — when the pipeline applies infrastructure changes.
• incident-triage — when a bad pipeline run has caused a production incident.

References

• GitHub Actions: Security hardening — pull_request_target, action pinning, and token scopes.
• GitHub Actions: OIDC hardening — the sub claim format used in cloud trust policies.
• GitHub Actions: Caching dependencies