agentleFS
carverautocarverauto/serviceradar/.agents/skills/release-cut-and-demo-roll/SKILL.md

release-cut-and-demo-roll

Skill934 starsChanged 2 days agoApache-2.0View on GitHub

Cut a ServiceRadar release and roll the Kubernetes `demo` namespace to the resulting published semver image tag through the guarded ArgoCD release branch. Use when the user asks to update `VERSION` and `CHANGELOG`, run `scripts/cut-release.sh`, push the release refs, wait for published release artifacts, and then refresh `demo` to that released version. Do not use for pre-release local testing with unpublished images; use `$demo-local-rollout` or `$demo-web-ng-fastpath` for that.

  • Deletes or force-pushes
  • Installs packages
  • Commits and pushes

The file

Show its rules and text

We split this skill into the rules it gives an agent

25 rules. About 0 are common, 7 are distinctive, and 18 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈7

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 18

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Update VERSION and the top CHANGELOG entry for the new semver.

    line 15
  2. Very specific:

    Run scripts/cut-release.sh --version <version> --push.

    line 17
  3. Distinctive:

    Wait for the published release artifacts to exist for the target semver tag.

    line 20
  4. Distinctive:

    Use this only for actual release cuts.

    line 28
  5. Very specific:

    Do not bypass scripts/cut-release.sh unless the user explicitly asks for a different path.

    line 29
  6. Distinctive:

    Do not treat the release as deployable until the published artifacts for the release version actually exist.

    line 30
  7. Distinctive:

    Prefer published release artifacts over rebuilding images locally in this workflow.

    line 31
  8. Very specific:

    Do not re-enable Image Updater, prune, or automated self-heal until the generated-secret, CNPG, and deployment drift called out in …

    line 33
  9. Very specific:

    Run all of these BEFORE touching VERSION.

    line 39
  10. Very specific:

    Must end Build completed successfully with no FAIL:.

    line 49
  11. Distinctive:

    Before cutting the release:

    line 74
  12. Very specific:

    add the matching top entry in CHANGELOG

    line 77
  13. Distinctive:

    Create the release commit and annotated tag, then publish the release branch while retaining the tag locally:

    line 89
  14. Very specific:

    Do not run the no-push form first and then rerun with --push.

    line 96
  15. Very specific:

    Do not squash or rebase the release PR because that changes or discards the tagged commit.

    line 106
  16. Very specific:

    Keep the release commit SHA for reporting and traceability.

    line 124
  17. Very specific:

    Do not roll demo until the release artifacts for the target semver tag exist.

    line 128
  18. Very specific:

    Verify all of these BEFORE pushing, because the push is what rolls demo:

    line 161
  19. Very specific:

    After the user pushes, Argo picks the new revision up on its own.

    line 222
  20. Very specific:

    Use an authenticated Argo CLI context or the Argo UI.

    line 254
  21. Very specific:

    Do not use --core: it looks for argocd-cm in the active namespace and does not work with this cluster layout.

    line 261
  22. Distinctive:

    Do not run an operator-triggered sync while the guarded automatic operation is progressing or has succeeded.

    line 274
  23. Very specific:

    Do not patch global.imageTag directly for a formal release.

    line 281
  24. Very specific:

    Do not use sr_demo_deploy for formal releases.

    line 283
  25. Very specific:

    Do not report the rollout finished until the key workloads are running the new v<version> tag and the environment is healthy.

    line 306

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Release Cut And Demo Roll

Overview

Use this skill for the formal release path: update release metadata, cut the release commit and tag, publish the refs, confirm that the release artifacts exist, and then verify the automatic demo rollout to the released semver image tag, such as v1.2.41. Prefer the repo's existing release script, CI publish path, guarded demo/prod-release branch, and conservative ArgoCD auto-sync over ad hoc local image pushes.

Workflow

  1. Work from the repo root on the intended release branch.
  2. Update VERSION and the top CHANGELOG entry for the new semver.
  3. Dry-run the release cut to verify the changelog entry is wired correctly.
  4. Run scripts/cut-release.sh --version <version> --push; this publishes the release branch only and keeps the annotated tag local.
  5. Open the release pull request and merge it into staging with gh pr merge <PR> --merge after CI passes. Squash/rebase would discard the tagged commit.
  6. Fetch origin/staging, prove the local tag commit is now an ancestor, and publish the tag with an explicit tag refspec.
  7. Wait for the published release artifacts to exist for the target semver tag.
  8. Confirm the release workflow advanced demo/prod-release and its Argo source file to v<version>.
  9. Confirm serviceradar-demo-prod still has automated sync enabled with prune and self-heal disabled, and observe Argo pick up the guarded branch without an operator-triggered sync.
  10. Review any residual drift and watch Argo until demo reaches Synced|Healthy|Succeeded; use a manual sync only as an investigated recovery action.
  11. Report the release version, release commit, release tag, release-branch revision, automatic-sync result, and final demo rollout status.

Guardrails

  • Use this only for actual release cuts. Do not use it for one-off local testing or unpublished commits.
  • Do not bypass scripts/cut-release.sh unless the user explicitly asks for a different path.
  • Do not treat the release as deployable until the published artifacts for the release version actually exist.
  • Prefer published release artifacts over rebuilding images locally in this workflow.
  • Roll formal releases with the semver image tag, for example v1.2.41. The sha-<commit> path is for unpublished local demo testing only.
  • Do not re-enable Image Updater, prune, or automated self-heal until the generated-secret, CNPG, and deployment drift called out in k8s/argocd/applications/demo-prod.yaml has been reviewed and normalized.
  • While that hold is active, serviceradar-demo-image-updater is expected to report zero matched applications and zero managed images. Treat 0|0|NoErrors:False as the configured hold state, not a failed release.
  • Automatic sync is authorized only for the guarded, publication-verified demo/prod-release branch and must remain non-pruning and non-self-healing. Generated secrets, CNPG resources, and unrelated live drift still require explicit operator judgment.

Pre-Cut Gates

Run all of these BEFORE touching VERSION. Each one has broken a release already.

  1. Every unit test, the way CI runs them. A tag pins its own source, so a fix pushed to staging afterwards is invisible to a re-run of that tag -- a broken test found after tagging costs a new tag, not a re-run.

    make test   # bazel test -c opt --config=ci //... --test_tag_filters=-integration_test,-acceptance_test

    Must end Build completed successfully with no FAIL:. go test / cargo test / mix test do NOT cover this: the Elixir unit shards exist only as bazel targets (//elixir/serviceradar_core:unit_tests_*, //elixir/web-ng:unit_tests_*), which is how two broken Elixir suites reached the v1.4.30 tag.

  2. Duplicate migration versions -- must print nothing:

    ls elixir/serviceradar_core/priv/repo/migrations/ | grep -oE '^[0-9]+' | sort | uniq -d

    This bit v1.3.9 badly: three migrations collided on 20260630120000, and Ecto refuses to run any migration when versions collide, silently bricking auth, metrics and plugin assignment on every fresh install. CI passed because it does not migrate against a prior baseline. If it prints anything, renumber the extras before cutting.

  3. Prod release assembles -- broke v1.3.8. cd elixir/serviceradar_core && MIX_ENV=prod mix release must produce no "Duplicated modules" (the hackney/h2 vs grpcbox/chatterbox conflict). Gate on BOTH serviceradar_core and serviceradar_core_elx.
  4. macOS needs gsed (brew install gnu-sed) -- cut-release.sh uses GNU sed for in-place edits.

Update Release Metadata

Before cutting the release:

  • update VERSION
  • add the matching top entry in CHANGELOG

Then dry-run the cut:

scripts/cut-release.sh --version <version> --dry-run

Fix metadata issues before continuing.

Cut And Push The Release

Create the release commit and annotated tag, then publish the release branch while retaining the tag locally:

scripts/cut-release.sh --version <version> --push

Do not run the no-push form first and then rerun with --push; the first run already creates the local tag, so the second correctly refuses to proceed.

Open the resulting release pull request and use the ancestry-preserving merge commit method:

gh pr merge <PR> --merge

Do not squash or rebase the release PR because that changes or discards the tagged commit. Only after that commit is on staging, publish the tag with the mechanically chained ancestry check printed by the script:

git fetch origin refs/heads/staging:refs/remotes/origin/staging
git merge-base --is-ancestor 'v<version>^{commit}' \
  refs/remotes/origin/staging && \
  git push origin refs/tags/v<version>:refs/tags/v<version>

Afterward, capture:

git rev-parse HEAD
git describe --tags --exact-match

Keep the release commit SHA for reporting and traceability. Use the release version as the demo image tag source: v<version>.

Published Artifact Expectations

Do not roll demo until the release artifacts for the target semver tag exist. The GitHub Actions release.yml workflow is the supported formal publisher because the complete release also includes packages, the Helm chart, managed-agent assets, native add-ons, Wasm plugins, signatures, both import catalogs, and source/image security bundles. make push_all_release is retired and exits without publishing. Re-run the failed release.yml job at the release tag (gh run rerun <run-id> --failed) when recovery is required.

Recovery: Release Published But demo/prod-release Was Not Advanced

The Advance demo release source branch step in .github/workflows/release.yml is conditional: it runs only when the release is not a prerelease AND the parallel-asset and finalize steps both succeeded. An unrelated asset failure -- a Wasm plugin publish, for example -- therefore leaves a fully published, correctly signed release with demo/prod-release still pointing at the previous version. The release is fine; only the deploy pointer is stale.

What that step actually does is force-push the release commit itself:

git push --force-with-lease demo-release "HEAD:refs/heads/demo/prod-release"

where HEAD is the release commit, git rev-list -n1 "refs/tags/v<version>^{commit}". So the recovery is to reproduce that push, NOT to hand-write a commit that edits .argocd-source-serviceradar-demo-prod.yaml. That push force-rewrites a shared branch, which agents must never do: run the checks below, then hand the user the exact command and let them run it. scripts/cut-release.sh already wrote global.imageTag: v<version> into that file at the release commit, and that commit is also the only tree carrying the matching Chart.yaml version and migration expectedVersion. Adding a tag-only commit on top of the stale branch would run new images against the old chart.

Verify all of these BEFORE pushing, because the push is what rolls demo:

# 1. Release commit is the tag's commit and is reachable from staging.
RC="$(git rev-list -n1 'v<version>^{commit}')"
git merge-base --is-ancestor "$RC" refs/remotes/origin/staging
./scripts/validate-release-metadata.sh "v<version>" "$RC"

# 2. The file at that commit already carries the tag, with no digest pins.
git show "$RC:helm/serviceradar/.argocd-source-serviceradar-demo-prod.yaml"

# 3. Every image the chart renders exists and is signed with the key Kyverno
#    enforces. Render from the release tree, not the working tree.
helm template serviceradar <release-tree>/helm/serviceradar -n demo \
  -f <release-tree>/helm/serviceradar/values-demo.yaml \
  --set-string global.imageTag=v<version> |
  grep -oE 'registry[.]carverauto[.]dev/serviceradar/[a-z0-9.-]+:[^ "]+' | sort -u
cosign verify --key docs/cosign.pub --insecure-ignore-tlog=true <each image>

docs/cosign.pub is the same key the verify-serviceradar-images ClusterPolicy carries; diff them rather than assuming. That policy sets mutateDigest: true, so admitted pods show :<tag>@sha256:... -- comparing that digest against crane digest <image>:v<version> is the proof of which artifact is running.

The digest-pin trap

Between releases, demo/prod-release accumulates ad-hoc commits that add image.digests.<service> parameters pinning individual services to sha-... builds. Per serviceradar.imageRefSuffix in templates/_helpers.tpl, a digest pin completely overrides global.imageTag. Bumping only the tag while those pins remain is inert for exactly the services people care about most.

The force-push discards those commits, which is intended -- but confirm first that nothing regresses. Map each pinned digest back to its build commit and prove it is contained in the release:

for t in $(crane ls registry.carverauto.dev/serviceradar/<image> | grep '^sha-'); do
  [ "$(crane digest registry.carverauto.dev/serviceradar/<image>:$t)" = "<pinned digest>" ] && echo "$t"
done
git merge-base --is-ancestor <build-commit> 'v<version>^{commit}'

A pin whose build commit is NOT an ancestor is not automatically a regression -- branch builds are often squash-merged, so the work lands under a different SHA. Resolve it by diffing the owning package between that commit and the release tag rather than by trusting the ancestry check alone. Pinning commits also sometimes carry chart edits that were never upstreamed; diff origin/demo/prod-release..v<version> -- helm/serviceradar/ and account for every removal before pushing.

The command to hand over, with an explicit lease on the branch head you verified and an explicit refspec:

OLD="$(git rev-parse refs/remotes/origin/demo/prod-release)"
git push --force-with-lease="refs/heads/demo/prod-release:${OLD}" \
  origin "${RC}:refs/heads/demo/prod-release"

After the user pushes, Argo picks the new revision up on its own. Do not sync manually; confirm the operation was automatic with .status.operationState.operation.initiatedBy.automated.

Roll Demo To The Release Tag

The release workflow advances demo/prod-release only after the complete release, catalog, and security asset set verifies. Confirm that branch now carries the target tag:

git fetch origin \
  refs/heads/demo/prod-release:refs/remotes/origin/demo/prod-release
git show origin/demo/prod-release:helm/serviceradar/.argocd-source-serviceradar-demo-prod.yaml

The source file must set global.imageTag to v<version>. scripts/validate-release-metadata.sh fails the release when that pin is not the release tag (docs/RELEASE_PUBLISHING.md). Confirm the conservative automatic-sync policy is still active:

kubectl get application -n argocd serviceradar-demo-prod \
  -o jsonpath='{.spec.syncPolicy.automated.enabled}{"|"}{.spec.syncPolicy.automated.prune}{"|"}{.spec.syncPolicy.automated.selfHeal}{"\n"}'

Expect true|false|false. Then check the Image Updater hold state for context:

kubectl get imageupdater -n argocd serviceradar-demo-image-updater \
  -o jsonpath='{.status.applicationsMatched}{"|"}{.status.imagesManaged}{"|"}{range .status.conditions[?(@.type=="Error")]}{.reason}{":"}{.status}{":"}{.message}{end}{"\n"}'

Expect zero matched applications, zero managed images, and an Error condition with reason NoErrors and status False while the Image Updater hold is active.

Use an authenticated Argo CLI context or the Argo UI. The usual local CLI context expects the Argo server on 127.0.0.1:18443; start its port-forward in a separate shell when it is not already reachable:

kubectl port-forward -n argocd svc/argocd-server 18443:443
argocd app get serviceradar-demo-prod --refresh

Do not use --core: it looks for argocd-cm in the active namespace and does not work with this cluster layout.

Refresh the application and inspect any residual drift while the guarded automatic sync progresses:

kubectl get application -n argocd serviceradar-demo-prod -o json |
  jq -r '.status.resources[] | select(.status != "Synced") | [.group, .kind, .namespace, .name, .status] | @tsv'
argocd app diff serviceradar-demo-prod --hard-refresh
kubectl get application -n argocd serviceradar-demo-prod -o yaml

argocd app diff exits 1 when a diff exists and intentionally omits Kubernetes Secrets. Inspect every non-release change in its output, inspect each OutOfSync Secret's owner/hook annotations separately, and review CNPG resources explicitly. Remove or account for stale live-only Application parameters rather than broadening the automatic policy.

Do not run an operator-triggered sync while the guarded automatic operation is progressing or has succeeded. If automatic sync fails or remains stuck after investigation, a reviewed recovery sync may be dry-run and executed without prune:

argocd app sync serviceradar-demo-prod --dry-run
argocd app sync serviceradar-demo-prod --assumeYes

Do not patch global.imageTag directly for a formal release. That creates another live-only override and bypasses the verified demo/prod-release source.

Do not use sr_demo_deploy for formal releases; that helper rolls demo to a sha-... tag and is only appropriate for local unpublished test builds.

Verify Demo Rollout

Watch the Argo app or Helm result until the deployment is healthy. For Argo-backed environments, wait for:

Synced|Healthy|Succeeded

Useful checks:

kubectl get application -n argocd serviceradar-demo-prod \
  -o jsonpath='{.status.sync.status}{"|"}{.status.health.status}{"|"}{.status.operationState.phase}{"\n"}'

kubectl get pods -n demo

kubectl get deploy -n demo \
  serviceradar-web-ng serviceradar-core serviceradar-agent serviceradar-tools \
  -o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{range .spec.template.spec.containers[*]}{.image}{" "}{end}{"\n"}{end}'

Do not report the rollout finished until the key workloads are running the new v<version> tag and the environment is healthy.

Report Back

Close with:

  • release version
  • release commit SHA
  • release tag
  • whether refs were pushed
  • whether release artifacts were confirmed published
  • demo/prod-release revision and image tag
  • Image Updater hold status
  • automatic-sync result, or the investigated manual recovery result when one was required
  • final demo rollout status
  • any residual risk or follow-up needed

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.