release-cut-and-demo-roll
Skill934 starsChanged 2 days agoApache-2.0View on GitHub
Cut a ServiceRadar release and roll the Kubernetes `demo` namespace to the resulting published semver image tag through the guarded ArgoCD release branch. Use when the user asks to update `VERSION` and `CHANGELOG`, run `scripts/cut-release.sh`, push the release refs, wait for published release artifacts, and then refresh `demo` to that released version. Do not use for pre-release local testing with unpublished images; use `$demo-local-rollout` or `$demo-web-ng-fastpath` for that.
- Deletes or force-pushes
- Installs packages
- Commits and pushes
In this repository18 other files it gives its agents.
- AGENTS.mdAGENTS.md
- docs/AGENTS.mdAGENTS.md
- openspec/AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- daisyuiSkill
- demo-cnpg-local-web-ngSkill
- demo-local-rolloutSkill
- demo-web-ng-fastpathSkill
- fieldsurvey-local-web-ngSkill
- playwright-cliSkill
- serviceradar-bazel-guardrailsSkill
- serviceradar-build-testSkill
- serviceradar-dependenciesSkill
- serviceradar-elixirSkill
- serviceradar-openspecSkill
- serviceradar-operationsSkill
- srql-fixtures-db-testsSkill
- web-ng-docker-loopSkill
The file
Show its rules and text
We split this skill into the rules it gives an agent
25 rules. About 0 are common, 7 are distinctive, and 18 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈7
Distinctive · estimate
This author's own choices. Worth a look.
18
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 15
Update VERSION and the top CHANGELOG entry for the new semver.
- Very specific: line 17
Run scripts/cut-release.sh --version <version> --push.
- Distinctive: line 20
Wait for the published release artifacts to exist for the target semver tag.
- Distinctive: line 28
Use this only for actual release cuts.
- Very specific: line 29
Do not bypass scripts/cut-release.sh unless the user explicitly asks for a different path.
- Distinctive: line 30
Do not treat the release as deployable until the published artifacts for the release version actually exist.
- Distinctive: line 31
Prefer published release artifacts over rebuilding images locally in this workflow.
- Very specific: line 33
Do not re-enable Image Updater, prune, or automated self-heal until the generated-secret, CNPG, and deployment drift called out in …
- Very specific: line 39
Run all of these BEFORE touching VERSION.
- Very specific: line 49
Must end Build completed successfully with no FAIL:.
- Distinctive: line 74
Before cutting the release:
- Very specific: line 77
add the matching top entry in CHANGELOG
- Distinctive: line 89
Create the release commit and annotated tag, then publish the release branch while retaining the tag locally:
- Very specific: line 96
Do not run the no-push form first and then rerun with --push.
- Very specific: line 106
Do not squash or rebase the release PR because that changes or discards the tagged commit.
- Very specific: line 124
Keep the release commit SHA for reporting and traceability.
- Very specific: line 128
Do not roll demo until the release artifacts for the target semver tag exist.
- Very specific: line 161
Verify all of these BEFORE pushing, because the push is what rolls demo:
- Very specific: line 222
After the user pushes, Argo picks the new revision up on its own.
- Very specific: line 254
Use an authenticated Argo CLI context or the Argo UI.
- Very specific: line 261
Do not use --core: it looks for argocd-cm in the active namespace and does not work with this cluster layout.
- Distinctive: line 274
Do not run an operator-triggered sync while the guarded automatic operation is progressing or has succeeded.
- Very specific: line 281
Do not patch global.imageTag directly for a formal release.
- Very specific: line 283
Do not use sr_demo_deploy for formal releases.
- Very specific: line 306
Do not report the rollout finished until the key workloads are running the new v<version> tag and the environment is healthy.
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Release Cut And Demo Roll
Overview
Use this skill for the formal release path: update release metadata, cut the release commit and tag, publish the refs, confirm that the release artifacts exist, and then verify the automatic demo rollout to the released semver image tag, such as v1.2.41. Prefer the repo's existing release script, CI publish path, guarded demo/prod-release branch, and conservative ArgoCD auto-sync over ad hoc local image pushes.
Workflow
- Work from the repo root on the intended release branch.
- Update
VERSIONand the topCHANGELOGentry for the new semver. - Dry-run the release cut to verify the changelog entry is wired correctly.
- Run
scripts/cut-release.sh --version <version> --push; this publishes the release branch only and keeps the annotated tag local. - Open the release pull request and merge it into
stagingwithgh pr merge <PR> --mergeafter CI passes. Squash/rebase would discard the tagged commit. - Fetch
origin/staging, prove the local tag commit is now an ancestor, and publish the tag with an explicit tag refspec. - Wait for the published release artifacts to exist for the target semver tag.
- Confirm the release workflow advanced
demo/prod-releaseand its Argo source file tov<version>. - Confirm
serviceradar-demo-prodstill has automated sync enabled with prune and self-heal disabled, and observe Argo pick up the guarded branch without an operator-triggered sync. - Review any residual drift and watch Argo until
demoreachesSynced|Healthy|Succeeded; use a manual sync only as an investigated recovery action. - Report the release version, release commit, release tag, release-branch revision, automatic-sync result, and final demo rollout status.
Guardrails
- Use this only for actual release cuts. Do not use it for one-off local testing or unpublished commits.
- Do not bypass
scripts/cut-release.shunless the user explicitly asks for a different path. - Do not treat the release as deployable until the published artifacts for the release version actually exist.
- Prefer published release artifacts over rebuilding images locally in this workflow.
- Roll formal releases with the semver image tag, for example
v1.2.41. Thesha-<commit>path is for unpublished local demo testing only. - Do not re-enable Image Updater, prune, or automated self-heal until the generated-secret, CNPG, and deployment drift called out in
k8s/argocd/applications/demo-prod.yamlhas been reviewed and normalized. - While that hold is active,
serviceradar-demo-image-updateris expected to report zero matched applications and zero managed images. Treat0|0|NoErrors:Falseas the configured hold state, not a failed release. - Automatic sync is authorized only for the guarded, publication-verified
demo/prod-releasebranch and must remain non-pruning and non-self-healing. Generated secrets, CNPG resources, and unrelated live drift still require explicit operator judgment.
Pre-Cut Gates
Run all of these BEFORE touching VERSION. Each one has broken a release already.
Every unit test, the way CI runs them. A tag pins its own source, so a fix pushed to
stagingafterwards is invisible to a re-run of that tag -- a broken test found after tagging costs a new tag, not a re-run.make test # bazel test -c opt --config=ci //... --test_tag_filters=-integration_test,-acceptance_testMust end
Build completed successfullywith noFAIL:.go test/cargo test/mix testdo NOT cover this: the Elixir unit shards exist only as bazel targets (//elixir/serviceradar_core:unit_tests_*,//elixir/web-ng:unit_tests_*), which is how two broken Elixir suites reached the v1.4.30 tag.Duplicate migration versions -- must print nothing:
ls elixir/serviceradar_core/priv/repo/migrations/ | grep -oE '^[0-9]+' | sort | uniq -dThis bit v1.3.9 badly: three migrations collided on
20260630120000, and Ecto refuses to run any migration when versions collide, silently bricking auth, metrics and plugin assignment on every fresh install. CI passed because it does not migrate against a prior baseline. If it prints anything, renumber the extras before cutting.- Prod release assembles -- broke v1.3.8.
cd elixir/serviceradar_core && MIX_ENV=prod mix releasemust produce no "Duplicated modules" (the hackney/h2 vs grpcbox/chatterbox conflict). Gate on BOTHserviceradar_coreandserviceradar_core_elx. - macOS needs
gsed(brew install gnu-sed) --cut-release.shuses GNU sed for in-place edits.
Update Release Metadata
Before cutting the release:
- update
VERSION - add the matching top entry in
CHANGELOG
Then dry-run the cut:
scripts/cut-release.sh --version <version> --dry-runFix metadata issues before continuing.
Cut And Push The Release
Create the release commit and annotated tag, then publish the release branch while retaining the tag locally:
scripts/cut-release.sh --version <version> --pushDo not run the no-push form first and then rerun with --push; the first run
already creates the local tag, so the second correctly refuses to proceed.
Open the resulting release pull request and use the ancestry-preserving merge commit method:
gh pr merge <PR> --mergeDo not squash or rebase the release PR because that changes or discards the
tagged commit. Only after that commit is on staging, publish the tag with the
mechanically chained ancestry check printed by the script:
git fetch origin refs/heads/staging:refs/remotes/origin/staging
git merge-base --is-ancestor 'v<version>^{commit}' \
refs/remotes/origin/staging && \
git push origin refs/tags/v<version>:refs/tags/v<version>Afterward, capture:
git rev-parse HEAD
git describe --tags --exact-matchKeep the release commit SHA for reporting and traceability. Use the release version as the demo image tag source: v<version>.
Published Artifact Expectations
Do not roll demo until the release artifacts for the target semver tag exist.
The GitHub Actions release.yml workflow is the supported formal publisher
because the complete release also includes packages, the Helm chart,
managed-agent assets, native add-ons, Wasm plugins, signatures, both import
catalogs, and source/image security bundles. make push_all_release is
retired and exits without publishing. Re-run the failed release.yml job at the release tag
(gh run rerun <run-id> --failed) when recovery is required.
Recovery: Release Published But demo/prod-release Was Not Advanced
The Advance demo release source branch step in .github/workflows/release.yml
is conditional: it runs only when the release is not a prerelease AND the
parallel-asset and finalize steps both succeeded. An unrelated asset failure --
a Wasm plugin publish, for example -- therefore leaves a fully published,
correctly signed release with demo/prod-release still pointing at the previous
version. The release is fine; only the deploy pointer is stale.
What that step actually does is force-push the release commit itself:
git push --force-with-lease demo-release "HEAD:refs/heads/demo/prod-release"where HEAD is the release commit, git rev-list -n1 "refs/tags/v<version>^{commit}".
So the recovery is to reproduce that push, NOT to hand-write a commit that edits
.argocd-source-serviceradar-demo-prod.yaml. That push force-rewrites a shared
branch, which agents must never do: run the checks below, then hand the user the
exact command and let them run it. scripts/cut-release.sh already
wrote global.imageTag: v<version> into that file at the release commit, and
that commit is also the only tree carrying the matching Chart.yaml version and
migration expectedVersion. Adding a tag-only commit on top of the stale branch
would run new images against the old chart.
Verify all of these BEFORE pushing, because the push is what rolls demo:
# 1. Release commit is the tag's commit and is reachable from staging.
RC="$(git rev-list -n1 'v<version>^{commit}')"
git merge-base --is-ancestor "$RC" refs/remotes/origin/staging
./scripts/validate-release-metadata.sh "v<version>" "$RC"
# 2. The file at that commit already carries the tag, with no digest pins.
git show "$RC:helm/serviceradar/.argocd-source-serviceradar-demo-prod.yaml"
# 3. Every image the chart renders exists and is signed with the key Kyverno
# enforces. Render from the release tree, not the working tree.
helm template serviceradar <release-tree>/helm/serviceradar -n demo \
-f <release-tree>/helm/serviceradar/values-demo.yaml \
--set-string global.imageTag=v<version> |
grep -oE 'registry[.]carverauto[.]dev/serviceradar/[a-z0-9.-]+:[^ "]+' | sort -u
cosign verify --key docs/cosign.pub --insecure-ignore-tlog=true <each image>docs/cosign.pub is the same key the verify-serviceradar-images ClusterPolicy
carries; diff them rather than assuming. That policy sets mutateDigest: true,
so admitted pods show :<tag>@sha256:... -- comparing that digest against
crane digest <image>:v<version> is the proof of which artifact is running.
The digest-pin trap
Between releases, demo/prod-release accumulates ad-hoc commits that add
image.digests.<service> parameters pinning individual services to sha-...
builds. Per serviceradar.imageRefSuffix in templates/_helpers.tpl, a digest
pin completely overrides global.imageTag. Bumping only the tag while those
pins remain is inert for exactly the services people care about most.
The force-push discards those commits, which is intended -- but confirm first that nothing regresses. Map each pinned digest back to its build commit and prove it is contained in the release:
for t in $(crane ls registry.carverauto.dev/serviceradar/<image> | grep '^sha-'); do
[ "$(crane digest registry.carverauto.dev/serviceradar/<image>:$t)" = "<pinned digest>" ] && echo "$t"
done
git merge-base --is-ancestor <build-commit> 'v<version>^{commit}'A pin whose build commit is NOT an ancestor is not automatically a regression --
branch builds are often squash-merged, so the work lands under a different SHA.
Resolve it by diffing the owning package between that commit and the release tag
rather than by trusting the ancestry check alone. Pinning commits also sometimes
carry chart edits that were never upstreamed; diff
origin/demo/prod-release..v<version> -- helm/serviceradar/ and account for
every removal before pushing.
The command to hand over, with an explicit lease on the branch head you verified and an explicit refspec:
OLD="$(git rev-parse refs/remotes/origin/demo/prod-release)"
git push --force-with-lease="refs/heads/demo/prod-release:${OLD}" \
origin "${RC}:refs/heads/demo/prod-release"After the user pushes, Argo picks the new revision up on its own. Do not sync manually;
confirm the operation was automatic with
.status.operationState.operation.initiatedBy.automated.
Roll Demo To The Release Tag
The release workflow advances demo/prod-release only after the complete
release, catalog, and security asset set verifies. Confirm that branch now
carries the target tag:
git fetch origin \
refs/heads/demo/prod-release:refs/remotes/origin/demo/prod-release
git show origin/demo/prod-release:helm/serviceradar/.argocd-source-serviceradar-demo-prod.yamlThe source file must set global.imageTag to v<version>. scripts/validate-release-metadata.sh fails the release when that pin is not the release tag (docs/RELEASE_PUBLISHING.md). Confirm the conservative automatic-sync policy is still active:
kubectl get application -n argocd serviceradar-demo-prod \
-o jsonpath='{.spec.syncPolicy.automated.enabled}{"|"}{.spec.syncPolicy.automated.prune}{"|"}{.spec.syncPolicy.automated.selfHeal}{"\n"}'Expect true|false|false. Then check the Image Updater hold state for context:
kubectl get imageupdater -n argocd serviceradar-demo-image-updater \
-o jsonpath='{.status.applicationsMatched}{"|"}{.status.imagesManaged}{"|"}{range .status.conditions[?(@.type=="Error")]}{.reason}{":"}{.status}{":"}{.message}{end}{"\n"}'Expect zero matched applications, zero managed images, and an Error condition with reason NoErrors and status False while the Image Updater hold is active.
Use an authenticated Argo CLI context or the Argo UI. The usual local CLI context expects the Argo server on 127.0.0.1:18443; start its port-forward in a separate shell when it is not already reachable:
kubectl port-forward -n argocd svc/argocd-server 18443:443
argocd app get serviceradar-demo-prod --refreshDo not use --core: it looks for argocd-cm in the active namespace and does not work with this cluster layout.
Refresh the application and inspect any residual drift while the guarded automatic sync progresses:
kubectl get application -n argocd serviceradar-demo-prod -o json |
jq -r '.status.resources[] | select(.status != "Synced") | [.group, .kind, .namespace, .name, .status] | @tsv'
argocd app diff serviceradar-demo-prod --hard-refresh
kubectl get application -n argocd serviceradar-demo-prod -o yamlargocd app diff exits 1 when a diff exists and intentionally omits Kubernetes Secrets. Inspect every non-release change in its output, inspect each OutOfSync Secret's owner/hook annotations separately, and review CNPG resources explicitly. Remove or account for stale live-only Application parameters rather than broadening the automatic policy.
Do not run an operator-triggered sync while the guarded automatic operation is progressing or has succeeded. If automatic sync fails or remains stuck after investigation, a reviewed recovery sync may be dry-run and executed without prune:
argocd app sync serviceradar-demo-prod --dry-run
argocd app sync serviceradar-demo-prod --assumeYesDo not patch global.imageTag directly for a formal release. That creates another live-only override and bypasses the verified demo/prod-release source.
Do not use sr_demo_deploy for formal releases; that helper rolls demo to a sha-... tag and is only appropriate for local unpublished test builds.
Verify Demo Rollout
Watch the Argo app or Helm result until the deployment is healthy. For Argo-backed environments, wait for:
Synced|Healthy|SucceededUseful checks:
kubectl get application -n argocd serviceradar-demo-prod \
-o jsonpath='{.status.sync.status}{"|"}{.status.health.status}{"|"}{.status.operationState.phase}{"\n"}'
kubectl get pods -n demo
kubectl get deploy -n demo \
serviceradar-web-ng serviceradar-core serviceradar-agent serviceradar-tools \
-o jsonpath='{range .items[*]}{.metadata.name}{"\t"}{range .spec.template.spec.containers[*]}{.image}{" "}{end}{"\n"}{end}'Do not report the rollout finished until the key workloads are running the new v<version> tag and the environment is healthy.
Report Back
Close with:
- release version
- release commit SHA
- release tag
- whether refs were pushed
- whether release artifacts were confirmed published
demo/prod-releaserevision and image tag- Image Updater hold status
- automatic-sync result, or the investigated manual recovery result when one was required
- final
demorollout status - any residual risk or follow-up needed
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

