agentleFS
UKGovernmentBEISUKGovernmentBEIS/inspect_evals/.claude/skills/prepare-release/SKILL.md

prepare-release

Skill735 starsChanged 35 days agoMITView on GitHub

Prepare a new release of inspect_evals by creating a release branch, collecting changelog fragments, and opening a PR. Use when user asks to cut/prepare/create a new release or version bump.

  • Commits and pushes

49 other projects use a version of this.

Adopted by

Most recently taken up by

Used with

Projects with it also use publish-release, sync-docs and 7 more.

Other files people use alongside this one.

Files found in the same public projects as a version of this one, more often than they turn up anywhere else. Each project counts once, large collections count for less, and this file's own repository is left out.
  • Model Context ProtocolSkill

    publish-release

    Publish a GitHub release for the C# MCP SDK after a prepare-release PR has been merged. Refreshes release notes to include any PRs merged since preparation, warns about version or breaking change impacts from late-arriving PRs, and creates a draft GitHub release. Use when asked to publish a release, finalize a release, create release notes, or complete a release after the prepare-release PR has been merged.

    Also in 3 projects that use this

  • Eipi-TongSkill

    sync-docs

    Update SPEC.md and docs/DECISIONS.md to reflect recent code changes. Use after completing a feature, or when the user says "sync docs", "update the spec", or "log this decision".

    Also in 4 projects that use this

  • thlaureSkill

    review-change

    Use this skill when the user asks for a code review, a correctness pass, a check before PR, or asks whether a Signalist change follows project conventions.

    Also in 3 projects that use this

  • DockerSkill

    fix-issue

    Fix a single GitHub issue end-to-end: triage, research, write the fix, review, and create a PR. Use when asked to fix an issue: "fix issue 1234", "resolve #500", "create a PR for issue 200".

    Also in 4 projects that use this

  • kiminmonacoSkill

    implement-feature

    Guide for implementing features in ClaudeBar following architecture-first design, TDD, rich domain models, and Swift 6.2 patterns. Use this skill when: (1) Adding new functionality to the app (2) Creating domain models that follow user's mental model (3) Building SwiftUI views that consume domain models directly (4) User asks "how do I implement X" or "add feature Y" (5) Implementing any feature that spans Domain, Infrastructure, and App layers

    Also in 3 projects that use this

  • SentrySkill

    write-docs

    Create or update documentation pages for the Abacus docs site. Use when adding new documentation, updating existing pages, or ensuring docs stay in sync with code changes. Ensures pages follow Starlight patterns and build correctly.

    Also in 3 projects that use this

The file

We split this skill into the rules it gives an agent

5 rules. 0 are common, 2 are distinctive, and 3 are very specific to this project.

  • 0

    Common

    Most of the 48 edited versions we read keep these.

  • 2

    Distinctive

    This author's own choices. Worth a look.

  • 3

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    If the merge fails, stop and inform the user that their local main has diverged from origin/main.

    line 28
  2. Very specific:

    Ask the user what kind of bump this is, presenting the semver table from PACKAGE_VERSIONING.md:

    line 42
  3. Distinctive:

    If the second pass still reports changes, resolve them before committing.

    line 93
  4. Very specific:

    No manual action is required.

    line 135
  5. Distinctive:

    If something goes wrong mid-workflow, the release branch can be deleted and the process restarted.

    line 140

“Common” is measured: kept by at least 6 in 10 of the 48 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Prepare a Release

This workflow prepares a new release of inspect_evals. It creates a release branch, collects changelog fragments, bumps the version tag, and opens a PR. After merge it tags the merge commit and creates a GitHub release.

Reference: PACKAGE_VERSIONING.md

Prerequisites

  • The gh CLI must be authenticated
  • You must have push access to the repository
  • There must be changelog fragments in changelog.d/ (beyond .gitkeep and TEMPLATE.md)

Phase 1 — Prepare the release branch

  1. Ensure main is up to date:

    git fetch origin
    git checkout main
    git merge --ff-only origin/main

    If the merge fails, stop and inform the user that their local main has diverged from origin/main.

  2. Check for changelog fragments:

    List files in changelog.d/ excluding .gitkeep, TEMPLATE.md, and README.*. If there are no fragments, stop and tell the user there is nothing to release.

  3. Determine the new version:

    1. Find the current version from the latest v* git tag:

      git tag --sort=-v:refname --list 'v*' | head -1
    2. Ask the user what kind of bump this is, presenting the semver table from PACKAGE_VERSIONING.md:

      ComponentWhen to bumpExamples
      MajorBreaking changesRemoving an eval, API changes, scorer output format changes
      MinorNew featuresAdding new evals, new task parameters, new utilities
      PatchBug fixesEval fixes, scorer fixes, dataset loading fixes
    3. Compute the new version string (e.g. 0.4.0, 0.3.107). Confirm with the user.
  4. Create the release branch:

    The branch name is release-YYYY-MM-DD using today's date.

    git checkout -b release-YYYY-MM-DD

Phase 2 — Collect the changelog

  1. Run scriv collect:

    uv run scriv collect --version <NEW_VERSION>

    This removes the individual fragment files from changelog.d/ and prepends a new section to CHANGELOG.md.

  2. Present the changelog diff to the user for review:

    git diff CHANGELOG.md

    Tell the user: "Please review the collected changelog above. You can edit CHANGELOG.md directly — let me know when you are satisfied, or tell me what changes to make."

    Do not proceed until the user confirms the changelog is ready.

  3. Normalise the collected changelog:

    changelog.d/ fragments are excluded from the mdformat and markdownlint-fix pre-commit hooks (fragments are partial documents), so a hand-edit during review — or an unusual fragment — can leave CHANGELOG.md in a state those hooks would rewrite. That turns the Markdown Lint and README check jobs red on main after the release PR merges (see #2225, #2329). Run the hooks over CHANGELOG.md, then confirm a second pass is clean:

    uv run --group dev pre-commit run mdformat --files CHANGELOG.md || true
    uv run --group dev pre-commit run markdownlint-fix --files CHANGELOG.md || true
    # A second pass must be clean: applying fixes is fine, failing to
    # converge means a real error that would still break main.
    uv run --group dev pre-commit run mdformat --files CHANGELOG.md
    uv run --group dev pre-commit run markdownlint-fix --files CHANGELOG.md

    If the second pass still reports changes, resolve them before committing.

  4. Stage and commit:

    git add CHANGELOG.md changelog.d/
    git commit -m "Prepare release v<NEW_VERSION>"

Phase 3 — Push and open a PR

  1. Push the branch:

    git push -u origin release-YYYY-MM-DD
  2. Open a draft PR:

    <!-- markdownlint-disable-next-line no-space-in-code -->

    Extract the new version's section from CHANGELOG.md (everything from the version heading up to but not including the next ## heading). Use this as the PR body:

    gh pr create --draft \
      --title "Release v<NEW_VERSION>" \
      --body "<EXTRACTED_CHANGELOG_SECTION>"

    Important: The PR title must start with Release v (e.g. Release v0.4.0) — this is how the release-on-merge.yml workflow identifies release PRs.

    Tell the user the PR URL and that it is in draft. They should mark it ready for review when appropriate.

Phase 4 — After the PR is merged (automated)

Once the release PR is merged into main, the .github/workflows/release-on-merge.yml GitHub Actions workflow automatically:

  1. Tags the merge commit with v<NEW_VERSION>
  2. Creates a GitHub release with the changelog section as the body
  3. Builds and publishes the package to PyPI
  4. Notifies Slack

No manual action is required. The package version is derived from the new tag by setuptools_scm.

Notes

  • The package version is derived entirely from git tags via setuptools_scm — there is no version file to edit.
  • If something goes wrong mid-workflow, the release branch can be deleted and the process restarted.
  • The v prefix on tags is required (e.g. v0.3.107, not 0.3.107).
  • The weekly-release.yml workflow can also automate Phases 1–3 on a schedule or via manual dispatch, creating the release branch and draft PR for you.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.