agentleFS
r0shir0shi/harborclerk/.claude/skills/acceptance/SKILL.md

acceptance

Skill3 starsChanged 32 days agoMITView on GitHub

Run the API-tier acceptance suite against a live Harbor Clerk instance, write the smoke-matrix report into docs/reports/, and open the report PR as the machine user. Use before a release, after a change to ingest, search, MCP, CLI or keys, or on a schedule.

  • Commits and pushes

7 other projects use a version of this.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

6 rules. 0 are common, 5 are distinctive, and 1 is very specific to this project.

  • 0

    Common

    Most of the 6 edited versions we read keep these.

  • 5

    Distinctive

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    If either is missing, stop and say which one.

    line 27
  2. Very specific:

    Only on the mini, pass the native app's config so the CLI-gate checks can flip and restore enable_cli_access.

    line 36
  3. Distinctive:

    Read the summary line and the exit code.

    line 59
  4. Distinctive:

    Commit or stash, then:

    line 66
  5. Distinctive:

    Use a scratch worktree so the operator's checkout is never switched, add the one file the generator printed, and sign nothing with the …

    line 89
  6. Distinctive:

    Write the PR body to a file first: the counts and duration, each failing check with its assertion message, the expected failures with their …

    line 106

“Common” is measured: kept by at least 6 in 10 of the 6 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Acceptance run

Drives a running instance through every product claim the API tier can check (docs/superpowers/specs/2026-09-16-acceptance-suite-design.md), then turns the result into docs/reports/YYYY-MM-DD-acceptance-<host>.md and a PR. All commands run from the repository root.

Preflight

  1. The instance must be up. On the mini it is the native app on http://localhost:8100:
    curl -s http://localhost:8100/api/system/health | jq '{status, build, enable_cli_access}'
    status must be healthy. Note build; the report records it.
  2. Credentials come from Keychain, never from a prompt or a file:
    • admin login: service harbor-clerk-acceptance, account = the admin email;
    • the machine user's GitHub token: account harborclerk-bot, service github-token.
    security find-generic-password -s harbor-clerk-acceptance | sed -n 's/.*"acct"<blob>="\(.*\)"/admin: \1/p'
    If either is missing, stop and say which one.
  3. Decide the mode. Folder-scoped is the default and is safe on any instance. It adds and removes four watched folders (the fixtures, an empty one, one with a single document, one for H2), soft-deletes one of its own documents, creates and deletes two conversations, and leaves audit rows and soft-deleted keys; no other document is touched. Wipe mode (HC_ACCEPTANCE_WIPE=1) empties the instance and is only for the mini: it also needs HC_ACCEPTANCE_DISPOSABLE=1, a loopback HC_API_BASE, and at most 500 documents on the instance.
  4. Only on the mini, pass the native app's config so the CLI-gate checks can flip and restore enable_cli_access; set CONFIG_JSON empty elsewhere (the suite refuses it for a remote instance anyway). The Ask checks use the model that is active; with HC_ACCEPTANCE_DISPOSABLE=1 or HC_ACCEPTANCE_ALLOW_MODEL_SWAP=1 they may activate the smallest downloaded model when none is; it then stays active and becomes the app's persisted default (activation writes llm_model_id to config.json).

Run

set -o pipefail; RUN=acc-$(date +%Y%m%d-%H%M); WORK=$(mktemp -d)   # private: the log carries assertion bodies
CONFIG_JSON="$HOME/Library/Application Support/Harbor Clerk/config.json"   # mini only; "" elsewhere
HC_API_BASE=http://localhost:8100 \
HC_USERNAME="$(security find-generic-password -s harbor-clerk-acceptance | sed -n 's/.*"acct"<blob>="\(.*\)"/\1/p')" \
HC_PASSWORD="$(security find-generic-password -s harbor-clerk-acceptance -w)" \
HC_ACCEPTANCE_RUN_ID=$RUN HC_ACCEPTANCE_CONFIG_JSON="$CONFIG_JSON" \
uv run pytest acceptance/ -m acceptance -v -p no:cacheprovider --junitxml=$WORK/$RUN.xml 2>&1 | tee $WORK/$RUN.log | tail -40
echo "pytest exit: $?"   # pipefail makes this pytest's code in bash and zsh alike

Expect six to seven minutes: ingest with OCR, a reprocess, a live-added file, a rate-limit recovery wait, an expiring key, and two Asks if a model is active. Read the summary line and the exit code. A failure is information, not something to retry blind.

Report

The tree must be clean first: the report cites the suite commit, and a -dirty suffix in the header means it was generated from uncommitted code. Commit or stash, then:

HC_USERNAME="$(security find-generic-password -s harbor-clerk-acceptance | sed -n 's/.*"acct"<blob>="\(.*\)"/\1/p')" \
HC_PASSWORD="$(security find-generic-password -s harbor-clerk-acceptance -w)" \
uv run python -m acceptance.report --junit $WORK/$RUN.xml --out docs/reports/ \
  --api-base http://localhost:8100 --run-id $RUN --probe ${HC_ACCEPTANCE_WIPE:+--wipe}

It prints the path it wrote (one file per run, never overwritten). --probe logs in and records the instance version and build, the active model and the platform in the header.

The generator redacts every HC_* value before writing. Read the report once before committing it: it is the durable trace of this run.

Publish as the machine user

Agent-produced work is opened as the bot (AGENTS.md, Identities). Branch from a fresh main so the report PR never stacks. GH_TOKEN steers gh only; git pushes through a credential helper, so give git the bot's token per command via an inline helper (it never reaches the process list or the environment).

Use a scratch worktree so the operator's checkout is never switched, add the one file the generator printed, and sign nothing with the owner's key.

REPORT=docs/reports/$(ls -t docs/reports | head -1)   # the file --out just printed
git fetch -q origin main && WT=$(mktemp -d) && git worktree add -q "$WT" -b report/acceptance-$RUN origin/main && \
cp "$REPORT" "$WT/$REPORT" && git -C "$WT" add "$REPORT" && \
git -C "$WT" -c user.name="John-Doebot" -c user.email="330166301+John-Doebot@users.noreply.github.com" -c commit.gpgsign=false \
  commit -m "docs(reports): acceptance run $RUN" && \
git -C "$WT" -c credential.helper= \
    -c 'credential.helper=!f() { echo username=John-Doebot; echo "password=$(security find-generic-password -a harborclerk-bot -s github-token -w)"; }; f' \
    push -u origin report/acceptance-$RUN && git worktree remove "$WT"

The inline helper covers the HTTPS remote this repository uses; an SSH remote would need the bot's deploy key instead.

Write the PR body to a file first: the counts and duration, each failing check with its assertion message, the expected failures with their issue references, and what the run left on the instance. Then:

GH_TOKEN="$(security find-generic-password -a harborclerk-bot -s github-token -w)" \
  gh pr create --base main --title "docs(reports): acceptance run $RUN" --body-file /tmp/hc-acceptance/$RUN.body.md

CODEOWNERS routes the review to the owner. Failing checks become issues only when the owner asks.

Afterwards

Confirm the instance is clean. The suite fails loudly when a key cannot be deleted or the CLI gate did not follow the file; folder and conversation removal are attempted, not verified, so verify them:

HC_USERNAME="$(security find-generic-password -s harbor-clerk-acceptance | sed -n 's/.*"acct"<blob>="\(.*\)"/\1/p')" \
HC_PASSWORD="$(security find-generic-password -s harbor-clerk-acceptance -w)" \
uv run python -m acceptance.report --junit $WORK/$RUN.xml --out /dev/null --api-base http://localhost:8100 --run-id $RUN --verify-clean

Exit 0 and no output means clean; each line printed is something the run left (a hc-acceptance-* folder, an active acceptance-* key, an acceptance-* conversation). Also confirm config.json's enable_cli_access is as it was. If anything is left, say so first, before anything about the results.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.