agentleFS
mckinseymckinsey/agents-at-scale-ark/tests/CLAUDE.md

agents-at-scale-ark / tests

CLAUDE.md438 starsChanged 7 months agoApache-2.0View on GitHub

This document covers best practices for writing chainsaw tests in the Ark project. Tests use labels in chainsaw-test.yaml metadata to control when they run. bash chainsaw test Use assert: only for post-completion validation where the query is already

The file

Show its rules and text

We split this CLAUDE.md into the rules it gives an agent

28 rules. About 0 are common, 10 are distinctive, and 18 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈10

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 18

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Use a00-, a01-, etc. prefixes to control application order

    line 57
  2. Distinctive:

    All tests that create queries MUST include RBAC configuration:

    line 64
  3. Distinctive:

    Use default service account, not custom ones

    line 94
  4. Very specific:

    Use ($namespace) template for namespace references

    line 97
  5. Very specific:

    Use Go template syntax: {{.parameter_name}}

    line 102
  6. Very specific:

    Use wait: with the Completed condition to wait for query completion.

    line 135
  7. Very specific:

    Use assert: only for post-completion validation where the query is already known to be done:

    line 148
  8. Distinctive:

    Use Chainsaw's JP functions for response validation instead of shell scripts:

    line 210
  9. Distinctive:

    Test queries with label selectors to validate target selection:

    line 255
  10. Very specific:

    Create a mock-llm-values.yaml in the test directory to script the mock's responses.

    line 309
  11. Very specific:

    Use --skip-delete flag to keep resources after test failure:

    line 367
  12. Distinctive:

    When queries fail, examine events to understand the issue:

    line 373
  13. Distinctive:

    Avoid adding labels to resources unless specifically required:

    line 387
  14. Very specific:

    Don't assert status.phase on resources that don't have it (like Agents).

    line 435
  15. Distinctive:

    Ensure ConfigMaps and Secrets exist before resources that reference them.

    line 438
  16. Very specific:

    All tests that use helm install should include explicit cleanup sections to uninstall Helm releases.

    line 538
  17. Very specific:

    Add a cleanup section at the same indentation level as catch or try:

    line 542
  18. Very specific:

    Placement: Add cleanup at the same indentation as catch blocks within the last step

    line 557
  19. Very specific:

    Timeout: Use --wait --timeout=180s to match chainsaw's cleanup timeout

    line 560
  20. Very specific:

    Error handling: Always use || true to prevent cleanup failures if releases don't exist

    line 561
  21. Distinctive:

    All chainsaw tests with Query assertions should include catch blocks to reduce verbosity and provide debugging information:

    line 607
  22. Distinctive:

    When validating events in tests, check for presence rather than exact counts to avoid flakiness:

    line 623
  23. Very specific:

    Do NOT require [data-side] in the listbox selector — Radix Popper sets it asynchronously and it may not be present immediately in headless …

    line 696
  24. Very specific:

    Use page.evaluate(fn, handle) rather than locator.evaluate(fn) — the latter can cause Playwright to refocus the element, which closes Radix …

    line 727
  25. Distinctive:

    No RBAC permission errors in events

    line 734
  26. Very specific:

    Use this label when a test needs to verify broker message/event/session persistence, expires_at, or other Postgres-specific behavior.

    line 740
  27. Very specific:

    Use the shared psql-query.sh script to run SQL against ark-storage-dev:

    line 744
  28. Very specific:

    Use psql-poll.sh, which resolves the connection once and runs the whole retry loop inside a single kubectl exec against local psql.

    line 768

≈ “Common” is an estimate: at least 1 in 5 of the 27,641 CLAUDE.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
CLAUDE.md

Chainsaw Testing Guide

This document covers best practices for writing chainsaw tests in the Ark project.

Test Taxonomy

Tests use labels in chainsaw-test.yaml metadata to control when they run.

Label-Based Selectors

LabelMeaningCI Trigger
(no label)Standard tests, use mock-llmAlways runs (!llm,!postgresql or !llm)
llm: "true"Requires real LLM API keyse2e-tests-llm job only
postgresql: "true"Requires PostgreSQL backend + broker with postgres backendExcluded from etcd-only runs
etcd-only: "true"Requires etcd backend (e.g., uses cluster-scoped CRDs not served by embedded apiserver)Excluded from postgresql backend runs
requires-images: "true"Requires built container imagesConditional
standard: "true"Explicit standard markerAlways runs

Basic Test Layout

tests/
├── test-name/
│   ├── chainsaw-test.yaml
│   ├── README.md             # Required test documentation
│   └── manifests/
│       ├── a00-rbac.yaml
│       ├── a01-secrets.yaml
│       ├── a02-configmaps.yaml
│       ├── a03-model.yaml
│       ├── a04-agent.yaml
│       └── a05-query.yaml

README Documentation

Each test directory MUST include a README.md file with this format:

# Test Name

Brief description of what the test validates.

## What it tests
- Specific functionality being tested
- Key components or integrations
- Expected behaviors or outcomes

## Running
```bash
chainsaw test

One sentence explaining what successful test completion validates.


### File Naming Convention
- Use `a00-`, `a01-`, etc. prefixes to control application order
- RBAC files should be first (`a00-rbac.yaml`)
- Dependencies should come before dependents (models before agents, agents before queries)

## RBAC Requirements

### Essential Pattern
All tests that create queries MUST include RBAC configuration:

```yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: query-test-role
rules:
- apiGroups: ["ark.mckinsey.com"]
  resources: ["*"]
  verbs: ["*"]
- apiGroups: [""]
  resources: ["secrets", "configmaps"]
  verbs: ["get", "list", "watch"]
---
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: query-test-rolebinding
subjects:
- kind: ServiceAccount
  name: default
  namespace: ($namespace)
roleRef:
  kind: Role
  name: query-test-role
  apiGroup: rbac.authorization.k8s.io

Key Points

  • Use default service account, not custom ones
  • Grant full permissions to ark.mckinsey.com resources
  • Include secrets and configmaps access for parameter resolution
  • Use ($namespace) template for namespace references

Parameter Templating

Template Syntax

Use Go template syntax: {{.parameter_name}}

Parameter Sources

  • Direct values: {{.agent_name}}
  • ConfigMap references: {{.response_style}}
  • Secret references: {{.special_instruction}}

Example Agent with Parameters

spec:
  prompt: |
    You are {{.agent_name}} running in {{.test_mode}} mode.
    Your role is: {{.agent_role}}
    Response style: {{.response_style}}
    Maximum tokens per response: {{.max_tokens}}
    API endpoint for additional data: {{.api_endpoint}}
    Special instructions: {{.special_instruction}}

Resource Assertions

Agent Assertions

Agents don't have a status.phase field, so only assert existence:

- assert:
    resource:
      apiVersion: ark.mckinsey.com/v1alpha1
      kind: Agent
      metadata:
        name: test-agent

Query Assertions

Use wait: with the Completed condition to wait for query completion. This uses a Kubernetes watch instead of polling, which reduces API server load:

- wait:
    apiVersion: ark.mckinsey.com/v1alpha1
    kind: Query
    name: test-query
    timeout: 4m
    for:
      condition:
        name: Completed
        value: 'True'

Use assert: only for post-completion validation where the query is already known to be done:

- assert:
    resource:
      apiVersion: ark.mckinsey.com/v1alpha1
      kind: Query
      metadata:
        name: test-query
      status:
        phase: done

Never use contains() on response fields without a preceding wait.

JMESPath's contains() requires a string or array — it throws a hard type error if the value is nil. Before the query completes, response is nil, so any assertion like (contains(response.content, 'foo')): true will immediately error rather than retry:

# Bad - crashes on nil if query hasn't completed yet
- apply:
    file: manifests/a04-query.yaml
- assert:
    resource:
      ...
      status:
        (contains(response.content, 'expected text')): true

# Good - wait until response.content is populated, then assert against it
- apply:
    file: manifests/a04-query.yaml
- wait:
    apiVersion: ark.mckinsey.com/v1alpha1
    kind: Query
    name: test-query
    timeout: 2m
    for:
      jsonPath:
        path: '{.status.response.content}'
- assert:
    resource:
      ...
      status:
        (contains(response.content, 'expected text')): true

Prefer for.jsonPath over for.condition: Completed=True whenever the next step reads response.content. Completed=True can become visible on a watch before response.content has propagated to a subsequent read (especially on the postgresql backend, where watch events hop through the WAL consumer), and contains(nil, ...) errors rather than failing-with-retry. Waiting on the jsonpath itself fires only once the field is non-empty, so the follow-up assert and any shell-script kubectl get … jsonpath='{.status.response.content}' (which has no retry of its own) both see a populated value.

Completed=True is still the right wait for tests that only care that the query terminated — for example, error queries where the body of response.content is not what's under test.

Model Assertions

Models should assert existence and readiness:

- assert:
    resource:
      apiVersion: ark.mckinsey.com/v1alpha1
      kind: Model
      metadata:
        name: test-model

Query Response Validation

Using JP Functions

Use Chainsaw's JP functions for response validation instead of shell scripts:

# Validate response exists
- assert:
    resource:
      apiVersion: ark.mckinsey.com/v1alpha1
      kind: Query
      metadata:
        name: test-query
      status:
        (response != null): true

# Validate specific agent responded
- assert:
    resource:
      apiVersion: ark.mckinsey.com/v1alpha1
      kind: Query
      metadata:
        name: test-query
      status:
        (response.target.name): 'expected-agent'

# Validate agent did NOT respond (check different target name)
- assert:
    resource:
      apiVersion: ark.mckinsey.com/v1alpha1
      kind: Query
      metadata:
        name: test-query
      status:
        (response.target.name != 'excluded-agent'): true

# Validate response content length
- assert:
    resource:
      apiVersion: ark.mckinsey.com/v1alpha1
      kind: Query
      metadata:
        name: test-query
      status:
        (length(response.content) > `50`): true

Label Selector Testing

Test queries with label selectors to validate target selection:

# Query with matchLabels selector
apiVersion: ark.mckinsey.com/v1alpha1
kind: Query
metadata:
  name: test-query-selector
spec:
  input: Test query for label selection
  selector:
    matchLabels:
      environment: production
      type: specialist

Using Mock LLM

Standard tests use mock-llm instead of a real LLM. Mock-llm is a configurable HTTP server that intercepts LLM API calls and returns scripted responses, making tests fast, deterministic, and runnable without API keys.

Only use a real LLM (with llm: "true" label) when the test genuinely requires actual language model reasoning — for example, multi-provider behavioral testing under tests/llm-tests/. Everything else should use mock-llm.

Setup Pattern

Install mock-llm via the shared script, then wait for the Model CR it creates:

- name: setup-mock-llm
  try:
  - script:
      timeout: 180s
      content: |
        bash ../shared/install-mock-llm.sh
      env:
      - name: NAMESPACE
        value: ($namespace)

- name: wait-for-model-ready
  try:
  - wait:
      apiVersion: ark.mckinsey.com/v1alpha1
      kind: Model
      name: test-model-mock
      timeout: 90s
      for:
        condition:
          name: ModelAvailable
          value: 'True'

The script installs the mock-llm Helm release using mock-llm-values.yaml from the test directory (if present) or the shared defaults. It creates a Model CR named test-model-mock.

Configuring Responses

Create a mock-llm-values.yaml in the test directory to script the mock's responses. Rules are evaluated in order — last match wins, so place the 500 fallback first and specific rules last:

config:
  rules:
  - path: "/v1/chat/completions"
    response:
      status: 500
      content: '"Unrecognised request"'

  - path: "/v1/chat/completions"
    match: "contains(body.messages[0].content || '', 'my agent')"
    response:
      status: 200
      content: '{"choices":[{"message":{"role":"assistant","content":"Hello from mock"},"finish_reason":"stop"}],"usage":{"prompt_tokens":10,"completion_tokens":5,"total_tokens":15}}'

Cleanup

Uninstall mock-llm in the last step's cleanup block:

    cleanup:
    - script:
        content: |
          helm uninstall mock-llm --namespace $NAMESPACE --wait --timeout=180s || true
        env:
        - name: NAMESPACE
          value: ($namespace)

Resource Dependencies

Dependency Order

  1. RBAC (Role, RoleBinding)
  2. mock-llm setup (if needed)
  3. Other dependencies (ConfigMaps, Tools, etc.)
  4. Agents (reference test-model-mock)
  5. Queries (depend on Agents)

Model Reference Pattern

Agents reference the mock model by name:

# Agent references mock model
spec:
  modelRef:
    name: test-model-mock

# Query references agent
spec:
  agent: test-agent

Debugging and Troubleshooting

Keep Resources for Investigation

Use --skip-delete flag to keep resources after test failure:

chainsaw test tests/queries/ --skip-delete

Check Kubernetes Events

When queries fail, examine events to understand the issue:

kubectl get events --sort-by='.lastTimestamp'
kubectl describe query test-query

Common Event Messages

  • agents.ark.mckinsey.com "test-agent" is forbidden - Missing RBAC permissions
  • secrets "test-token" is forbidden - Missing secrets access in RBAC
  • Query stuck in phase: error - Check agent/model dependencies

Common Pitfalls and Spec Errors

Don't Use Labels

Avoid adding labels to resources unless specifically required:

# Bad
metadata:
  name: test-agent
  labels:
    test: "true"

# Good  
metadata:
  name: test-agent

Resource Spec Format Errors

Agent Model Reference

# Wrong - causes "unknown field spec.model" error
spec:
  model: test-model

# Correct - use modelRef
spec:
  modelRef:
    name: test-model

Team Members vs Targets

# Wrong - causes "unknown field spec.targets" error
spec:
  strategy: sequential
  targets:
    - type: agent
      name: test-agent

# Correct - use members
spec:
  strategy: sequential
  members:
    - type: agent
      name: test-agent

Missing RBAC

Query tests will fail with forbidden errors without proper RBAC configuration.

Wrong Phase Assertion

Don't assert status.phase on resources that don't have it (like Agents).

Parameter Resolution

Ensure ConfigMaps and Secrets exist before resources that reference them.

Testing Services with Helm Charts

MCP Services Pattern

For MCP services and other services that have Helm chart packaging, use this deployment pattern:

# Deploy service using Helm chart
- name: deploy-service-with-helm
  try:
  - script:
      content: |
        helm install service-name ../chart --namespace $NAMESPACE --wait --timeout=30s
      env:
      - name: NAMESPACE
        value: ($namespace)

Service Test Structure

mcp/service-name/test/
├── chainsaw-test.yaml
├── README.md
└── manifests/
    ├── a00-rbac.yaml
    ├── a04-secret.yaml
    ├── a05-model.yaml
    ├── a06-agent.yaml
    └── a07-query.yaml

MCPServer Integration

Services deployed via Helm that create MCPServer resources require:

  1. Wait for MCPServer readiness:
- assert:
    resource:
      apiVersion: ark.mckinsey.com/v1alpha1
      kind: MCPServer
      metadata:
        name: service-name
  1. Agent tools reference auto-generated Tool resources:
spec:
  tools:
  - name: service-name-tool-1
    type: mcp
  - name: service-name-tool-2
    type: mcp
  1. Additional RBAC for service discovery:
rules:
- apiGroups: ["ark.mckinsey.com"]
  resources: ["*"]
  verbs: ["*"]
- apiGroups: [""]
  resources: ["secrets", "configmaps", "services"]
  verbs: ["get", "list", "watch"]

Response Content Validation

For functional testing, validate that service operations actually worked:

- name: validate-response-content
  try:
  - assert:
      resource:
        apiVersion: ark.mckinsey.com/v1alpha1
        kind: Query
        metadata:
          name: test-query
        status:
          # Validate response mentions expected operations
          (contains(response.content, 'operation-evidence')): true
  - script:
      content: |
        RESPONSE=$(kubectl -n $NAMESPACE get query test-query -o jsonpath='{.status.response.content}')
        
        echo "=== Query Response Content ==="
        echo "$RESPONSE"
        echo "=========================="
        
        # Validate specific operations mentioned in response
        if echo "$RESPONSE" | grep -qi "expected-operation"; then
          echo "✓ Response mentions expected operation"
        else
          echo "✗ Response missing expected operation"
          exit 1
        fi

Cleanup Requirements

All tests that use helm install should include explicit cleanup sections to uninstall Helm releases.

Cleanup Pattern

Add a cleanup section at the same indentation level as catch or try:

    cleanup:
    - script:
        content: |
          helm uninstall ark-tenant --namespace $NAMESPACE --wait --timeout=180s || true
          helm uninstall mock-llm --namespace $NAMESPACE --wait --timeout=180s || true
        env:
        - name: NAMESPACE
          value: ($namespace)

Key Points

  • Placement: Add cleanup at the same indentation as catch blocks within the last step
  • Blank line: Include one blank line before the cleanup: section
  • Order: Uninstall charts in reverse order of installation when multiple charts exist
  • Timeout: Use --wait --timeout=180s to match chainsaw's cleanup timeout
  • Error handling: Always use || true to prevent cleanup failures if releases don't exist
  • Why required: Explicit uninstalls are faster and more reliable than cascading namespace deletion

Example Test Structure

spec:
  steps:
  - name: setup-and-test
    try:
    - script:
        content: |
          # requireBroker=false: an ephemeral test namespace has no broker, and
          # the chart's preflight refuses to install without one.
          helm install ark-tenant ../../charts/ark-tenant --namespace $NAMESPACE --create-namespace --wait \
            --set memory.requireBroker=false
    - apply:
        file: manifests/*.yaml
    - wait:
        apiVersion: ark.mckinsey.com/v1alpha1
        kind: Query
        name: test-query
        timeout: 4m
        for:
          condition:
            name: Completed
            value: 'True'
    catch:
    - events: {}
    - describe:
        apiVersion: ark.mckinsey.com/v1alpha1
        kind: Query
        name: test-query

    cleanup:
    - script:
        content: |
          helm uninstall ark-tenant --namespace $NAMESPACE --wait --timeout=180s || true
        env:
        - name: NAMESPACE
          value: ($namespace)

Error Handling and Verbosity

Standard Catch Blocks

All chainsaw tests with Query assertions should include catch blocks to reduce verbosity and provide debugging information:

catch:
- events: {}
- describe:
    apiVersion: ark.mckinsey.com/v1alpha1
    kind: Query
    name: query-name

Catch Block Purpose

  • events: {} - Suppresses detailed event logging noise during normal operation
  • describe: - Provides structured debugging information for Query resources when failures occur

Event Validation Best Practices

When validating events in tests, check for presence rather than exact counts to avoid flakiness:

# Good - robust presence checking
if [ "$target_execution_complete" -gt 0 ]; then
  echo "✓ TargetExecutionComplete events found"
fi

# Bad - exact count matching (flaky)
if [ "$target_execution_complete" -eq 1 ]; then
  echo "✓ Exactly 1 TargetExecutionComplete event"
fi

Test Structure for Timing

Separate query completion waiting from validation steps to ensure proper timing:

- name: wait-for-query-completion
  try:
  - wait:
      apiVersion: ark.mckinsey.com/v1alpha1
      kind: Query
      name: test-query
      timeout: 4m
      for:
        condition:
          name: Completed
          value: 'True'

- name: validate-response
  try:
  - assert:
      resource:
        apiVersion: ark.mckinsey.com/v1alpha1
        kind: Query
        metadata:
          name: test-query
        status:
          (response != null): true

Test Execution

Local Testing

# Run all tests
chainsaw test tests/

# Run specific test
chainsaw test tests/queries/

# Debug mode with cleanup disabled 
chainsaw test tests/ --test-dir tests/queries --pause-on-failure

Validation

  • Each test should pass independently when run individually

Playwright UI Testing

Radix UI Select

Two things make Radix UI Select options unstable for Playwright:

  1. React re-render race: Filling a form field and immediately clicking a Select trigger can race with React Hook Form's blur/validation re-render, causing the trigger to be briefly detached or the select to open and immediately close.
  2. Open animation: data-state="open" fires at the start of the entry animation (zoom-in, slide-in), not the end. Playwright sees the bounding box still changing and reports "element is not stable". The animation must fully complete before options are clickable.

Best practices for reliable Select interaction:

  1. Scope the trigger selector to the dialog to avoid matching other comboboxes on the page.
  2. Blur the form input before clicking the trigger, so React re-renders from validation happen before the click.
  3. Retry the click if the listbox doesn't appear (handles transient close).
  4. Do NOT require [data-side] in the listbox selector — Radix Popper sets it asynchronously and it may not be present immediately in headless CI.
name_input.fill(tool_name)
name_input.blur()  # Trigger form validation re-render before clicking select

type_trigger = page.locator("[role='dialog'] [role='combobox']").first
type_trigger.scroll_into_view_if_needed()
type_trigger.wait_for(state="visible", timeout=15000)

listbox = page.locator("[role='listbox'][data-state='open']")
for attempt in range(3):
    type_trigger.click()
    try:
        listbox.wait_for(state="visible", timeout=5000)
        break
    except Exception:
        pass  # retry

self.wait_for_animations_complete(listbox)
page.locator("[role='option']:has-text('HTTP')").first.click()

wait_for_animations_complete uses the Web Animations API to block until all running animations on the element and its subtree finish:

handle = locator.element_handle(timeout=timeout)
if handle:
    page.evaluate("el => Promise.allSettled(el.getAnimations({subtree: true}).map(a => a.finished))", handle)

Use page.evaluate(fn, handle) rather than locator.evaluate(fn) — the latter can cause Playwright to refocus the element, which closes Radix dropdowns.

{subtree: true} is required — without it, getAnimations() only checks the listbox container, not the option elements that are actually animating.

If options are still detaching after this, the likely cause is a parent component re-rendering while the dropdown is open (e.g. a form.watch() call in React Hook Form re-rendering on blur/validation). Fix it in the component by replacing form.watch(name) with useWatch({ control, name }), which only re-renders when the field value changes.

  • Query tests should reach phase: done
  • No RBAC permission errors in events

PostgreSQL Broker Tests

Tests labeled postgresql: "true" run in the storage-backend: postgresql CI matrix, where the broker uses Postgres for messages, events, and sessions (backends.message=postgres, backends.event=postgres, backends.sessions=postgres). This means broker messages, events, and sessions are persisted in the messages, events, sessions, and session_queries tables of the ark-storage-dev Postgres instance in ark-system.

Use this label when a test needs to verify broker message/event/session persistence, expires_at, or other Postgres-specific behavior.

Querying Postgres from a chainsaw test

Use the shared psql-query.sh script to run SQL against ark-storage-dev:

- name: verify-postgres
  try:
  - script:
      timeout: 30s
      content: |
        TTL=$(bash ../shared/psql-query.sh \
          "SELECT EXTRACT(EPOCH FROM (expires_at - created_at))::int FROM messages WHERE query_id='my-query' LIMIT 1;" \
          | tr -d ' \n')
        echo "ttl_seconds: ${TTL}"
        [ "${TTL}" = "3600" ] || { echo "expected 3600, got ${TTL}"; exit 1; }
      env:
      - name: NAMESPACE
        value: ($namespace)

The script reads the password from the ark-storage-dev-password secret in ark-system and execs psql on the ark-storage-dev deployment.

Polling for async writes: use psql-poll.sh, not a shell retry loop

Broker writes land asynchronously, so verify steps must poll until the rows appear. Do not loop over psql-query.sh: each call is three kubectl round-trips (namespace lookup, secret decode, exec), so an N-iteration loop pays that N times and, under CI Postgres contention, exceeds the step timeout and is killed. signal: killed in a chainsaw script means the step timeout was hit, not that an assertion failed.

Use psql-poll.sh, which resolves the connection once and runs the whole retry loop inside a single kubectl exec against local psql. It polls <sql> until the trimmed result equals <expected>, then prints it; non-zero exit if it never matched:

- script:
    timeout: 60s
    content: |
      COUNT=$(bash ../shared/psql-poll.sh \
        "SELECT count(*) FROM session_queries WHERE session_id='${SESSION_ID}' AND phase='done';" \
        "2" 80 0.5) || { echo "expected 2, got '${COUNT}'"; exit 1; }

Arguments are "<sql>" "<expected>" [attempts] [sleep_seconds] (defaults 80 and 0.5, a 40s budget). Use psql-query.sh only for single reads once the poll has confirmed the data is present.

Derive per-run identifiers from ($namespace)

Hardcoded sessionIds (and similar identifiers) accumulate rows across runs on a persistent Postgres. CI is fresh each run, but local re-runs and any shared DB are not, so a fixed id breaks exact-count assertions on the second run. Set the id from ($namespace) (unique per run) in the query manifests and reference the same value in verify steps, then delete the run's own rows in a cleanup step.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.