consume-hive

---
name: consume-hive
description: >-
  Run locally filled fixtures against execution clients with a selected Hive
  simulator and a network client configuration from hive-tests. Use when
  testing clients against a devnet or client releases in Hive, or when
  building a Hive client YAML.
---

# Consume Hive

Usage:

`/consume-hive <fixtures> <network|client.yaml|releases> <engine|enginex|rlp|sync>`

Require all three inputs; ask for missing ones. Run only the selected
simulator, with no simulator default. `execute hive` is intentionally left
for a separate skill.

## Resolve inputs

- **Fixtures:** resolve the local fixture root, preserving its `.meta` data.
  Check that it contains fixtures for the selected simulator: `engine` uses
  `blockchain_tests_engine`, `enginex` uses `blockchain_tests_engine_x`,
  `rlp` uses `blockchain_tests`, and `sync` uses `blockchain_tests_sync`
  (generated by tests marked `verify_sync`). Report missing formats or an
  empty selection; do not substitute released fixtures. If filling is needed,
  use `/fill-tests`.
- **Network/client YAML:** fetch a snapshot from the default branch of
  [ethpandaops/hive-tests](https://github.com/ethpandaops/hive-tests),
  pinned to its current commit; no clone is needed. The upstream files are
  usable as-is:

  ```bash
  REPO=repos/ethpandaops/hive-tests
  REV=$(gh api "$REPO/commits/master" --jq .sha)
  CONFIGS="$REPO/contents/.github/configs/hive"
  gh api "$CONFIGS?ref=$REV" --jq '.[].name'
  gh api "$CONFIGS/$NAME?ref=$REV" -H 'Accept: application/vnd.github.raw' \
    > "$RUN_DIR/client.yaml"
  ```

  For `mainnet` or `generic`, inspect `.github/workflows/generic.yaml` at
  `$REV` and use its default `client_file` (currently `master.yaml`, not
  `generic.yaml`). Otherwise, resolve the specified network name or YAML path
  against the listing above. If absent or ambiguous, show the available
  choices and ask; do not silently substitute another devnet. Record `$REV`
  with the saved YAML. Preserve its client list, image tags, and build
  arguments.
- **Client releases:** hive-tests has no file for client release images. When
  the user asks for releases or a public network (for example Sepolia or
  mainnet releases), [create a client YAML](#create-a-client-yaml) instead.

## Create a client YAML

For the file format, see
[Client configuration](../../../docs/running_tests/hive/client_config.md).
Unless the user names other clients, use the clients in hive-tests'
`master.yaml`. Write the file to `$RUN_DIR/client.yaml` with one entry per
client, `nametag: release`, and `build_args` set to the release image:

```yaml
- client: go-ethereum
  nametag: release
  build_args:
    baseimage: ethereum/client-go
    tag: <release-tag>
```

| Client | Release repository | Image | Tag format |
| --- | --- | --- | --- |
| `besu` | `besu-eth/besu` | `hyperledger/besu` | release tag |
| `erigon` | `erigontech/erigon` | `erigontech/erigon` | release tag |
| `ethrex` | `lambdaclass/ethrex` | `ghcr.io/lambdaclass/ethrex` | no `v` prefix |
| `go-ethereum` | `ethereum/go-ethereum` | `ethereum/client-go` | release tag |
| `nethermind` | `NethermindEth/nethermind` | `nethermind/nethermind` | release tag |
| `nimbus-el` | `status-im/nimbus-eth1` | `statusim/nimbus-eth1` | release tag |
| `reth` | `paradigmxyz/reth` | `ghcr.io/paradigmxyz/reth` | release tag |

Resolve each tag with
`gh api repos/<release-repository>/releases/latest --jq .tag_name`, then
check the image exists with
`docker buildx imagetools inspect <image>:<tag>` before starting Hive. If a
check fails, stop and report it; do not fall back to another tag.

- Always set `build_args`. Hive's default Dockerfiles mostly build
  development branches (for example Besu `develop`, Nethermind `master`), so
  an entry without them does not test a release.
- Never use `latest` or another moving tag.
- Report the file as generated locally, not from hive-tests, list each
  client's release tag, and offer to upstream it to hive-tests.

## Start Hive

Use a built [ethereum/hive](https://github.com/ethereum/hive) checkout and a
fresh run directory for logs and reports. For environment and platform setup,
see [Hive dev mode](../../../docs/running_tests/hive/dev_mode.md).

From the Hive root, start a dedicated server on an unused localhost port:

```bash
./hive --dev --dev.addr "127.0.0.1:$HIVE_PORT" \
  --client-file "$CLIENT_YAML" \
  --docker.pull --docker.nocache '^hive/clients/' --docker.buildoutput \
  --results-root "$RUN_DIR/hive-results"
```

Always enable `--docker.pull` to refresh base images; `--docker.nocache` also
rebuilds client wrappers. The rebuild runs for every client on each start, so
expect a slow start with a large YAML. A failed pull is not permission to use
stale images.
Do not add a client filter unless requested. Record the server PID, wait for
its API, and verify `/clients` contains the YAML's clients. Capture build logs
and image IDs/digests: mutable tags alone do not identify what was tested.

## Consume fixtures

Run from execution-specs with `HIVE_SIMULATOR` pointing to this server. Choose
worker count for the host (four is a reasonable start). Set these options:

| Simulator | Additional consume options |
| --- | --- |
| `engine` | `--disable-strict-exception-matching=nimbus-el` |
| `enginex` | `--disable-strict-exception-matching=nimbus-el` |
| `rlp` | None |
| `sync` | None |

The Nimbus exception matches the Engine/EngineX Dockerfile defaults tracked in
[issue #3603](https://github.com/ethereum/execution-specs/issues/3603). Native
commands do not inherit those defaults. Keep invalid-payload rejection checks
enabled and other clients' exception matching strict; do not skip negative
tests. RLP import does not verify Engine API rejection messages.

`sync` parametrizes both source and syncing clients from the YAML, running
client pairs. Account for the extra containers when choosing workers and
report results per pair.

Put the selected options above in a Bash array `SIMULATOR_ARGS`, then run
the commands below with Bash. If the shell is not Bash (for example fish),
wrap them in `bash -c`:

```bash
export HIVE_SIMULATOR="http://127.0.0.1:$HIVE_PORT"
uv run consume "$SIMULATOR" --input "$FIXTURES" \
  "${SIMULATOR_ARGS[@]}" -n "$WORKERS" --timing-data \
  --html "$RUN_DIR/report.html" --junitxml "$RUN_DIR/report.xml"
```

Preserve command output, exit status, and Hive logs. Report per-client
pass/fail/skip/error counts, fixture scope, commands, repository revisions,
client versions/images, and report paths. Distinguish infrastructure failures
from test failures; an empty run is not a pass. Diagnose failures without
weakening fixture expectations. Stop only the Hive process started for this
run and retain its evidence.

More agent context in ethereum/execution-specs

14 other files this repository gives its agents.