agentleFS
tradingstrategy-aitradingstrategy-ai/web3-ethereum-defi/CLAUDE.md

web3-ethereum-defi

CLAUDE.md830 starsChanged 10 days agoMITView on GitHub

Repo-local reference docs that Claude should consult when the task touches the relevant area: - .claude/docs/agent-tricks-and-troubleshooting.md — Claude, Codex, and Grok CLI usage patterns, cross-agent review commands, streaming output, timeouts, and common failure modes. See Agents below for when this file is mandatory. - .claude/docs/vault-integration.md — read before changing ERC-4626-style vault adapters, their guard deployment or executor integration; defines existing ownership, descriptor handling and checks against unnecessary layers. - docs/README-lighter-core-execution.md — read before changing Lighter pool execution or multichain activation…

  • Reads credentials

The file

Show its rules and text

We split this CLAUDE.md into the rules it gives an agent

88 rules. About 0 are common, 39 are distinctive, and 49 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈39

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 49

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  • could be automatic: a setting could enforce it
  1. Very specific:

    Before invoking any external agent CLI (claude, claude -p, claude ultrareview, codex, codex exec, grok, or equivalent), read …

    line 21
  2. Very specific:

    When a task matches one of the folder names, open the corresponding SKILL.md first and follow it before doing ad hoc exploration.

    line 32
  3. Distinctive:

    If the user mentions a skill by name, use it.

    line 36
  4. Distinctive:

    If the task clearly matches one of the skill directory names above, use that skill even if the user did not mention it explicitly.

    line 37
  5. Very specific:

    Start by reading only .claude/skills/<skill-name>/SKILL.md.

    line 38
  6. Distinctive:

    If the skill references extra files, open only the files needed for the current task.

    line 39
  7. Distinctive:

    Prefer scripts, templates and checklists referenced by the skill over re-creating the workflow manually.

    line 40
  8. Distinctive:

    After completing a feature, bug fix or maintained content change, use clean-up-for-pr before the review handoff.

    line 42
  9. Distinctive:

    Use UK/British English instead of US English

    line 50
  10. Very specific:

    Always spell onchain without a hyphen.

    line 52
  11. Distinctive:

    Read and follow the description writing style guide when writing or editing manually maintained public descriptions and notes shown on the …

    line 57
  12. Very specific:

    When running a Python script use poetry run python command instead of plain python command, so that the virtual environment is activated.

    line 75
  13. Distinctive:

    Do not edit or rerun a successful initial migration for a later accessor change.

    line 89
  14. Distinctive:

    When a newly discovered, repaired or reclassified vault lacks early price history, add a fixed-scope price backfill to the numbered …

    line 90
  15. Very specific:

    All three services mount ${HOME}/.tradingstrategy from the production host at /root/.tradingstrategy in the container.

    line 117
  16. Very specific:

    Do not describe JSON_RPC_MONAD as an archive node, retry failed old eth_call requests, or attempt a genesis-to-head state backfill.

    line 144
  17. Very specific:

    Use Hypersync for historical Monad events and timestamps.

    line 147
  18. Very specific:

    Check the block type before pricing from latest: it can be a large block in a different fee market.

    line 163
  19. Very specific:

    Do not rely on eth_gasPrice, as its meaning depends on the provider.

    line 166
  20. Very specific:

    Use the block gasLimit to tell the two apart: 30M for large blocks.

    line 170
  21. Distinctive:

    Always prefix pytest command with relevant source command, otherwise the test cannot find environment variables.

    line 187
  22. Distinctive:

    Avoid running the whole test suite as it takes several minutes.

    line 190
  23. Distinctive:

    Every new or materially changed external integration must include at least one minimal real integration test that exercises the actual …

    line 194
  24. Distinctive:

    Prefer an automatic CI integration test whenever credentials can be supplied securely and the provider is reliable enough for CI.

    line 200
  25. Very specific:

    When writing a new Anvil mainnet-fork test, you must use the shared Anvil fork + fixed fork-block + warm RPC-cache pattern rather than …

    line 219
  26. Very specific:

    When running pytest or any test commands, always use an extended timeout by specifying timeout: 180000 (3 minutes) in the bash tool …

    line 231
  27. Very specific:

    If you need extra output pass --log-cli-level=info argument to pytest.

    line 234
  28. Very specific:

    If there is a space in RPC URL given by a environment variable like JSON_RPC_ETHEREUM, it can be only used with Python call …

    line 242
  29. Very specific:

    If you are going to use this RPC URL with other commands, like curl, you need to parse the RPC environment variable by spltting it by …

    line 243
  30. Distinctive:

    Run Ruff through Poetry on the Python files changed for the task.

    line 250
  31. Very specific:

    If unsure where the main checkout is, use git worktree list and copy from the non-.omnara/worktrees checkout that already has …

    line 267
  32. Very specific:

    When starting work in a new worktree, check that .venv and .claude folders are symlinked into the worktree so that skills and .claude/docs …

    line 270
  33. Distinctive:

    No test plan or verification section.

    line 282
  34. Distinctive:

    Only push changes to remote when asked, never update pull requess automatically.

    line 289
  35. Distinctive:

    Never push directly to a master if not told explicitly

    line 290
  36. Very specific:

    If the user ask to open a pull request as feature then start the PR title with "feat:" prefix and also add one line about the feature into …

    line 291
  37. Distinctive:

    Before opening or updating a pull request, format the code

    line 293
  38. Distinctive:

    If a pull request adds or changes user-facing website or data text, such as vault, protocol, curator, stablecoin or fee descriptions and …

    line 294
  39. Distinctive:

    When merging pull request, squash and merge commits and use the PR description as the commit message

    line 302
  40. Very specific:

    When watching CI for pull request merge readiness, never wait for documentation-only workflows like Build documentation.

    line 303
  41. Distinctive:

    If continuous integration (CI) tests fail on your PR, and they are marked flaky, run tests locally to repeat the issue if it is real …

    line 304
  42. Distinctive:

    Use this escalation process for tests that fail nondeterministically because of RPC providers, Anvil forks, live APIs, indexed data …

    line 308
  43. Very specific:

    Identify and reproduce the issue: Read the complete failed-job traceback, record the exact failing test and symptom, and check whether the …

    line 313
  44. Very specific:

    Mark a genuinely nondeterministic test flaky: Add @flaky.flaky and a source line comment immediately above it.

    line 319
  45. Distinctive:

    Disable an endemic test on CI only as a last resort: If a flaky test continues to fail on at least two distinct CI runs, still passes …

    line 327
  46. Distinctive:

    If you push directly to master, the commit message most follow *Commentary format* section

    line 341
  47. Distinctive:

    Use threaded instead of async Python code

    line 350
  48. Distinctive:

    Always type hint function arguments and return values

    line 351
  49. Very specific:

    Use any() and all() with generators and list comprehension when checking if a collection member has one or more matches, instead of using …

    line 353
  50. Very specific:

    All functions that do network reads to get data should be prefixed with fetch_ instead of get_

    line 354
  51. Very specific:

    Always try to return Iterator instead of list from a function call to make functions faster

    line 355
  52. Distinctive:

    Use module level imports, not function level lazy imports, whenever possible

    line 358
  53. Very specific:

    Never write generic Exception e: catch but always catch a specific exception if we can

    line 359
  54. Distinctive:

    Never silently swallow exceptions and th

    line 360
  55. Very specific:

    Do not add PRIMARY KEY or UNIQUE constraints to large, file-backed bulk ingestion tables.

    line 370
  56. Distinctive:

    Test the migration and duplicate-ingestion path with a file-backed database containing production-scale row counts.

    line 379
  57. Distinctive:

    If a. class function overloads a function inherited from the parent, and there is nothing to comment, do not repeat the code comment and …

    line 389
  58. Very specific:

    Use HexAddress instead of str for blockchain addresses

    line 394
  59. Very specific:

    When logging using logger.info(), logging.debug() or similar, prefer %s and %f unexpanded string syntax instead of Python string …

    line 400
  60. Very specific:

    All API modules should have stub entry under docs/source/api and cross-referenced in docs/source/api/index table of contents

    line 405
  61. Distinctive:

    When writing documentation, in sentences, include inline links to the source pages.

    line 407
  62. Distinctive:

    Use naive UTC datetimes everywhere

    line 411
  63. Very specific:

    When using datetime class use import datetime.datetime and use datetime.datetime and datetime.timedelta as type hints

    line 412
  64. Very specific:

    Instead of datetime.datetime.utcnow() use native_datetime_utc_now() that is compatible across Python versions

    line 413
  65. Distinctive:

    Never use test classes in pytest

    line 421
  66. Very specific:

    Instead of manual float fuzzy comparison like assert abs(aave_total_pnl - 96.6087) < 0.01 use pytest.approx()

    line 423
  67. Distinctive:

    Always use fixture and test functions, never use test classes

    line 425
  68. Very specific:

    When adding or updating dependencies in pyproject.toml, always add a comment why this dependency is needed for this project

    line 433
  69. Very specific:

    Use scripts/erc-4626/scab-vaults.py as an example how to set up logger and read any needed environment variables as input.

    line 444
  70. Distinctive:

    When configuration is needed, use environment variables.

    line 445
  71. Distinctive:

    Use threading backend unless explicitly specified otherwise

    line 451
  72. Very specific:

    All functions using joblib.Parallel must take max_workers argument.

    line 453
  73. Distinctive:

    Prefer a blockchain explorer like Etherscan over Python and Curl when trying to read proxy contract address

    line 457
  74. Very specific:

    Prefer Python snippets instead of curl when trying to read data directly from a blockchain explorer

    line 458
  75. Very specific:

    Never try to figure out RPC URL yourself - always use environment variables from the local environment given by the user.

    line 460
  76. Very specific:

    All contract ABI JSON files live under eth_defi/abi/ (shared contracts at the top level, protocol-specific interfaces under …

    line 464
  77. Very specific:

    Create clients through eth_defi.hypersync.utils.configure_hypersync_from_env() and open streams through …

    line 477
  78. Distinctive:

    Do not define inline ABIs in Python unless the fragment contains at most one or two functions.

    line 511
  79. Very specific:

    Never directly edit auto-generated sphinx files in _autosummary* folders.

    line 522
  80. Very specific:

    Never silently discard existing data: If a schema migration (migrate_parquet_schema(), cast()) fails, the pipeline must abort with a hard …

    line 528
  81. Very specific:

    Never catch ArrowInvalid and reset to empty: If the existing parquet cannot be read or migrated, raise the exception so the operator can …

    line 529
  82. Very specific:

    New columns must have null defaults: Add them via migrate_parquet_schema() with pa.nulls().

    line 530
  83. Very specific:

    Type changes need explicit migration: If changing a column's type (e.g.

    line 531
  84. Very specific:

    Don't do hardcoded token decimal multiply, use TokenDetails.convert_to_raw()

    line 538
  85. Very specific:

    Use TokenDetails.transfer() and similar - do not do raw ERC-20 contract calls unless needed

    line 539
  86. Very specific:

    Use eth_defi.hotwallet.HotWallet for deployer accounts and signing transactions when possible

    line 540
  87. Very specific:

    When fetching web pages, if web_fetch returns a 403 error, retry the request using the Chrome MCP tool to load the page in a real browser …

    line 544
  88. Distinctive:

    When using browser tools, Claude may ask for permission to visit specific domains.

    line 555

≈ “Common” is an estimate: at least 1 in 5 of the 27,641 CLAUDE.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
CLAUDE.md

Instructions to work with the code base

Reference docs for Claude

Repo-local reference docs that Claude should consult when the task touches the relevant area:

  • .claude/docs/agent-tricks-and-troubleshooting.md — Claude, Codex, and Grok CLI usage patterns, cross-agent review commands, streaming output, timeouts, and common failure modes. See Agents below for when this file is mandatory.
  • .claude/docs/vault-integration.md — read before changing ERC-4626-style vault adapters, their guard deployment or executor integration; defines existing ownership, descriptor handling and checks against unnecessary layers.
  • docs/README-lighter-core-execution.md — read before changing Lighter pool execution or multichain activation scripts; covers script defaults, custody boundaries, recovery and the remaining funded verification requirements.

Agents

Before invoking any external agent CLI (claude, claude -p, claude ultrareview, codex, codex exec, grok, or equivalent), read .claude/docs/agent-tricks-and-troubleshooting.md in the current session and follow its instructions. Do not run any of those CLIs until you have checked that guidance. Detailed review patterns and failure modes live only in that file — do not restate them here.

Skills

Repo-local skills live as folders under .claude/skills.

When a task matches one of the folder names, open the corresponding SKILL.md first and follow it before doing ad hoc exploration.

Skill discovery rules:

  • If the user mentions a skill by name, use it.
  • If the task clearly matches one of the skill directory names above, use that skill even if the user did not mention it explicitly.
  • Start by reading only .claude/skills/<skill-name>/SKILL.md.
  • If the skill references extra files, open only the files needed for the current task.
  • Prefer scripts, templates and checklists referenced by the skill over re-creating the workflow manually.

After completing a feature, bug fix or maintained content change, use clean-up-for-pr before the review handoff. It covers code simplification, factual copy, documentation, repository conformance and an independent review with the opposite agent. Run it once for the completed change; delegated reviewers must not start another cleanup review.

English

  • Use UK/British English instead of US English
  • Say things like visualise instead of visualize
  • Always spell onchain without a hyphen; correct on-chain to onchain
  • For headings, only capitalise the first letter of heading, do not use title case

Writing style for user-facing website and data text

Read and follow the description writing style guide when writing or editing manually maintained public descriptions and notes shown on the website or in published data, including vault, protocol, curator, stablecoin and fee descriptions in YAML, Python or Markdown files. Repository engineering documentation, such as READMEs, Sphinx pages, .claude docs, docstrings, code comments, CHANGELOG.md entries and pull request descriptions, follows only the English rules above.

Installing dependencies

Install dependencies with all required extras:

poetry install -E data -E test -E docs -E hypersync -E ccxt -E cloudflare_r2 -E duckdb

Running Python scripts

When running a Python script use poetry run python command instead of plain python command, so that the virtual environment is activated.

poetry run python scripts/logos/post-process-logo.py

Production

For database repairs and persisted accessor refreshes, follow numbered migration scripts. Add a new numbered script and a PR run reminder. Standalone migrate-*.py, fix-*.py and one-off backfill entrypoints are legacy format and must not be used for these tasks. Port a required legacy repair into a new numbered script; existing commands in old runbooks are historical references, not templates. Do not edit or rerun a successful initial migration for a later accessor change. When a newly discovered, repaired or reclassified vault lacks early price history, add a fixed-scope price backfill to the numbered migration; see historical price backfills. Both preview and apply require the looped scanner to be stopped; use the direct Compose command in the production runbook, preserving the lifetime-guard-before-writer-lock order. The host launcher is optional for Docker preflight and automatic image pinning. Migration provenance comes from the git commit stamped inside the image; no host Docker hash is required.

Production vault scanning runs from ~/vault-scanner/web3-ethereum-defi with environment variables loaded from ~/vault-scanner/vault-rpc.env. The Compose configuration is docker-compose.yml; inspect it before running production maintenance because its service entrypoint and mounted state determine the effect of a command.

  • vault-scanner-oneshot is a profile-only, single-run service. It inherits the image entrypoint from Dockerfile.vault-scanner, which runs scripts/erc-4626/scan-vaults-all-chains.py once. Use it for an intentional scan or override its entrypoint for production maintenance, e.g. source ~/vault-scanner/vault-rpc.env && (cd ~/vault-scanner/web3-ethereum-defi && docker compose run --entrypoint /bin/bash vault-scanner-oneshot).
  • vault-scanner-looped is the persistent scanner started by docker compose up -d. It explicitly runs python scripts/erc-4626/scan-vaults-all-chains.py and uses LOOP_INTERVAL_SECONDS, SCAN_CYCLES and DEFAULT_CYCLE to schedule work.
  • post-scanner is the persistent feed collector. Its entrypoint is python scripts/erc-4626/scan-vault-posts.py.

All three services mount ${HOME}/.tradingstrategy from the production host at /root/.tradingstrategy in the container. This is persistent production state, including the vault metadata pickle, price Parquet files, reader state and dense per-chain timestamp caches under /root/.tradingstrategy/block-timestamp/{chain_id}-timestamps.duckdb. One-shot containers therefore share the same state as the looped scanner; do not change HOME, use an unmounted container, or delete/recreate this directory while performing a migration. The Compose services also mount host .cache, repository logs, and the mutable stablecoin/feed data directories where applicable.

For metadata-only repairs, explicitly disable historical price scanning (for example UPSHIFT_SCAN_PRICES=false). Price scans require the dense timestamp cache to be available inside the container; restore or prepopulate that cache on the host before a historical backfill rather than allowing a one-shot repair to bootstrap it with sparse timestamp requests.

Chain quirks

Monad

Monad (chain ID 143) does not provide archive-complete historical state. Its full nodes retain all historical blocks, transactions, receipts, events and traces, but expose only a provider- and disk-dependent recent window of contract state. No provider offers arbitrary-depth historical state; see the Monad historical data documentation.

  • Do not describe JSON_RPC_MONAD as an archive node, retry failed old eth_call requests, or attempt a genesis-to-head state backfill. Once a state trie has been evicted, those operations cannot succeed.
  • Use Hypersync for historical Monad events and timestamps. For historical vault values, scan_historical_prices_to_parquet() probes Multicall and begins at the oldest state block the configured provider can execute. Backfill a Monad price gap soon after finding it, while the state is still readable; see the Monad backfill rules.
  • Preserve existing Monad price rows before that dynamically detected boundary: the current provider cannot reconstruct them. Monad fork and historical tests must use current-state or state-relative assertions.

HyperEVM

HyperEVM (chain IDs 999 and 998) interleaves small and large blocks with independent mempools and base fees, see the dual-block architecture documentation. The sender's usingBigBlocks flag selects the mempool, not the gas limit.

  • Check the block type before pricing from latest: it can be a large block in a different fee market. Use estimate_gas_price(), which takes the block type as block_type. web3.py's default fee filler does not make this check.
  • Do not rely on eth_gasPrice, as its meaning depends on the provider. See the measurements in estimate_gas_price().
  • Anvil forks keep chain ID 999 but have one fee market, and their blocks can have a 30M gas limit, so do not apply these rules to a fork.
  • Use the block gasLimit to tell the two apart: 30M for large blocks. On 5 November 2026 small blocks had a 3M limit.

Running tests

If we have not run tests before make sure the user has created a gitignored file .local-test.env in the repository root. This will use source shell command to include the actual test secrets which lie outside the repository structure. Note: this file does not contain actual environment variables, just a source command to get them from elsewhere. Never edit this file.

If .local-test.env is missing in a git worktree when pytest needs to be run, do not give up. First follow the git worktree instructions below and copy .local-test.env from the main repository checkout into the current worktree root. Only ask the user to prepare .local-test.env if it is also missing from the main repository checkout or cannot be found.

To run tests you need to use the installed Poetry environment, with given environment secrets file.

To run tests use the pytest wrapper command:

source .local-test.env && poetry run pytest {test case name or pattern here}

Always prefix pytest command with relevant source command, otherwise the test cannot find environment variables.

Avoid running the whole test suite as it takes several minutes. Only run specific test cases.

External integration tests

Every new or materially changed external integration must include at least one minimal real integration test that exercises the actual provider end-to-end. For authenticated APIs, this means using a supplied API token and verifying a real successful response; mocked requests and fixture-only tests are necessary unit coverage but do not replace this check.

  • Prefer an automatic CI integration test whenever credentials can be supplied securely and the provider is reliable enough for CI.
  • Otherwise, provide a focused manual command, guarded by the required environment variable or token, so an operator can run the check without modifying source code. Never print, commit or include a token in the command or its output.
  • The pull request that adds the integration must include a PR comment with the real-test result: the exact test or command, whether it passed, the date, and the redacted environment or provider used. State clearly when the result is a manual run rather than CI. If the provider rate-limits a bulk migration, record the response and the safe retry plan instead of repeatedly retrying the same request burst.

Hypersync block scan tests (tests/erc_4626/test_scan.py) are disabled on CI by default because they dominated the CI critical path; they still run locally. To run them on CI (e.g. when touching Hypersync discovery code), see docs/README-hypersync-tests.md. Apply the same skip_hypersync_scan_on_ci marker pattern to any new multi-minute Hypersync scan test.

When writing a new Anvil mainnet-fork test, you must use the shared Anvil fork + fixed fork-block + warm RPC-cache pattern rather than launching a per-file fork_network_anvil at latest or an ad-hoc block. The canonical, authoritative description of this pattern — why it matters (warm CI RPC cache) and exactly how to apply it (shared anvil_fork_pool fixture, chain *_MIDNIGHT_BLOCK constant, xdist_group marker, read-only vs mutating caveat, copy-paste skeleton) — is the module docstring of eth_defi/testing/anvil_fork_pool.py. Read it first. Use evm_snapshot_revert for per-test isolation and deploy expensive Safe/vault fixtures once per session. Reference tests: tests/erc_4626/vault_protocol/test_goat.py (read-only pooled) and tests/lagoon/conftest.py (shared fork + once-per-session deployment).

When running pytest or any test commands, always use an extended timeout by specifying timeout: 180000 (3 minutes) in the bash tool parameters.

If you need extra output pass --log-cli-level=info argument to pytest.

Environment variable configuration and RPC URL format

We use environment variables like JSON_RPC_ETHEREUM, JSON_RPC_ARBITRUM` to interact with various EVM-based blockchains.

In the environment file, the RPC URLs are provided in the project-specific space-separated fallback format, as described in mev-blocker.rst.

  • If there is a space in RPC URL given by a environment variable like JSON_RPC_ETHEREUM, it can be only used with Python call create_multi_provider_web3()
  • If you are going to use this RPC URL with other commands, like curl, you need to parse the RPC environment variable by spltting it by spaces and taking the first entry
  • Scanner RPC variables should point to archive-capable EVM nodes, except JSON_RPC_MONAD, which can only serve its provider-dependent recent historical-state window.

Formatting code

Run Ruff through Poetry on the Python files changed for the task. Pass the reviewed paths explicitly, including new files, so unrelated work is preserved:

poetry run ruff format path/to/changed.py path/to/new.py

Logging retries

For a failure that is expected to be retried, log a concise WARNING without a traceback. Include enough context to identify the deferred operation. Log an ERROR with its traceback only after the final retry has failed and no further retry will be attempted.

Git worktrees

  • For git worktrees, copy .local-test.env from the main repository checkout root into the current worktree root.
  • If unsure where the main checkout is, use git worktree list and copy from the non-.omnara/worktrees checkout that already has .local-test.env.
  • Example: cp /path/to/main/repo/.local-test.env .local-test.env
  • For worktrees, unless you are changing package dependencies, use poetry run from the parent repo virtualenv
  • When starting work in a new worktree, check that .venv and .claude folders are symlinked into the worktree so that skills and .claude/docs can be found, and the poetry command runs with already installed packages from the parent.

Commentary format

Pull request description must have sections:

  • Why: the rational of change
  • Lessons learnt: memory
  • Summary: what was changed
  • Related issues and PRs: if the new PR continues other PRs and issues
  • Unrelated CI and test fixes: if we clean up misc CI integration tests

No test plan or verification section. Use Markdown formatting, headings.

Pull requests

  • GitHub plugins are not needed for pull-request work. Prefer the local git and gh command-line tools to create, inspect, update, cancel CI for, and merge pull requests.
  • Only push changes to remote when asked, never update pull requess automatically.
  • Never push directly to a master if not told explicitly
  • If the user ask to open a pull request as feature then start the PR title with "feat:" prefix and also add one line about the feature into CHANGELOG.md
  • Each changelog entry should follow the date of the PR in YYYY-MM-DD format. Example: Something was updated (2026-01-01).
  • Before opening or updating a pull request, format the code
  • If a pull request adds or changes user-facing website or data text, such as vault, protocol, curator, stablecoin or fee descriptions and vault notes, check the affected text against the description writing style guide before opening or updating it, unless that version of the text was already checked earlier. Reuse the earlier review for unchanged text and check any subsequent substantive edits. Record completion or the earlier review in the PR checklist; mark it not applicable when there are no such changes. Engineering documentation is outside the guide's scope.
  • When merging pull request, squash and merge commits and use the PR description as the commit message
  • When watching CI for pull request merge readiness, never wait for documentation-only workflows like Build documentation; merge once non-documentation required checks are green, unless the user explicitly asks to wait for docs.
  • If continuous integration (CI) tests fail on your PR, and they are marked flaky, run tests locally to repeat the issue if it is real flakiness or regression

Healing flaky CI tests

Use this escalation process for tests that fail nondeterministically because of RPC providers, Anvil forks, live APIs, indexed data services, timing or shared CI resources. A deterministic product or test regression must be fixed and must never be hidden behind retries or a CI skip.

  1. Identify and reproduce the issue. Read the complete failed-job traceback, record the exact failing test and symptom, and check whether the affected code differs from master. Run the focused test locally with .local-test.env. Attempt a bounded root-cause fix first, such as pinning a fork block, correcting an assertion, reducing cross-test contention or installing a missing workflow dependency.
  2. Mark a genuinely nondeterministic test flaky. Add @flaky.flaky and a source line comment immediately above it. The comment must state when the problem was first observed, the concrete CI symptom, and the evidence that the test passes locally, on retry or in a later CI run. Keep the test enabled in CI; a flaky marker is a temporary diagnostic and retry mechanism, not a substitute for fixing a reproducible defect. Move resource-heavy live or fork tests to the slow workflow when reduced contention is the appropriate fix, and ensure that workflow installs all required dependencies.
  3. Disable an endemic test on CI only as a last resort. If a flaky test continues to fail on at least two distinct CI runs, still passes locally, and the external or runner-specific cause cannot be fixed in this repository, add a dated @pytest.mark.skipif(CI, reason=...). The line comment and skip reason must describe the recurring symptom. Keep the test runnable locally; never add an unconditional skip. Retain or create focused local coverage so the disabled integration does not silently disappear.

Periodically audit CI-disabled tests. Re-enable one when the provider, workflow or fixture has been healed, run it locally, and require a green CI run before removing its flaky history comment.

Pushing to master

  • If you push directly to master, the commit message most follow Commentary format section

Specific rules

Python rules

  • We use Python 3.14
  • Using assert for invariant and configuration checks is acceptable. We never run CPython with optimisation enabled (python -O or PYTHONOPTIMIZE), so assertions are not stripped at runtime.
  • For data structures, prefer dataclass(slots=True)
  • Use threaded instead of async Python code
  • Always type hint function arguments and return values
  • Try to use Python and Pandas apply() and other functional helpers instead of slow for and while loops
  • Use any() and all() with generators and list comprehension when checking if a collection member has one or more matches, instead of using slow for loops
  • All functions that do network reads to get data should be prefixed with fetch_ instead of get_
  • Always try to return Iterator instead of list from a function call to make functions faster
  • Observability required: All long-running actions must be observable in the run, through logging or terminal output. For long-running loops, scanners, scripts, use the tqdm and tqdm_loggable.auto module for a progress bar. Make sure no script runs without producing output for more than 1 minute. As an example, see lead_scan_core.py. If agent runs long-running scripts or processes, it must be done in a way the agent can tell whether the process is making work or hung.
  • For visualusations, use Plotly. For chart titles, use heading case as explained above.
  • Use module level imports, not function level lazy imports, whenever possible
  • Never write generic Exception e: catch but always catch a specific exception if we can
  • Never silently swallow exceptions and th

DuckDB and Python 3.14

DuckDB 1.5.0 file-backed databases have caused native heap corruption in this repository under Python 3.14 when large tables use ART-backed PRIMARY KEY or UNIQUE constraints. The process can abort with allocator errors such as free(): chunks in smallbin corrupted, so Python cannot catch or recover from the failure. See DuckDB issue #18190.

  • Do not add PRIMARY KEY or UNIQUE constraints to large, file-backed bulk ingestion tables. Use an unconstrained staging table and hash joins for conflict detection and application-level deduplication.
  • Migrate existing ART-backed tables transactionally, preserving every row, before continuing a backfill. Inspect duckdb_constraints() to detect the old schema.
  • For bulk ingestion, set wal_autocheckpoint = '1TB' to avoid automatic checkpoints during writes. This is an additional safeguard, not a substitute for removing ART indexes.
  • Test the migration and duplicate-ingestion path with a file-backed database containing production-scale row counts. A small in-memory test does not reproduce this class of crash.
  • Rerunning an unchanged backfill is not a fix: it can reach the same native corruption threshold and abort again.

Code comments

  • For code comments, Use Sphinx restructured text style
  • For documenting dataclass and Enum members, use Sphinx #: comment here line comment above variable, not :param: or :ivar: blocks
  • If a. class function overloads a function inherited from the parent, and there is nothing to comment, do not repeat the code comment and leave it empty instead
  • Each function should minimum have 1) one liner summary description 2) one paragraph longer description 3) arguments and return values documented 4) for APIs and integrations, links to authoritative and canonical documentation 5) for dataframes and series expectations of columns and value types

Type hinting

  • Use HexAddress instead of str for blockchain addresses
  • For percent like numbers, do not use raw float, but use eth_defi.types.Percent type alias

Logging

  • For logging, use the module level logger = logging.getLogger(__name__) pattern
  • When logging using logger.info(), logging.debug() or similar, prefer %s and %f unexpanded string syntax instead of Python string interpolation, because of performance reasons

Documentation

  • All API modules should have stub entry under docs/source/api and cross-referenced in docs/source/api/index table of contents
  • See docs/source/api/index.rst and docs/source/api/lagoon/index.rst as examples
  • When writing documentation, in sentences, include inline links to the source pages. Link each page only once, preferably earler in the text.

datetime

  • Use naive UTC datetimes everywhere
  • When using datetime class use import datetime.datetime and use datetime.datetime and datetime.timedelta as type hints
  • Instead of datetime.datetime.utcnow() use native_datetime_utc_now() that is compatible across Python versions

Enum

  • For string enums, both members and values must in snake_case

Pytest

  • Never use test classes in pytest
  • pytest tests should not have stdout output like print
  • Instead of manual float fuzzy comparison like assert abs(aave_total_pnl - 96.6087) < 0.01 use pytest.approx()
  • For DuckDB testing, make sure the database is always closed using finally clause or fixtures
  • Always use fixture and test functions, never use test classes
  • For Anvil mainnet fork based tests, whici use a fixed block number, in asserts check for absolute number values instead of relative values like above zero, because values never change. Exception: Monad retains only a moving recent historical-state window, so use latest-head or state-relative assertions instead of fixed historical blocks.
  • For reuseable testing code, use testing modules under eth_defi - do not nyt try to import "tests" as it does not work with pytest

pyproject.toml

  • When adding or updating dependencies in pyproject.toml, always add a comment why this dependency is needed for this project

Python notebooks

  • Whenever possible, prefer table output instead of print(). Use Pandas DataFrame and notebook's built-in display() function to render tabular data.

Command line scripts

  • Scripts must use sane defaults and run without additional environment inputs wherever possible. Require an environment variable only when its value cannot be safely defaulted or inferred, such as private keys, API credentials or the user's RPC connections. Do not invent credentials or RPC URLs.
  • Routine settings such as amounts, fee allowances, chain selections, timeouts, output paths and logging levels must have sensible defaults appropriate to the script's purpose. Keep environment overrides optional; do not make users supply routine settings merely to acknowledge a default.
  • Document the default behaviour and clearly distinguish genuinely required inputs from optional overrides. Example commands should use the defaults after the private environment has been loaded, rather than require a list of routine environment assignments. Validate defaults and overrides before sending transactions or doing other irreversible work.
  • Use scripts/erc-4626/scab-vaults.py as an example how to set up logger and read any needed environment variables as input.
  • When configuration is needed, use environment variables. Do not attempt to create command line parsers unless explicitly asked.
  • For tabular output, do not use print() loops but use tabulate.tabulate() function, see whitelist-vaults.py as an example

Parallerisation and optimising long running data reading pipelines

  • Uses joblib.Parallel to parallerise API reading of multiple entries
  • Use threading backend unless explicitly specified otherwise
  • For example, see lead_scan_core.py
  • All functions using joblib.Parallel must take max_workers argument. This must be exposed to command line scripts as MAX_WORKERS environment variable, see scripts/erc-4626/scan-vaults.py as an example.

Working with RPC and blockchain explorers

  • Prefer a blockchain explorer like Etherscan over Python and Curl when trying to read proxy contract address
  • Prefer Python snippets instead of curl when trying to read data directly from a blockchain explorer
  • To get the latest block number, use given JSON-RPC URL and Python's Web3.py web3.eth.block_number call
  • Never try to figure out RPC URL yourself - always use environment variables from the local environment given by the user. See eth_defi.chain.CHAIN_NAMES for aliases like chain id 999 -> JSON_RPC_HYPERLIQUD. Stop and ask user if you cannot figure out.

Contract ABIs

  • All contract ABI JSON files live under eth_defi/abi/ (shared contracts at the top level, protocol-specific interfaces under eth_defi/abi/<protocol>/) and are loaded through the shared helpers in eth_defi.abi.
  • Before fetching a new ABI, or loading/using any ABI file, read eth_defi/abi/README.md first. It explains what EVM ABIs are, how to fetch verified ABIs from Sourcify, Etherscan and GitHub (including proxy → implementation resolution), where to store them, and the API references for loading and using ABI files in their different roles (contract binding, deployment, ABI-less encoded calls, event decoding).

Event logs

  • Never use JSON-RPC eth_getLogs for event discovery, bulk event reads or historical event reads. Always use Hypersync, which avoids provider range limits and provides indexed event streaming.
  • Create clients through eth_defi.hypersync.utils.configure_hypersync_from_env() and open streams through eth_defi.hypersync.session.open_hypersync_stream(). These wrappers apply the repository's configured endpoint, rate limiting and stream tuning.
  • For vault lead discovery, use eth_defi.erc_4626.hypersync_discovery.HypersyncVaultDiscover. For a targeted event query, build a hypersync.Query with hypersync.LogSelection and consume it through open_hypersync_stream(); see eth_defi.vault.flow_events.fetch_vault_flow_logs_hypersync_async() for the canonical pattern.

Block timestamps

  • For all bulk or historical blockchain timestamp operations, use the cache-aware Hypersync API: fetch_block_timestamps_using_hypersync_cached() or the fetch_block_timestamps_multiprocess_auto_backend() wrapper with a Hypersync client. Do not stream timestamps through get_block_timestamps_using_hypersync*() directly unless explicitly repairing known cache gaps.
  • Preserve and reuse the per-chain DuckDB cache at ~/.tradingstrategy/block-timestamp/{chain_id}-timestamps.duckdb. Do not use the legacy ~/.tradingstrategy/block-timestamps.* location.

For JSON-RPC URL configuration, environment variables. The variables are in the format JSON_RPC_{CHAIN} where {CHAIN} is the uppercase chain name:

  • JSON_RPC_ETHEREUM - Ethereum mainnet
  • JSON_RPC_ARBITRUM - Arbitrum One
  • JSON_RPC_BASE - Base
  • JSON_RPC_POLYGON - Polygon
  • JSON_RPC_BINANCE - BNB Smart Chain a.k.a. Binance a.k.a. BNB chain
  • JSON_RPC_HYPERLIQUID - HyperEVM

You chan find these in CHAIN_NAMES and in eth_defi.provider.env

Building integrated smart contracts

You can use Makefile commands make guard safe-integration to rebuild smart contracts for Satfe and Lagoon integration.

ABIs

  • Read eth_defi/abi/README.md first — it is the canonical reference for fetching, storing, loading and using ABI files.
  • Store contract ABIs in eth_defi/abi/<protocol>/ as JSON files and load them through the shared ABI helpers.
  • Do not define inline ABIs in Python unless the fragment contains at most one or two functions.
  • Regenerate ABI JSON for this repository's integrated smart contracts with the compiler. For external deployments, commit the verified or application-exported interface JSON and record its canonical source alongside it.

Documentation

Documentation uses Sphinx v4.5 for API and narrative documentation and lives in docs folder.

Ignore Sphinx documentation builds during development. Agents must not attempt to build the Sphinx documentation locally; documentation builds and their failures are outside the development verification workflow.

Never directly edit auto-generated sphinx files in _autosummary* folders.

Parquet schema migrations

The vault price pipeline accumulates months of historical data in vault-prices-1h.parquet. Losing this data requires days of re-scanning from archive nodes and may make older Monad data unrecoverable, because Monad does not provide archive-complete historical state.

  • Never silently discard existing data. If a schema migration (migrate_parquet_schema(), cast()) fails, the pipeline must abort with a hard error — never fall back to existing_table = None. Silent data loss is worse than a crash.
  • Never catch ArrowInvalid and reset to empty. If the existing parquet cannot be read or migrated, raise the exception so the operator can restore from a backup.
  • New columns must have null defaults. Add them via migrate_parquet_schema() with pa.nulls(). Never require a value in existing rows.
  • Type changes need explicit migration. If changing a column's type (e.g. uint32 → uint64), verify cast() works on production data before merging — test with a copy of the real parquet, not just synthetic test data.
  • Always test schema changes against the production parquet. Download the current file and verify the migration path locally before deploying.
  • Reader state loss causes full data wipe. The scanner deletes existing chain rows from start_block onwards. If reader-state.pickle is lost, start_block falls back to the earliest vault block, deleting all historical data for that chain. Treat reader state files as critical production state.
  • Preserve old Monad rows. A Monad rescan can replace rows only at or after the dynamically probed provider boundary. Never reset its reader state or delete older rows in an attempt to rebuild unavailable state history.

ERC-20

  • Don't do hardcoded token decimal multiply, use TokenDetails.convert_to_raw()
  • Use TokenDetails.transfer() and similar - do not do raw ERC-20 contract calls unless needed
  • Use eth_defi.hotwallet.HotWallet for deployer accounts and signing transactions when possible

Web Fetching and 403

When fetching web pages, if web_fetch returns a 403 error, retry the request using the Chrome MCP tool to load the page in a real browser instead.

Prerequisites:

  1. Claude in Chrome extension (v1.0.36+) - Chrome Web Store
  2. Google Chrome running
  3. Direct Anthropic plan (Pro, Max, Team, or Enterprise)

Browser tools are automatically available when the Chrome extension is connected. Use @browser in your Visual Studio Code prompt to activate the connection.

When using browser tools, Claude may ask for permission to visit specific domains. Approve these prompts to allow browser automation. You can also pre-approve domains in the Chrome extension settings.

README files in the repository

Consult these for domain-specific context. Logo READMEs under eth_defi/data/vaults/original_logos/*/README.md document logo source URLs per protocol.

PathDescription
README.mdWeb3-Ethereum-Defi project overview
contracts/guard/README.mdGuardV0 — on-chain trade validation for asset management
contracts/in-house/README.mdWeb3-Eth-Defi integration contracts
contracts/safe-integration/README.mdTrading Strategy Zodiac-module for Safe multisig wallets
docs/README-Hypercore-guard.mdHypercore native vault guard integration
docs/README-hyperevm-goldsky-failure.mdHyperEVM goldsky eRPC "not enough agreement" consensus failure and Alchemy failover
docs/README-hyperevm-hypercore-read-gas.mdHyperEVM HyperCore read gas — why HyperCore-reading vaults abort Multicall3 batches with -32003 out of gas and revert with CoreReaderLib.ReadFailure, and why they are not blacklisted
docs/README-hypersync-tests.mdHypersync scan tests — disabled on CI by default, how to run them on CI when needed
docs/README-test-suite-performance.mdTest suite performance plan — CI caching, shared Anvil forks, vault-protocol gating
docs/README-vault-scan-manifest.mdPrivate readiness receipt: R2 keys, schema, sparse candle semantics, cache policy and rollout
docs/README-hypercore-permission-recovery.mdHypercore permission clocks, backed-up recovery and gated private R2 maintenance
docs/README-contract-size.mdContract sizes and compiler optimisation
docs/derive-onboarding/README-derive-trader.mdDerive session key for vault traders
docs/protocol-research/README.mdAI-assisted vault protocol discovery notes
docs/source/api/derive/README.mdDerive.xyz integration — implementation summary
eth_defi/aave_v3/README.mdAbout Aave v3 integration
eth_defi/abi/README.mdContract ABIs — what EVM ABIs are, how to fetch them from Sourcify/Etherscan/GitHub, storage layout, and the loader/usage API reference. Read before fetching or using any ABI file
eth_defi/abi/ipor/README.mdIPOR ABI source links
eth_defi/abi/lagoon/README.mdLagoon ABI source links
eth_defi/abi/yield_basis/README.mdYieldBasis ABI sources and reviewed deployment interfaces
eth_defi/abi/uniswap-swap-contracts/README.mdSwapRouter02 deployment on Base
eth_defi/cctp/README-cctp.mdCircle CCTP V2 integration
eth_defi/core3/README-core3.mdCore3 risk intelligence integration — modules, database schema, scripts, API reference
eth_defi/xerberus/README-xerberus.mdXerberus vault risk scores — per-vault pool ratings, DuckDB, export, scripts
eth_defi/currency_api/README-currency-api.mdHistorical exchange rate ingestion (fawazahmed0 Exchange API) into DuckDB
eth_defi/data/vaults/README.mdVault protocol metadata and logo system
eth_defi/enzyme/README-Enzyme-vaults.mdEnzyme Blue and Onyx architecture, metadata and migration operations
eth_defi/tokenised_fund/asseto/README-Asseto.mdAsseto dynamic registry, daily NAV history and operations
eth_defi/erc_4626/vault_protocol/README-reader-states.mdVault reader states and warmup system
eth_defi/erc_4626/vault_protocol/README-utilisation.mdUtilisation and available liquidity metrics for lending vaults
eth_defi/erc_4626/vault_protocol/README-vault-redeemable.mdWhy utilisation ≠ redeemable liquidity for Morpho/IPOR multi-market vaults
eth_defi/gmx/README-GMX-Lagoon.mdGMX Lagoon integration security analysis
eth_defi/gmx/README-GMX-Vaults.mdGMX GM and GLV liquidity catalogue
eth_defi/gmx/README.mdGMX CCXT adapter for eth_defi
eth_defi/gmx/ccxt/README.mdGMX CCXT adapter implementation
eth_defi/gmx/graphql/README.mdGMX Subsquid GraphQL integration
eth_defi/vault_report/README-vault-report.mdMonthly best-performing stablecoin vaults blog report — data sources, sections, branded charts, Ghost draft workflow
eth_defi/vault_report/README-blog-post-outline.mdMonthly vault report blog post outline — section order, vault groups, selection rules, editor input and open decisions
eth_defi/vault_report/README-best-vaults-news.mdMonthly vault report blog post drafts — goal of the skeleton, GitHub PR comment previews, Ghost draft creation, architecture diagram
eth_defi/yield_basis/README-YieldBasis.mdYieldBasis LT vault model, valuation and historical pipeline
eth_defi/erc_4626/vault_protocol/rysk/README-Rysk-vaults.mdRysk Premium epoch-priced option pools, adapter accounting and backfill operations
eth_defi/erc_4626/vault_protocol/antarctic/README-Antarctic.mdAntarctic perpetual DEX, AMLP/AHLP settlement data and event-based vault support
eth_defi/lighter/README-lighter-guard.mdLighter (zk-rollup perps DEX) L1 deposit/withdraw guard integration — architecture, security model, operator flow
eth_defi/testing/README.mdFast Anvil fork tests — shared session forks, per-chain midnight block cache, snapshot/revert, once-per-session deployments, the committed fork RPC cache and token cache (create/rebuild/purge), Anvil 1.8 behaviour and mitigations (cache source_id, async mining, hedged receipt lookups), diagnosing slow fork tests, reference tests
eth_defi/testing/rpc_cache_seed/README.mdCommitted Anvil fork RPC cache seed — layout, how to capture and refresh it
scripts/base/README.mdBase chain related manual test scripts
scripts/debian-bullseye-compatibility/README.mdRunning on Debian Bullseye
scripts/erc-4626/README-vault-scripts.mdERC-4626 vault scripts
scripts/grvt/README-grvt-vaults.mdGRVT native vault metrics pipeline
scripts/hyperliquid/README-hyperliquid-copy-trading.mdHyperliquid copy trading platforms and HFT account identification
scripts/hyperliquid/README-hyperliquid-vaults.mdHyperliquid native vault metrics pipeline
scripts/hyperliquid/README-hyperliquid-vaults-high-frequency.mdHigh-frequency Hyperliquid vault data fetcher
scripts/lighter/README-lighter-vaults.mdLighter native pool metrics pipeline
tests/erc_4626/vault_protocol/README.mdVault protocol detection tests (mainnet-fork)
tests/gmx/README.mdTesting for GMX
tests/guard/README.mdIntegration tests for GuardV0 and TradingStrategyModuleV0
tests/provider/README.mdService provider integration tests
tests/rpc/README.mdJSON-RPC scenario tests

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.