web3-ethereum-defi
CLAUDE.md830 starsChanged 10 days agoMITView on GitHub
Repo-local reference docs that Claude should consult when the task touches the relevant area: - .claude/docs/agent-tricks-and-troubleshooting.md — Claude, Codex, and Grok CLI usage patterns, cross-agent review commands, streaming output, timeouts, and common failure modes. See Agents below for when this file is mandatory. - .claude/docs/vault-integration.md — read before changing ERC-4626-style vault adapters, their guard deployment or executor integration; defines existing ownership, descriptor handling and checks against unnecessary layers. - docs/README-lighter-core-execution.md — read before changing Lighter pool execution or multichain activation…
- Reads credentials
In this repository23 other files it gives its agents.
- add-curatorSkill
- add-stablecoinSkill
- add-vault-abi-methodsSkill
- add-vault-noteSkill
- add-vault-protocolSkill
- categorise-vault-strategySkill
- check-stablecoinsSkill
- check-top-list-vaultsSkill
- check-vault-onchainSkill
- clean-up-for-prSkill
- document-curator-incidentSkill
- extract-project-logoSkill
- extract-test-setSkill
- extract-vault-protocol-logoSkill
- find-new-curatorsSkill
- fix-sphinx-docsSkill
- get-block-numberSkill
- Identify vault protocolSkill
- maintain-curatorsSkill
- make-releaseSkill
- monthly-vault-reportSkill
- more-vaultsSkill
- post-process-logoSkill
The file
Show its rules and text
We split this CLAUDE.md into the rules it gives an agent
88 rules. About 0 are common, 39 are distinctive, and 49 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈39
Distinctive · estimate
This author's own choices. Worth a look.
49
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- could be automatic: a setting could enforce it
- Very specific: line 21
Before invoking any external agent CLI (claude, claude -p, claude ultrareview, codex, codex exec, grok, or equivalent), read …
- Very specific: line 32
When a task matches one of the folder names, open the corresponding SKILL.md first and follow it before doing ad hoc exploration.
- Distinctive: line 36
If the user mentions a skill by name, use it.
- Distinctive: line 37
If the task clearly matches one of the skill directory names above, use that skill even if the user did not mention it explicitly.
- Very specific: line 38
Start by reading only .claude/skills/<skill-name>/SKILL.md.
- Distinctive: line 39
If the skill references extra files, open only the files needed for the current task.
- Distinctive: line 40
Prefer scripts, templates and checklists referenced by the skill over re-creating the workflow manually.
- Distinctive: line 42
After completing a feature, bug fix or maintained content change, use clean-up-for-pr before the review handoff.
- Distinctive: line 50
Use UK/British English instead of US English
- Very specific: line 52
Always spell onchain without a hyphen.
- Distinctive: line 57
Read and follow the description writing style guide when writing or editing manually maintained public descriptions and notes shown on the …
- Very specific: line 75
When running a Python script use poetry run python command instead of plain python command, so that the virtual environment is activated.
- Distinctive: line 89
Do not edit or rerun a successful initial migration for a later accessor change.
- Distinctive: line 90
When a newly discovered, repaired or reclassified vault lacks early price history, add a fixed-scope price backfill to the numbered …
- Very specific: line 117
All three services mount ${HOME}/.tradingstrategy from the production host at /root/.tradingstrategy in the container.
- Very specific: line 144
Do not describe JSON_RPC_MONAD as an archive node, retry failed old eth_call requests, or attempt a genesis-to-head state backfill.
- Very specific: line 147
Use Hypersync for historical Monad events and timestamps.
- Very specific: line 163
Check the block type before pricing from latest: it can be a large block in a different fee market.
- Very specific: line 166
Do not rely on eth_gasPrice, as its meaning depends on the provider.
- Very specific: line 170
Use the block gasLimit to tell the two apart: 30M for large blocks.
- Distinctive: line 187
Always prefix pytest command with relevant source command, otherwise the test cannot find environment variables.
- Distinctive: line 190
Avoid running the whole test suite as it takes several minutes.
- Distinctive: line 194
Every new or materially changed external integration must include at least one minimal real integration test that exercises the actual …
- Distinctive: line 200
Prefer an automatic CI integration test whenever credentials can be supplied securely and the provider is reliable enough for CI.
- Very specific: line 219
When writing a new Anvil mainnet-fork test, you must use the shared Anvil fork + fixed fork-block + warm RPC-cache pattern rather than …
- Very specific: line 231
When running pytest or any test commands, always use an extended timeout by specifying timeout: 180000 (3 minutes) in the bash tool …
- Very specific: line 234
If you need extra output pass --log-cli-level=info argument to pytest.
- Very specific: line 242
If there is a space in RPC URL given by a environment variable like JSON_RPC_ETHEREUM, it can be only used with Python call …
- Very specific: line 243
If you are going to use this RPC URL with other commands, like curl, you need to parse the RPC environment variable by spltting it by …
- Distinctive: line 250
Run Ruff through Poetry on the Python files changed for the task.
- Very specific: line 267
If unsure where the main checkout is, use git worktree list and copy from the non-.omnara/worktrees checkout that already has …
- Very specific: line 270
When starting work in a new worktree, check that .venv and .claude folders are symlinked into the worktree so that skills and .claude/docs …
- Distinctive: line 282
No test plan or verification section.
- Distinctive: line 289
Only push changes to remote when asked, never update pull requess automatically.
- Distinctive: line 290
Never push directly to a master if not told explicitly
- Very specific: line 291
If the user ask to open a pull request as feature then start the PR title with "feat:" prefix and also add one line about the feature into …
- Distinctive: line 293
Before opening or updating a pull request, format the code
- Distinctive: line 294
If a pull request adds or changes user-facing website or data text, such as vault, protocol, curator, stablecoin or fee descriptions and …
- Distinctive: line 302
When merging pull request, squash and merge commits and use the PR description as the commit message
- Very specific: line 303
When watching CI for pull request merge readiness, never wait for documentation-only workflows like Build documentation.
- Distinctive: line 304
If continuous integration (CI) tests fail on your PR, and they are marked flaky, run tests locally to repeat the issue if it is real …
- Distinctive: line 308
Use this escalation process for tests that fail nondeterministically because of RPC providers, Anvil forks, live APIs, indexed data …
- Very specific: line 313
Identify and reproduce the issue: Read the complete failed-job traceback, record the exact failing test and symptom, and check whether the …
- Very specific: line 319
Mark a genuinely nondeterministic test flaky: Add @flaky.flaky and a source line comment immediately above it.
- Distinctive: line 327
Disable an endemic test on CI only as a last resort: If a flaky test continues to fail on at least two distinct CI runs, still passes …
- Distinctive: line 341
If you push directly to master, the commit message most follow *Commentary format* section
- Distinctive: line 350
Use threaded instead of async Python code
- Distinctive: line 351
Always type hint function arguments and return values
- Very specific: line 353
Use any() and all() with generators and list comprehension when checking if a collection member has one or more matches, instead of using …
- Very specific: line 354
All functions that do network reads to get data should be prefixed with fetch_ instead of get_
- Very specific: line 355
Always try to return Iterator instead of list from a function call to make functions faster
- Distinctive: line 358
Use module level imports, not function level lazy imports, whenever possible
- Very specific: line 359
Never write generic Exception e: catch but always catch a specific exception if we can
- Distinctive: line 360
Never silently swallow exceptions and th
- Very specific: line 370
Do not add PRIMARY KEY or UNIQUE constraints to large, file-backed bulk ingestion tables.
- Distinctive: line 379
Test the migration and duplicate-ingestion path with a file-backed database containing production-scale row counts.
- Distinctive: line 389
If a. class function overloads a function inherited from the parent, and there is nothing to comment, do not repeat the code comment and …
- Very specific: line 394
Use HexAddress instead of str for blockchain addresses
- Very specific: line 400
When logging using logger.info(), logging.debug() or similar, prefer %s and %f unexpanded string syntax instead of Python string …
- Very specific: line 405
All API modules should have stub entry under docs/source/api and cross-referenced in docs/source/api/index table of contents
- Distinctive: line 407
When writing documentation, in sentences, include inline links to the source pages.
- Distinctive: line 411
Use naive UTC datetimes everywhere
- Very specific: line 412
When using datetime class use import datetime.datetime and use datetime.datetime and datetime.timedelta as type hints
- Very specific: line 413
Instead of datetime.datetime.utcnow() use native_datetime_utc_now() that is compatible across Python versions
- Distinctive: line 421
Never use test classes in pytest
- Very specific: line 423
Instead of manual float fuzzy comparison like assert abs(aave_total_pnl - 96.6087) < 0.01 use pytest.approx()
- Distinctive: line 425
Always use fixture and test functions, never use test classes
- Very specific: line 433
When adding or updating dependencies in pyproject.toml, always add a comment why this dependency is needed for this project
- Very specific: line 444
Use scripts/erc-4626/scab-vaults.py as an example how to set up logger and read any needed environment variables as input.
- Distinctive: line 445
When configuration is needed, use environment variables.
- Distinctive: line 451
Use threading backend unless explicitly specified otherwise
- Very specific: line 453
All functions using joblib.Parallel must take max_workers argument.
- Distinctive: line 457
Prefer a blockchain explorer like Etherscan over Python and Curl when trying to read proxy contract address
- Very specific: line 458
Prefer Python snippets instead of curl when trying to read data directly from a blockchain explorer
- Very specific: line 460
Never try to figure out RPC URL yourself - always use environment variables from the local environment given by the user.
- Very specific: line 464
All contract ABI JSON files live under eth_defi/abi/ (shared contracts at the top level, protocol-specific interfaces under …
- Very specific: line 477
Create clients through eth_defi.hypersync.utils.configure_hypersync_from_env() and open streams through …
- Distinctive: line 511
Do not define inline ABIs in Python unless the fragment contains at most one or two functions.
- Very specific: line 522
Never directly edit auto-generated sphinx files in _autosummary* folders.
- Very specific: line 528
Never silently discard existing data: If a schema migration (migrate_parquet_schema(), cast()) fails, the pipeline must abort with a hard …
- Very specific: line 529
Never catch
ArrowInvalidand reset to empty: If the existing parquet cannot be read or migrated, raise the exception so the operator can … - Very specific: line 530
New columns must have null defaults: Add them via migrate_parquet_schema() with pa.nulls().
- Very specific: line 531
Type changes need explicit migration: If changing a column's type (e.g.
- Very specific: line 538
Don't do hardcoded token decimal multiply, use TokenDetails.convert_to_raw()
- Very specific: line 539
Use TokenDetails.transfer() and similar - do not do raw ERC-20 contract calls unless needed
- Very specific: line 540
Use eth_defi.hotwallet.HotWallet for deployer accounts and signing transactions when possible
- Very specific: line 544
When fetching web pages, if web_fetch returns a 403 error, retry the request using the Chrome MCP tool to load the page in a real browser …
- Distinctive: line 555
When using browser tools, Claude may ask for permission to visit specific domains.
≈ “Common” is an estimate: at least 1 in 5 of the 27,641 CLAUDE.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Instructions to work with the code base
Reference docs for Claude
Repo-local reference docs that Claude should consult when the task touches the relevant area:
.claude/docs/agent-tricks-and-troubleshooting.md— Claude, Codex, and Grok CLI usage patterns, cross-agent review commands, streaming output, timeouts, and common failure modes. See Agents below for when this file is mandatory..claude/docs/vault-integration.md— read before changing ERC-4626-style vault adapters, their guard deployment or executor integration; defines existing ownership, descriptor handling and checks against unnecessary layers.docs/README-lighter-core-execution.md— read before changing Lighter pool execution or multichain activation scripts; covers script defaults, custody boundaries, recovery and the remaining funded verification requirements.
Agents
Before invoking any external agent CLI (claude, claude -p,
claude ultrareview, codex, codex exec, grok, or equivalent), read
.claude/docs/agent-tricks-and-troubleshooting.md in the current session
and follow its instructions. Do not run any of those CLIs until you have
checked that guidance. Detailed review patterns and failure modes live only
in that file — do not restate them here.
Skills
Repo-local skills live as folders under .claude/skills.
When a task matches one of the folder names, open the corresponding SKILL.md first and follow it before doing ad hoc exploration.
Skill discovery rules:
- If the user mentions a skill by name, use it.
- If the task clearly matches one of the skill directory names above, use that skill even if the user did not mention it explicitly.
- Start by reading only
.claude/skills/<skill-name>/SKILL.md. - If the skill references extra files, open only the files needed for the current task.
- Prefer scripts, templates and checklists referenced by the skill over re-creating the workflow manually.
After completing a feature, bug fix or maintained content change, use clean-up-for-pr before the review handoff. It covers code simplification, factual copy, documentation, repository conformance and an independent review with the opposite agent. Run it once for the completed change; delegated reviewers must not start another cleanup review.
English
- Use UK/British English instead of US English
- Say things like
visualiseinstead ofvisualize - Always spell
onchainwithout a hyphen; correcton-chaintoonchain - For headings, only capitalise the first letter of heading, do not use title case
Writing style for user-facing website and data text
Read and follow the description writing style guide
when writing or editing manually maintained public descriptions and notes
shown on the website or in published data, including vault, protocol, curator,
stablecoin and fee descriptions in YAML, Python or Markdown files. Repository
engineering documentation, such as READMEs, Sphinx pages, .claude docs,
docstrings, code comments, CHANGELOG.md entries and pull request
descriptions, follows only the English rules above.
Installing dependencies
Install dependencies with all required extras:
poetry install -E data -E test -E docs -E hypersync -E ccxt -E cloudflare_r2 -E duckdbRunning Python scripts
When running a Python script use poetry run python command instead of plain python command, so that the virtual environment is activated.
poetry run python scripts/logos/post-process-logo.pyProduction
For database repairs and persisted accessor refreshes, follow
numbered migration scripts. Add a new
numbered script and a PR run reminder. Standalone migrate-*.py, fix-*.py
and one-off backfill entrypoints are legacy format and must not be used
for these tasks. Port a required legacy repair into a new numbered script;
existing commands in old runbooks are historical references, not templates.
Do not edit or rerun a successful initial migration for a later accessor change.
When a newly discovered, repaired or reclassified vault lacks early price
history, add a fixed-scope price backfill to the numbered migration; see
historical price backfills.
Both preview and apply require the looped scanner to be stopped; use the direct
Compose command in the production runbook, preserving the
lifetime-guard-before-writer-lock order. The host launcher is optional for
Docker preflight and automatic image pinning. Migration provenance comes from
the git commit stamped inside the image; no host Docker hash is required.
Production vault scanning runs from ~/vault-scanner/web3-ethereum-defi with
environment variables loaded from ~/vault-scanner/vault-rpc.env. The Compose
configuration is docker-compose.yml; inspect it before running production
maintenance because its service entrypoint and mounted state determine the
effect of a command.
vault-scanner-oneshotis a profile-only, single-run service. It inherits the image entrypoint fromDockerfile.vault-scanner, which runsscripts/erc-4626/scan-vaults-all-chains.pyonce. Use it for an intentional scan or override its entrypoint for production maintenance, e.g.source ~/vault-scanner/vault-rpc.env && (cd ~/vault-scanner/web3-ethereum-defi && docker compose run --entrypoint /bin/bash vault-scanner-oneshot).vault-scanner-loopedis the persistent scanner started bydocker compose up -d. It explicitly runspython scripts/erc-4626/scan-vaults-all-chains.pyand usesLOOP_INTERVAL_SECONDS,SCAN_CYCLESandDEFAULT_CYCLEto schedule work.post-scanneris the persistent feed collector. Its entrypoint ispython scripts/erc-4626/scan-vault-posts.py.
All three services mount ${HOME}/.tradingstrategy from the production host
at /root/.tradingstrategy in the container. This is persistent production
state, including the vault metadata pickle, price Parquet files, reader state
and dense per-chain timestamp caches under
/root/.tradingstrategy/block-timestamp/{chain_id}-timestamps.duckdb.
One-shot containers therefore share the same state as the looped scanner; do
not change HOME, use an unmounted container, or delete/recreate this directory
while performing a migration. The Compose services also mount host .cache,
repository logs, and the mutable stablecoin/feed data directories where
applicable.
For metadata-only repairs, explicitly disable historical price scanning (for
example UPSHIFT_SCAN_PRICES=false). Price scans require the dense timestamp
cache to be available inside the container; restore or prepopulate that cache
on the host before a historical backfill rather than allowing a one-shot repair
to bootstrap it with sparse timestamp requests.
Chain quirks
Monad
Monad (chain ID 143) does not provide archive-complete historical state. Its full nodes retain all historical blocks, transactions, receipts, events and traces, but expose only a provider- and disk-dependent recent window of contract state. No provider offers arbitrary-depth historical state; see the Monad historical data documentation.
- Do not describe
JSON_RPC_MONADas an archive node, retry failed oldeth_callrequests, or attempt a genesis-to-head state backfill. Once a state trie has been evicted, those operations cannot succeed. - Use Hypersync for historical Monad events and timestamps. For historical
vault values,
scan_historical_prices_to_parquet()probes Multicall and begins at the oldest state block the configured provider can execute. Backfill a Monad price gap soon after finding it, while the state is still readable; see the Monad backfill rules. - Preserve existing Monad price rows before that dynamically detected boundary: the current provider cannot reconstruct them. Monad fork and historical tests must use current-state or state-relative assertions.
HyperEVM
HyperEVM (chain IDs 999 and 998) interleaves small and large blocks with
independent mempools and base fees, see the
dual-block architecture documentation.
The sender's usingBigBlocks flag selects the mempool, not the gas limit.
- Check the block type before pricing from
latest: it can be a large block in a different fee market. Useestimate_gas_price(), which takes the block type asblock_type. web3.py's default fee filler does not make this check. - Do not rely on
eth_gasPrice, as its meaning depends on the provider. See the measurements inestimate_gas_price(). - Anvil forks keep chain ID 999 but have one fee market, and their blocks can have a 30M gas limit, so do not apply these rules to a fork.
- Use the block
gasLimitto tell the two apart: 30M for large blocks. On 5 November 2026 small blocks had a 3M limit.
Running tests
If we have not run tests before make sure the user has created a gitignored file .local-test.env in the repository root. This will use source shell command to include the actual test secrets which lie outside the repository structure. Note: this file does not contain actual environment variables, just a source command to get them from elsewhere. Never edit this file.
If .local-test.env is missing in a git worktree when pytest needs to be run, do not give up. First follow the git worktree instructions below and copy .local-test.env from the main repository checkout into the current worktree root. Only ask the user to prepare .local-test.env if it is also missing from the main repository checkout or cannot be found.
To run tests you need to use the installed Poetry environment, with given environment secrets file.
To run tests use the pytest wrapper command:
source .local-test.env && poetry run pytest {test case name or pattern here}Always prefix pytest command with relevant source command, otherwise the test cannot find environment variables.
Avoid running the whole test suite as it takes several minutes. Only run specific test cases.
External integration tests
Every new or materially changed external integration must include at least one minimal real integration test that exercises the actual provider end-to-end. For authenticated APIs, this means using a supplied API token and verifying a real successful response; mocked requests and fixture-only tests are necessary unit coverage but do not replace this check.
- Prefer an automatic CI integration test whenever credentials can be supplied securely and the provider is reliable enough for CI.
- Otherwise, provide a focused manual command, guarded by the required environment variable or token, so an operator can run the check without modifying source code. Never print, commit or include a token in the command or its output.
- The pull request that adds the integration must include a PR comment with the real-test result: the exact test or command, whether it passed, the date, and the redacted environment or provider used. State clearly when the result is a manual run rather than CI. If the provider rate-limits a bulk migration, record the response and the safe retry plan instead of repeatedly retrying the same request burst.
Hypersync block scan tests (tests/erc_4626/test_scan.py) are disabled on CI by
default because they dominated the CI critical path; they still run locally. To
run them on CI (e.g. when touching Hypersync discovery code), see
docs/README-hypersync-tests.md. Apply the same skip_hypersync_scan_on_ci
marker pattern to any new multi-minute Hypersync scan test.
When writing a new Anvil mainnet-fork test, you must use the shared Anvil
fork + fixed fork-block + warm RPC-cache pattern rather than launching a
per-file fork_network_anvil at latest or an ad-hoc block. The canonical,
authoritative description of this pattern — why it matters (warm CI RPC cache)
and exactly how to apply it (shared anvil_fork_pool fixture, chain
*_MIDNIGHT_BLOCK constant, xdist_group marker, read-only vs mutating
caveat, copy-paste skeleton) — is the module docstring of
eth_defi/testing/anvil_fork_pool.py. Read it first. Use evm_snapshot_revert
for per-test isolation and deploy expensive Safe/vault fixtures once per session.
Reference tests: tests/erc_4626/vault_protocol/test_goat.py (read-only pooled)
and tests/lagoon/conftest.py (shared fork + once-per-session deployment).
When running pytest or any test commands, always use an extended timeout
by specifying timeout: 180000 (3 minutes) in the bash tool parameters.
If you need extra output pass --log-cli-level=info argument to pytest.
Environment variable configuration and RPC URL format
We use environment variables like JSON_RPC_ETHEREUM, JSON_RPC_ARBITRUM` to interact with various EVM-based blockchains.
In the environment file, the RPC URLs are provided in the project-specific space-separated fallback format, as described in mev-blocker.rst.
- If there is a space in RPC URL given by a environment variable like
JSON_RPC_ETHEREUM, it can be only used with Python callcreate_multi_provider_web3() - If you are going to use this RPC URL with other commands, like
curl, you need to parse the RPC environment variable by spltting it by spaces and taking the first entry - Scanner RPC variables should point to archive-capable EVM nodes, except
JSON_RPC_MONAD, which can only serve its provider-dependent recent historical-state window.
Formatting code
Run Ruff through Poetry on the Python files changed for the task. Pass the reviewed paths explicitly, including new files, so unrelated work is preserved:
poetry run ruff format path/to/changed.py path/to/new.pyLogging retries
For a failure that is expected to be retried, log a concise WARNING without
a traceback. Include enough context to identify the deferred operation. Log an
ERROR with its traceback only after the final retry has failed and no further
retry will be attempted.
Git worktrees
- For git worktrees, copy
.local-test.envfrom the main repository checkout root into the current worktree root. - If unsure where the main checkout is, use
git worktree listand copy from the non-.omnara/worktreescheckout that already has.local-test.env. - Example:
cp /path/to/main/repo/.local-test.env .local-test.env - For worktrees, unless you are changing package dependencies, use
poetry runfrom the parent repo virtualenv - When starting work in a new worktree, check that
.venvand.claudefolders are symlinked into the worktree so that skills and.claude/docscan be found, and the poetry command runs with already installed packages from the parent.
Commentary format
Pull request description must have sections:
- Why: the rational of change
- Lessons learnt: memory
- Summary: what was changed
- Related issues and PRs: if the new PR continues other PRs and issues
- Unrelated CI and test fixes: if we clean up misc CI integration tests
No test plan or verification section. Use Markdown formatting, headings.
Pull requests
- GitHub plugins are not needed for pull-request work. Prefer the local
gitandghcommand-line tools to create, inspect, update, cancel CI for, and merge pull requests. - Only push changes to remote when asked, never update pull requess automatically.
- Never push directly to a master if not told explicitly
- If the user ask to open a pull request as feature then start the PR title with "feat:" prefix and also add one line about the feature into
CHANGELOG.md - Each changelog entry should follow the date of the PR in YYYY-MM-DD format. Example: Something was updated (2026-01-01).
- Before opening or updating a pull request, format the code
- If a pull request adds or changes user-facing website or data text, such as vault, protocol, curator, stablecoin or fee descriptions and vault notes, check the affected text against the description writing style guide before opening or updating it, unless that version of the text was already checked earlier. Reuse the earlier review for unchanged text and check any subsequent substantive edits. Record completion or the earlier review in the PR checklist; mark it not applicable when there are no such changes. Engineering documentation is outside the guide's scope.
- When merging pull request, squash and merge commits and use the PR description as the commit message
- When watching CI for pull request merge readiness, never wait for documentation-only workflows like
Build documentation; merge once non-documentation required checks are green, unless the user explicitly asks to wait for docs. - If continuous integration (CI) tests fail on your PR, and they are marked flaky, run tests locally to repeat the issue if it is real flakiness or regression
Healing flaky CI tests
Use this escalation process for tests that fail nondeterministically because of RPC providers, Anvil forks, live APIs, indexed data services, timing or shared CI resources. A deterministic product or test regression must be fixed and must never be hidden behind retries or a CI skip.
- Identify and reproduce the issue. Read the complete failed-job traceback,
record the exact failing test and symptom, and check whether the affected
code differs from
master. Run the focused test locally with.local-test.env. Attempt a bounded root-cause fix first, such as pinning a fork block, correcting an assertion, reducing cross-test contention or installing a missing workflow dependency. - Mark a genuinely nondeterministic test flaky. Add
@flaky.flakyand a source line comment immediately above it. The comment must state when the problem was first observed, the concrete CI symptom, and the evidence that the test passes locally, on retry or in a later CI run. Keep the test enabled in CI; a flaky marker is a temporary diagnostic and retry mechanism, not a substitute for fixing a reproducible defect. Move resource-heavy live or fork tests to the slow workflow when reduced contention is the appropriate fix, and ensure that workflow installs all required dependencies. - Disable an endemic test on CI only as a last resort. If a flaky test
continues to fail on at least two distinct CI runs, still passes locally,
and the external or runner-specific cause cannot be fixed in this
repository, add a dated
@pytest.mark.skipif(CI, reason=...). The line comment and skip reason must describe the recurring symptom. Keep the test runnable locally; never add an unconditional skip. Retain or create focused local coverage so the disabled integration does not silently disappear.
Periodically audit CI-disabled tests. Re-enable one when the provider, workflow or fixture has been healed, run it locally, and require a green CI run before removing its flaky history comment.
Pushing to master
- If you push directly to master, the commit message most follow Commentary format section
Specific rules
Python rules
- We use Python 3.14
- Using
assertfor invariant and configuration checks is acceptable. We never run CPython with optimisation enabled (python -OorPYTHONOPTIMIZE), so assertions are not stripped at runtime. - For data structures, prefer
dataclass(slots=True) - Use threaded instead of async Python code
- Always type hint function arguments and return values
- Try to use Python and Pandas
apply()and other functional helpers instead of slow for and while loops - Use
any()andall()with generators and list comprehension when checking if a collection member has one or more matches, instead of using slow for loops - All functions that do network reads to get data should be prefixed with
fetch_instead ofget_ - Always try to return
Iteratorinstead oflistfrom a function call to make functions faster - Observability required: All long-running actions must be observable in the run, through logging or terminal output. For long-running loops, scanners, scripts, use the
tqdmandtqdm_loggable.automodule for a progress bar. Make sure no script runs without producing output for more than 1 minute. As an example, seelead_scan_core.py. If agent runs long-running scripts or processes, it must be done in a way the agent can tell whether the process is making work or hung. - For visualusations, use Plotly. For chart titles, use heading case as explained above.
- Use module level imports, not function level lazy imports, whenever possible
- Never write generic
Exception e:catch but always catch a specific exception if we can - Never silently swallow exceptions and th
DuckDB and Python 3.14
DuckDB 1.5.0 file-backed databases have caused native heap corruption in this
repository under Python 3.14 when large tables use ART-backed PRIMARY KEY or
UNIQUE constraints. The process can abort with allocator errors such as
free(): chunks in smallbin corrupted, so Python cannot catch or recover from
the failure. See DuckDB issue #18190.
- Do not add
PRIMARY KEYorUNIQUEconstraints to large, file-backed bulk ingestion tables. Use an unconstrained staging table and hash joins for conflict detection and application-level deduplication. - Migrate existing ART-backed tables transactionally, preserving every row,
before continuing a backfill. Inspect
duckdb_constraints()to detect the old schema. - For bulk ingestion, set
wal_autocheckpoint = '1TB'to avoid automatic checkpoints during writes. This is an additional safeguard, not a substitute for removing ART indexes. - Test the migration and duplicate-ingestion path with a file-backed database containing production-scale row counts. A small in-memory test does not reproduce this class of crash.
- Rerunning an unchanged backfill is not a fix: it can reach the same native corruption threshold and abort again.
Code comments
- For code comments, Use Sphinx restructured text style
- For documenting dataclass and Enum members, use Sphinx
#: comment hereline comment above variable, not:param:or:ivar:blocks - If a. class function overloads a function inherited from the parent, and there is nothing to comment, do not repeat the code comment and leave it empty instead
- Each function should minimum have 1) one liner summary description 2) one paragraph longer description 3) arguments and return values documented 4) for APIs and integrations, links to authoritative and canonical documentation 5) for dataframes and series expectations of columns and value types
Type hinting
- Use
HexAddressinstead ofstrfor blockchain addresses - For percent like numbers, do not use raw float, but use
eth_defi.types.Percenttype alias
Logging
- For logging, use the module level
logger = logging.getLogger(__name__)pattern - When logging using
logger.info(),logging.debug()or similar, prefer %s and %f unexpanded string syntax instead of Python string interpolation, because of performance reasons
Documentation
- All API modules should have stub entry under
docs/source/apiand cross-referenced indocs/source/api/indextable of contents - See
docs/source/api/index.rstanddocs/source/api/lagoon/index.rstas examples - When writing documentation, in sentences, include inline links to the source pages. Link each page only once, preferably earler in the text.
datetime
- Use naive UTC datetimes everywhere
- When using datetime class use
import datetime.datetimeand usedatetime.datetimeanddatetime.timedeltaas type hints - Instead of
datetime.datetime.utcnow()usenative_datetime_utc_now()that is compatible across Python versions
Enum
- For string enums, both members and values must in snake_case
Pytest
- Never use test classes in pytest
pytesttests should not have stdout output likeprint- Instead of manual float fuzzy comparison like
assert abs(aave_total_pnl - 96.6087) < 0.01usepytest.approx() - For DuckDB testing, make sure the database is always closed using finally clause or fixtures
- Always use fixture and test functions, never use test classes
- For Anvil mainnet fork based tests, whici use a fixed block number, in asserts check for absolute number values instead of relative values like above zero, because values never change. Exception: Monad retains only a moving recent historical-state window, so use latest-head or state-relative assertions instead of fixed historical blocks.
- For reuseable testing code, use
testingmodules undereth_defi- do not nyt try to import "tests" as it does not work with pytest
pyproject.toml
- When adding or updating dependencies in
pyproject.toml, always add a comment why this dependency is needed for this project
Python notebooks
- Whenever possible, prefer table output instead of print(). Use Pandas DataFrame and notebook's built-in display() function to render tabular data.
Command line scripts
- Scripts must use sane defaults and run without additional environment inputs wherever possible. Require an environment variable only when its value cannot be safely defaulted or inferred, such as private keys, API credentials or the user's RPC connections. Do not invent credentials or RPC URLs.
- Routine settings such as amounts, fee allowances, chain selections, timeouts, output paths and logging levels must have sensible defaults appropriate to the script's purpose. Keep environment overrides optional; do not make users supply routine settings merely to acknowledge a default.
- Document the default behaviour and clearly distinguish genuinely required inputs from optional overrides. Example commands should use the defaults after the private environment has been loaded, rather than require a list of routine environment assignments. Validate defaults and overrides before sending transactions or doing other irreversible work.
- Use scripts/erc-4626/scab-vaults.py as an example how to set up logger and read any needed environment variables as input.
- When configuration is needed, use environment variables. Do not attempt to create command line parsers unless explicitly asked.
- For tabular output, do not use
print()loops but usetabulate.tabulate()function, seewhitelist-vaults.pyas an example
Parallerisation and optimising long running data reading pipelines
- Uses
joblib.Parallelto parallerise API reading of multiple entries - Use threading backend unless explicitly specified otherwise
- For example, see
lead_scan_core.py - All functions using
joblib.Parallelmust takemax_workersargument. This must be exposed to command line scripts asMAX_WORKERSenvironment variable, seescripts/erc-4626/scan-vaults.pyas an example.
Working with RPC and blockchain explorers
- Prefer a blockchain explorer like Etherscan over Python and Curl when trying to read proxy contract address
- Prefer Python snippets instead of
curlwhen trying to read data directly from a blockchain explorer - To get the latest block number, use given JSON-RPC URL and Python's Web3.py
web3.eth.block_numbercall - Never try to figure out RPC URL yourself - always use environment variables from the local environment given by the user. See
eth_defi.chain.CHAIN_NAMESfor aliases like chain id 999 -> JSON_RPC_HYPERLIQUD. Stop and ask user if you cannot figure out.
Contract ABIs
- All contract ABI JSON files live under
eth_defi/abi/(shared contracts at the top level, protocol-specific interfaces undereth_defi/abi/<protocol>/) and are loaded through the shared helpers ineth_defi.abi. - Before fetching a new ABI, or loading/using any ABI file, read
eth_defi/abi/README.mdfirst. It explains what EVM ABIs are, how to fetch verified ABIs from Sourcify, Etherscan and GitHub (including proxy → implementation resolution), where to store them, and the API references for loading and using ABI files in their different roles (contract binding, deployment, ABI-less encoded calls, event decoding).
Event logs
- Never use JSON-RPC
eth_getLogsfor event discovery, bulk event reads or historical event reads. Always use Hypersync, which avoids provider range limits and provides indexed event streaming. - Create clients through
eth_defi.hypersync.utils.configure_hypersync_from_env()and open streams througheth_defi.hypersync.session.open_hypersync_stream(). These wrappers apply the repository's configured endpoint, rate limiting and stream tuning. - For vault lead discovery, use
eth_defi.erc_4626.hypersync_discovery.HypersyncVaultDiscover. For a targeted event query, build ahypersync.Querywithhypersync.LogSelectionand consume it throughopen_hypersync_stream(); seeeth_defi.vault.flow_events.fetch_vault_flow_logs_hypersync_async()for the canonical pattern.
Block timestamps
- For all bulk or historical blockchain timestamp operations, use the cache-aware
Hypersync API:
fetch_block_timestamps_using_hypersync_cached()or thefetch_block_timestamps_multiprocess_auto_backend()wrapper with a Hypersync client. Do not stream timestamps throughget_block_timestamps_using_hypersync*()directly unless explicitly repairing known cache gaps. - Preserve and reuse the per-chain DuckDB cache at
~/.tradingstrategy/block-timestamp/{chain_id}-timestamps.duckdb. Do not use the legacy~/.tradingstrategy/block-timestamps.*location.
For JSON-RPC URL configuration, environment variables. The variables are in the format JSON_RPC_{CHAIN} where {CHAIN} is the uppercase chain name:
JSON_RPC_ETHEREUM- Ethereum mainnetJSON_RPC_ARBITRUM- Arbitrum OneJSON_RPC_BASE- BaseJSON_RPC_POLYGON- PolygonJSON_RPC_BINANCE- BNB Smart Chain a.k.a. Binance a.k.a. BNB chainJSON_RPC_HYPERLIQUID- HyperEVM
You chan find these in CHAIN_NAMES and in eth_defi.provider.env
Building integrated smart contracts
You can use Makefile commands make guard safe-integration to rebuild smart contracts for Satfe and Lagoon integration.
ABIs
- Read
eth_defi/abi/README.mdfirst — it is the canonical reference for fetching, storing, loading and using ABI files. - Store contract ABIs in
eth_defi/abi/<protocol>/as JSON files and load them through the shared ABI helpers. - Do not define inline ABIs in Python unless the fragment contains at most one or two functions.
- Regenerate ABI JSON for this repository's integrated smart contracts with the compiler. For external deployments, commit the verified or application-exported interface JSON and record its canonical source alongside it.
Documentation
Documentation uses Sphinx v4.5 for API and narrative documentation and lives in docs folder.
Ignore Sphinx documentation builds during development. Agents must not attempt to build the Sphinx documentation locally; documentation builds and their failures are outside the development verification workflow.
Never directly edit auto-generated sphinx files in _autosummary* folders.
Parquet schema migrations
The vault price pipeline accumulates months of historical data in vault-prices-1h.parquet. Losing this data requires days of re-scanning from archive nodes and may make older Monad data unrecoverable, because Monad does not provide archive-complete historical state.
- Never silently discard existing data. If a schema migration (
migrate_parquet_schema(),cast()) fails, the pipeline must abort with a hard error — never fall back toexisting_table = None. Silent data loss is worse than a crash. - Never catch
ArrowInvalidand reset to empty. If the existing parquet cannot be read or migrated, raise the exception so the operator can restore from a backup. - New columns must have null defaults. Add them via
migrate_parquet_schema()withpa.nulls(). Never require a value in existing rows. - Type changes need explicit migration. If changing a column's type (e.g.
uint32→uint64), verifycast()works on production data before merging — test with a copy of the real parquet, not just synthetic test data. - Always test schema changes against the production parquet. Download the current file and verify the migration path locally before deploying.
- Reader state loss causes full data wipe. The scanner deletes existing chain rows from
start_blockonwards. Ifreader-state.pickleis lost,start_blockfalls back to the earliest vault block, deleting all historical data for that chain. Treat reader state files as critical production state. - Preserve old Monad rows. A Monad rescan can replace rows only at or after the dynamically probed provider boundary. Never reset its reader state or delete older rows in an attempt to rebuild unavailable state history.
ERC-20
- Don't do hardcoded token decimal multiply, use
TokenDetails.convert_to_raw() - Use
TokenDetails.transfer()and similar - do not do raw ERC-20 contract calls unless needed - Use
eth_defi.hotwallet.HotWalletfor deployer accounts and signing transactions when possible
Web Fetching and 403
When fetching web pages, if web_fetch returns a 403 error, retry the request using the Chrome MCP tool to load the page in a real browser instead.
Prerequisites:
- Claude in Chrome extension (v1.0.36+) - Chrome Web Store
- Google Chrome running
- Direct Anthropic plan (Pro, Max, Team, or Enterprise)
Browser tools are automatically available when the Chrome extension is connected. Use @browser in your Visual Studio Code prompt to activate the connection.
When using browser tools, Claude may ask for permission to visit specific domains. Approve these prompts to allow browser automation. You can also pre-approve domains in the Chrome extension settings.
README files in the repository
Consult these for domain-specific context. Logo READMEs under eth_defi/data/vaults/original_logos/*/README.md document logo source URLs per protocol.
| Path | Description |
README.md | Web3-Ethereum-Defi project overview |
contracts/guard/README.md | GuardV0 — on-chain trade validation for asset management |
contracts/in-house/README.md | Web3-Eth-Defi integration contracts |
contracts/safe-integration/README.md | Trading Strategy Zodiac-module for Safe multisig wallets |
docs/README-Hypercore-guard.md | Hypercore native vault guard integration |
docs/README-hyperevm-goldsky-failure.md | HyperEVM goldsky eRPC "not enough agreement" consensus failure and Alchemy failover |
docs/README-hyperevm-hypercore-read-gas.md | HyperEVM HyperCore read gas — why HyperCore-reading vaults abort Multicall3 batches with -32003 out of gas and revert with CoreReaderLib.ReadFailure, and why they are not blacklisted |
docs/README-hypersync-tests.md | Hypersync scan tests — disabled on CI by default, how to run them on CI when needed |
docs/README-test-suite-performance.md | Test suite performance plan — CI caching, shared Anvil forks, vault-protocol gating |
docs/README-vault-scan-manifest.md | Private readiness receipt: R2 keys, schema, sparse candle semantics, cache policy and rollout |
docs/README-hypercore-permission-recovery.md | Hypercore permission clocks, backed-up recovery and gated private R2 maintenance |
docs/README-contract-size.md | Contract sizes and compiler optimisation |
docs/derive-onboarding/README-derive-trader.md | Derive session key for vault traders |
docs/protocol-research/README.md | AI-assisted vault protocol discovery notes |
docs/source/api/derive/README.md | Derive.xyz integration — implementation summary |
eth_defi/aave_v3/README.md | About Aave v3 integration |
eth_defi/abi/README.md | Contract ABIs — what EVM ABIs are, how to fetch them from Sourcify/Etherscan/GitHub, storage layout, and the loader/usage API reference. Read before fetching or using any ABI file |
eth_defi/abi/ipor/README.md | IPOR ABI source links |
eth_defi/abi/lagoon/README.md | Lagoon ABI source links |
eth_defi/abi/yield_basis/README.md | YieldBasis ABI sources and reviewed deployment interfaces |
eth_defi/abi/uniswap-swap-contracts/README.md | SwapRouter02 deployment on Base |
eth_defi/cctp/README-cctp.md | Circle CCTP V2 integration |
eth_defi/core3/README-core3.md | Core3 risk intelligence integration — modules, database schema, scripts, API reference |
eth_defi/xerberus/README-xerberus.md | Xerberus vault risk scores — per-vault pool ratings, DuckDB, export, scripts |
eth_defi/currency_api/README-currency-api.md | Historical exchange rate ingestion (fawazahmed0 Exchange API) into DuckDB |
eth_defi/data/vaults/README.md | Vault protocol metadata and logo system |
eth_defi/enzyme/README-Enzyme-vaults.md | Enzyme Blue and Onyx architecture, metadata and migration operations |
eth_defi/tokenised_fund/asseto/README-Asseto.md | Asseto dynamic registry, daily NAV history and operations |
eth_defi/erc_4626/vault_protocol/README-reader-states.md | Vault reader states and warmup system |
eth_defi/erc_4626/vault_protocol/README-utilisation.md | Utilisation and available liquidity metrics for lending vaults |
eth_defi/erc_4626/vault_protocol/README-vault-redeemable.md | Why utilisation ≠ redeemable liquidity for Morpho/IPOR multi-market vaults |
eth_defi/gmx/README-GMX-Lagoon.md | GMX Lagoon integration security analysis |
eth_defi/gmx/README-GMX-Vaults.md | GMX GM and GLV liquidity catalogue |
eth_defi/gmx/README.md | GMX CCXT adapter for eth_defi |
eth_defi/gmx/ccxt/README.md | GMX CCXT adapter implementation |
eth_defi/gmx/graphql/README.md | GMX Subsquid GraphQL integration |
eth_defi/vault_report/README-vault-report.md | Monthly best-performing stablecoin vaults blog report — data sources, sections, branded charts, Ghost draft workflow |
eth_defi/vault_report/README-blog-post-outline.md | Monthly vault report blog post outline — section order, vault groups, selection rules, editor input and open decisions |
eth_defi/vault_report/README-best-vaults-news.md | Monthly vault report blog post drafts — goal of the skeleton, GitHub PR comment previews, Ghost draft creation, architecture diagram |
eth_defi/yield_basis/README-YieldBasis.md | YieldBasis LT vault model, valuation and historical pipeline |
eth_defi/erc_4626/vault_protocol/rysk/README-Rysk-vaults.md | Rysk Premium epoch-priced option pools, adapter accounting and backfill operations |
eth_defi/erc_4626/vault_protocol/antarctic/README-Antarctic.md | Antarctic perpetual DEX, AMLP/AHLP settlement data and event-based vault support |
eth_defi/lighter/README-lighter-guard.md | Lighter (zk-rollup perps DEX) L1 deposit/withdraw guard integration — architecture, security model, operator flow |
eth_defi/testing/README.md | Fast Anvil fork tests — shared session forks, per-chain midnight block cache, snapshot/revert, once-per-session deployments, the committed fork RPC cache and token cache (create/rebuild/purge), Anvil 1.8 behaviour and mitigations (cache source_id, async mining, hedged receipt lookups), diagnosing slow fork tests, reference tests |
eth_defi/testing/rpc_cache_seed/README.md | Committed Anvil fork RPC cache seed — layout, how to capture and refresh it |
scripts/base/README.md | Base chain related manual test scripts |
scripts/debian-bullseye-compatibility/README.md | Running on Debian Bullseye |
scripts/erc-4626/README-vault-scripts.md | ERC-4626 vault scripts |
scripts/grvt/README-grvt-vaults.md | GRVT native vault metrics pipeline |
scripts/hyperliquid/README-hyperliquid-copy-trading.md | Hyperliquid copy trading platforms and HFT account identification |
scripts/hyperliquid/README-hyperliquid-vaults.md | Hyperliquid native vault metrics pipeline |
scripts/hyperliquid/README-hyperliquid-vaults-high-frequency.md | High-frequency Hyperliquid vault data fetcher |
scripts/lighter/README-lighter-vaults.md | Lighter native pool metrics pipeline |
tests/erc_4626/vault_protocol/README.md | Vault protocol detection tests (mainnet-fork) |
tests/gmx/README.md | Testing for GMX |
tests/guard/README.md | Integration tests for GuardV0 and TradingStrategyModuleV0 |
tests/provider/README.md | Service provider integration tests |
tests/rpc/README.md | JSON-RPC scenario tests |
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

