agentleFS
wingfoil-iowingfoil-io/wingfoil/CLAUDE.md

wingfoil

CLAUDE.md231 starsChanged 23 days agoApache-2.0View on GitHub

Guidance for Claude Code when working in this repository. Wingfoil is a Rust stream processing library for building directed acyclic graphs (DAGs) of data transformations, supporting both real-time and historical (backtesting) execution. The repository root is Wingfoil, the Op-pattern engine. It replaced the original MutableNode engine, which lived under legacy/ until it reached parity and was deleted at the cutover (docs/planning/cutover-plan.md). New to the engine? Read docs/wingfoil-architecture.md before your first non-trivial change — the shape of the thing, the one…

  • Installs packages
  • In this repositoryOne other file it gives its agents.

The file

Show its rules and text

We split this CLAUDE.md into the rules it gives an agent

15 rules. About 0 are common, 5 are distinctive, and 10 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈5

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 10

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Every example lives in its own directory with main.rs and README.md.

    line 96
  2. Very specific:

    Every target is declared explicitly in Cargo.toml with a path (autoexamples = false), under the # Examples section.

    line 97
  3. Very specific:

    Every example is linked from its group's README.md, and each group is linked from examples/README.md.

    line 99
  4. Very specific:

    Every crate carries a README.md, and crates/README.md is the crate map — keep them current when a crate's role changes.

    line 145
  5. Very specific:

    Fallibility: every lifecycle fn returns anyhow::Result.

    line 167
  6. Distinctive:

    Do the work, commit, and push the feature branch.

    line 271
  7. Distinctive:

    Use the setup script for the native Aeron and ZeroMQ build dependencies:

    line 314
  8. Very specific:

    .cargo-husky/hooks/pre-commit and pre-push: run it before their --all-targets step, so a low-headroom tree reclaims rather than dying …

    line 347
  9. Very specific:

    Caret always: Never =, never a bare major, never a wildcard.

    line 434
  10. Very specific:

    major.minor granularity: Add a .patch component only when a specific patch release introduced something we use, and say so in a comment.

    line 441
  11. Very specific:

    Major: only if a dependency's types are in our public API.

    line 494
  12. Very specific:

    No locks on the graph execution path (cycle / start etc.).

    line 546
  13. Distinctive:

    All three are living documents — keep them current: Every time you onboard a new op, adapter, or binding, *or change an existing one*, …

    line 567
  14. Distinctive:

    Before committing any changes, ALWAYS run:

    line 580
  15. Distinctive:

    All must pass without errors before creating a commit.

    line 614

≈ “Common” is an estimate: at least 1 in 5 of the 27,641 CLAUDE.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
CLAUDE.md

CLAUDE.md

Guidance for Claude Code when working in this repository.

Project Overview

Wingfoil is a Rust stream processing library for building directed acyclic graphs (DAGs) of data transformations, supporting both real-time and historical (backtesting) execution.

The repository root is Wingfoil, the Op-pattern engine. It replaced the original MutableNode engine, which lived under legacy/ until it reached parity and was deleted at the cutover (docs/planning/cutover-plan.md).

Repository Structure

crates/                     # every Cargo crate in the tree
  wingfoil/            # The engine: op.rs, interp.rs, fluent.rs, ops.rs, trading/,
                            #   stats.rs, adapters/, channel.rs, async_source.rs,
                            #   introspect.rs, latency.rs, runtime/, examples/,
                            #   tests/, benches/
  wingfoil-derive/     # nitro! / #[op] proc macros
  wingfoil-python/     # PyO3 Python bindings (built with maturin)
  wingfoil-python-derive/
  wingfoil-wire-types/      # Wire-format types shared by the web adapter and
                            #   wingfoil-wasm — survives cutover
  wingfoil-wasm/            # Browser-side WASM codec (excluded from the default
                            #   workspace) — survives cutover

docs/                       # User-facing docs live at the top level:
                            #   wingfoil-architecture.md (read this first),
                            #   adding-an-op.md (the op recipe + touch-point
                            #   table), migration.md (#[node] -> Op),
                            #   python-interop.md, comparison.md
  release-notes/            #   One page per version, newest first
  decisions/                #   One page per ruling, and every one is settled
                            #   and true of main: runtime ownership, source
                            #   lifecycle, macro extensibility
  planning/                 #   Internal planning: cutover-plan.md +
                            #   cutover-runbook.md (the remaining swap),
                            #   deviation-register.md, introspection-plan.md,
                            #   trading-roadmap.md. port-plan.md is a
                            #   *historical record* of the port, not a backlog
                            #   — open work is in issues
    proposals/              #   Designed and argued, not built: Project
                            #   Lightning — wired-graph codegen (#726) — and
                            #   Project Metal — FPGA/HDL backend (#727). The
                            #   tracking issue is the status; see docs/README.md
                            #   "Ruling or record?" for which dir a page goes in

js/                         # TypeScript client for the web adapter — an npm
                            #   package, not a crate (@wingfoil/client)

scripts/                    # Dev helpers (setup-dev.sh, ci-logs.sh, disk.sh,
                            #   bench-report.sh)

Start here

New to the engine? Read docs/wingfoil-architecture.md before your first non-trivial change — the shape of the thing, the one decision everything else follows from, and the rules that bite. Porting code off the legacy engine is docs/migration.md.

The design objective that got us here

Wingfoil had to become a strict superset of the legacy engine — every node, adapter, run mode, example, benchmark and binding — so that legacy/ could be deleted outright. It did, and it was. The consequence that still binds: a legacy capability, example or test case is never silently dropped. Where wingfoil deviates deliberately, the deviation is on record (docs/planning/deviation-register.md), and the capability matrix in docs/planning/port-plan.md is the historical account of the port.

The parity oracle is gone with the tree. Tests that recorded what legacy did survive as ordinary regression tests with their expectations pinned as captured constants — crates/wingfoil/tests/engine_semantics.rs is the pattern. Do not weaken those constants; they are the last word on the behaviour they describe.

Examples: every one is a directory with a README

crates/wingfoil/examples/ is grouped by what an example teaches:

  • core/ — engine concepts (wiring, run modes, nitro! tiers, threading, dynamism). No external services; runs with plain cargo run.
  • adapters/ — one directory per I/O adapter, named after the adapter (adapters/csv/), feature-gated.
  • trading/ — the wingfoil::trading layer (market/, execution/), feature-gated, offline.
  • showcase/ — multi-process, end-to-end latency demonstrations.

The rules, enforced by scripts/check-example-docs.sh in CI:

  1. Every example lives in its own directory with main.rs and README.md.
  2. Every target is declared explicitly in Cargo.toml with a path (autoexamples = false), under the # Examples section.
  3. Every example is linked from its group's README.md, and each group is linked from examples/README.md.

Target names are decoupled from directory names and must not change — adapters/csv/ holds the target csv_adapter, so cargo run --example csv_adapter keeps working. Renaming a target breaks users' muscle memory and every doc reference; renaming a directory is free.

One deliberate exception, on record: latency_e2e became trading_e2e — directory and both targets (trading_e2e_ws_server, trading_e2e_fix_gw) — because the old name described one of the things that example does rather than what it is. Renaming the directory alone would have left the command users type still saying latency, which is the whole thing being corrected. The cost the rule warns about was paid in full: every workflow, Dockerfile, Pulumi stack and doc reference moved with it, and so did the namespace the example emits into — Prometheus metrics and job names, iceoryx2 services, #[type_name] pins, the OTLP service name and the Grafana dashboard UID are all trading_e2e now. Every consumer of those names lives in the same directory and moved in the same commit.

The line that did hold is between names a binary emits and names that identify deployed state. The Pulumi project names, the Packer AMI name and the SSM parameter path keep wingfoil-latency-*, because a Pulumi project name is part of stack identity (renaming orphans running stacks) and the SSM path has an IAM grant scoped to it. Those move in a deploy window, not in a commit. Use that split for any future example rename.

House style differs by group (core/ uses ## Sentence-case title then prose, snippet, output; adapters/ uses # Name Adapter Example (wingfoil) then ## Prerequisites / ## Run / ## Code / ## Output). Match the group you are adding to.

Sample output in a README must be real — run the example and paste what it prints. Several adapter READMEs were written from invented output and had to be corrected against the actual println!s; do not repeat that.

So an example whose README pins its output must be deterministic — run it under RunMode::HistoricalFrom(..), not RealTime. The two rules collide otherwise: a realtime example driven by a worker thread prints a different interleaving on consecutive runs (values coalesce into bursts or land after the error that ends the run), so whatever you paste is wrong on the next run and the README rots by design. Where an example exists specifically to show realtime behaviour, keep the pinned section to the parts that do not vary and say in prose that the interleaving depends on scheduling — do not paste a sample that only reproduces sometimes.

Every crate carries a README.md, and crates/README.md is the crate map — keep them current when a crate's role changes.

Key concepts (how wingfoil differs from legacy)

  • Op trait (op.rs): semantics as associated functions — cycle(cfg, state, input, ctx) — never methods on an instantiated object. Cfg = construction-time config (closures live here), State = engine-owned mutable state, In<'a> = typed inputs passed in per cycle, Out = produced value. const ACTIVATION: Activation declares scheduling behaviour statically (NONE / SCHEDULES / THREADED / ALWAYS).
  • Tick<T>: Value (tick downstream), Silent (update the value slot without ticking — what delay needs), Quiet (nothing).
  • Wiring: GraphBuilder + Stream<T> with combinators as extension traits (SourceOps, StreamOps, StatisticsOps, adapter traits). New vocabulary is added via the two public primitives GraphBuilder::source and Stream::wire — never by editing Stream itself.
  • Sources: ticker / constant / external (threaded, realtime-only) / channel (both modes; timestamped historical replay via send_at) / poll (busy-spin, realtime-only) / feedback (wired in one place with feedback_loop). Burst delivery everywhere — same-instant values grouped, never latest-wins, never dropped.
  • Fallibility: every lifecycle fn returns anyhow::Result; sender.send_error(e) propagates a producer error into the graph and aborts the run with context.
  • nitro! has no per-op table: #[op(build = name)] on an Op impl generates both the interpreted builder method and the forwarders that compiled/nested emission dispatches through. Built-in and user ops take the identical path.

Two clocks: engine time and the wall snap — and who owns each

Kernel keeps them as separate fields, and conflating them is the mistake to avoid:

  • time (Ctx::time()) — engine time, source-driven. Under HistoricalFrom it is pure logic: begin_cycle pops the earliest scheduled callback and sets time = next.max(time + 1), consulting no clock at all. Under RealTime it is NanoTime::now().max(time + 1). This is the only one business logic may use — it is what makes replay deterministic.
  • wall_time (Ctx::wall_time()) — this cycle's wall-clock snap, for latency stamping and telemetry. A real clock read in both run modes, so "time spent" means the same in a backtest as live. Never branch business logic on it.

The kernel owns the snap and takes it lazily, on first read. Kernel::wall_time() snaps NanoTime::now() only if this cycle hasn't yet, caches it in a Cell, and returns the same instant to every later reader in that cycle. begin_cycle does not snap — it only invalidates (wall_time.set(None)) on each of its paths. So a cycle in which no op stamps reads the clock zero times, which matters because the read is ~24 ns (see the nanotime bench) and almost nothing wires latency::Stamp.

Three invariants follow, all easy to break by accident:

  1. Don't snap eagerly in begin_cycle — that is what the Cell exists to avoid. A constant restored there lands entirely in the per-cycle intercept.
  2. Ctx::new must not copy the snap. Ctx is built once per node per cycle, so an eager copy forces the kernel's snap every cycle and undoes the laziness. It leaves wall_time: None and defers through Sink::Kernel.
  3. Keep Ctx::wall_time() on &self — the Cell (rather than a &mut self accessor) is what allows this.

The three tiers differ only in who holds the kernel:

  • compiled() owns one outright — let mut __k = Kernel::new(..) as a stack local, driving its own while __k.begin_cycle(..) loop. Nodes get Ctx::new(&mut __k, i), i.e. the identical lazy path. Nothing about time is special-cased for this tier.
  • interpreted — the Runner holds the kernel; same Ctx::new, same laziness.
  • nested() (island) borrows the outer kernel's values once per activation (__ctx.time() / __ctx.wall_time() / __ctx.start_time()) and hands every inner node a Ctx::nested(..) carrying wall_time: Some(snap), with the sink swapped to the composite's private TimeQueue — only next_time() is forwarded outward. Time stays globally consistent because the island reads the outer engine's clock rather than keeping its own.

    The island is the one place laziness is given up, necessarily: the composite must resolve the snap before running its interior and cannot know whether any inner node will look, so it takes exactly one snap per activation either way. Don't "fix" that by making it lazy per inner node — the previous design called NanoTime::now() per inner node per activation (~24 ns each), which was the island tier's entire per-node deficit against compiled().

Two documented island consequences, deliberate: an op's start observes NanoTime::ZERO for the wall snap (true of flat graphs too — start runs before the first cycle), and is_last_cycle / run_mode are not propagated into an island.

TimeQueue deduplicates by design — don't "fix" it

runtime::time_queue::TimeQueue<T> (backs the graph scheduler, feedback, delay, and legacy's CallBackStream) intentionally suppresses duplicate (value, time) pairs: pushing a (value, time) already queued is a no-op. This is a feature, not a bug — e.g. a node scheduled for the same instant twice, or the same feedback value sent twice in one cycle, must collapse to a single event. Do not remove duplicate suppression.

Because dedup only needs equality (not hashing), the bound is T: PartialEq, not Hash + Eq. That is deliberate: it lets f64 (and other float payloads, which implement PartialEq but neither Hash nor Eq) flow through delay and feedback. Keep the bound at PartialEq.

Dedup is scoped to the instant being pushed to, and must stay that way. Entries live in a BTreeMap keyed by time, so a push compares only against that time's bucket. A flat scan of every pending entry — which is what this queue used to do — is O(pending) per schedule, and pending here is not small: the scheduler holds one entry per timer at all times, because a ticker re-arms itself the moment it fires. That put every timer in the graph on the cost of every other timer's tick (22 ns per schedule at one pending entry, 686 ns at 1024), and delay paid the same shape with every value still in flight. The earliest instant is additionally held out of the map and refilled lazily, so a single-timer graph — and a fast timer among slow ones — never touches the map at all.

Branching: everything merges into main

next was the long-lived integration branch that staged the replacement engine in one place. It has landed on main, so main is the trunk for this repository. There is no longer a second integration branch.

  • NEVER edit files directly on main.
  • The workflow for any change, anywhere in the tree:
    1. Cut a feature branch from main: git checkout main && git pull origin main && git checkout -b <branch-name>.
    2. Do the work, commit, and push the feature branch.
    3. Open a pull request with base main.
  • Branch naming: simple descriptive names (e.g. add-metrics, fix-error-handling).

next is retired, not renamed, and the branch is gone. The CI push/pull_request triggers and cache save-if guards that still named next alongside main became inert with it, and were stripped on 2026-08-12 (rust-test.yml, python-test.yml, security-audit.yml).

Build Commands

# Build
cargo build
cargo build --release

# Test
cargo test
cargo test -p wingfoil --all-features

# Python tests
cd crates/wingfoil-python && maturin develop && pytest

# TypeScript client tests
cd js && pnpm test

# Benchmarks
cargo bench

# Lint (these aliases live in .cargo/config.toml and mirror CI exactly)
cargo lint        # default features
cargo lint-all    # all features — catches code behind `fix`, `csv`, `iceoryx2`, etc.
cargo fmt --all -- --check

protoc is required on the build machine (a transitive dependency builds proto files). On Debian/Ubuntu: sudo apt-get install -y protobuf-compiler, or run scripts/setup-dev.sh. Note this is needed for a plain cargo build --workspace, not just lint-all — see the feature-unification note below.

All-features system dependencies

Use the setup script for the native Aeron and ZeroMQ build dependencies:

scripts/setup-dev.sh --all-features

This keeps the ordinary protoc-only setup small. The explicit mode also checks that CMake is at least 3.30 because the vendored rusteron-media-driver Aeron sources set that floor in their own cmake_minimum_required; older Linux packages are replaced with the pinned Kitware build. Missing libbsd otherwise surfaces only at link time as rust-lld: error: unable to find library -lbsd.

Disk space

Builds here are big enough to exhaust a dev sandbox, so it is worth knowing where the space goes. scripts/disk.sh reports it and reclaims it:

scripts/disk.sh          # what is using space
scripts/disk.sh light    # drop examples/benches/incremental, keep deps/
scripts/disk.sh deep     # also remove every target/ dir in the tree
scripts/disk.sh auto     # light (then deep) only if headroom is low

Reach for light mid-session: it keeps target/*/deps, so the next build relinks instead of recompiling 700+ crates.

auto is the unattended form, and it is wired up in three places so you should rarely have to think about this at all:

  • .claude/settings.json runs it after every Bash call in a Claude Code session, and sets CARGO_INCREMENTAL=0 for those sessions (see below — worth 2.6GB, and an agent doing one-shot builds never collects on incremental).
  • .cargo-husky/hooks/pre-commit and pre-push run it before their --all-targets step, so a low-headroom tree reclaims rather than dying partway through the link.

It costs one df and prints nothing when there is room, so it is cheap enough to hang off a per-command hook. Thresholds are in GB and tunable: WINGFOIL_DISK_MIN_GB (default 10) is where it runs light, and WINGFOIL_DISK_FLOOR_GB (default 4) is where a still-full tree escalates to deep. The 10GB default is sized off what it has to leave room for: a --all-targets dev build lands around 9.2GB.

This bites hardest in a Claude Code cloud sandbox, where the writable disk is a fixed per-session allowance (~30GB) rather than the size df reports for the device. Two --all-targets feature sets plus incremental will spend all of it. On "no space left on device", light first — deletes still succeed while writes fail, and the freed space is immediately usable, so the session is recoverable without starting a new one.

Three things make the tree large, and they compound:

  • Feature unification makes a --workspace build wide. Cargo unifies features across the workspace, so a plain cargo build --workspace compiles the union of what every member asks for; cargo lint and cargo lint-all differ only by aeron and iceoryx2. The figures below were measured while legacy/ was still in the tree and are therefore worst-case — a root build now compiles this tree's own feature selection and nothing more.
  • lint-all cannot reuse lint's work. A different feature set means a different metadata hash, so the two pre-commit lints build two full artifact sets back to back. If space is tight, scripts/disk.sh light between them.
  • --all-targets links ~69 example and bench binaries, each statically carrying the entire dependency graph.

That last one is why [profile.dev.package."*"] debug = false is set in the root Cargo.toml: debuginfo was over half the bulk and got copied into every one of those binaries. It takes a single example binary from 260MB to 64MB, and a full --all-targets build (79 binaries) measures 9.2GB with it — against roughly 17GB without, derived from the same per-artifact ratios. The override applies to dependencies only — cargo excludes workspace members from package."*" — so backtraces and debugger stepping through wingfoil code are unaffected; only frames inside third-party crates lose line numbers.

target/debug/incremental is another 2.6GB of that 9.2GB. It pays for itself across edit-rebuild cycles, but CARGO_INCREMENTAL=0 (what CI sets) reclaims it outright if you are only doing one-shot builds.

Also note the git hooks installed by .cargo-husky: pre-commit runs cargo fmt --all and a full cargo clippy --workspace --all-targets, and pre-push runs cargo build plus cargo test across the workspace. Every commit and push rebuilds, so expect commits to take minutes, not seconds — and both now open with scripts/disk.sh auto, because that --all-targets step is the largest single demand this repo makes on a disk.

WINGFOIL_HOOKS scales that down when the change does not warrant it. fast runs fmt plus clippy --workspace --lib --bins (26s on a warm tree against minutes, because it does not link the ~69 example and bench binaries) and, on push, cargo test -p wingfoil; off skips both hooks. Default is full, which is the behaviour above. Use fast for a docs-, shell- or config-only change and full for anything touching Rust that CI would lint as a target — the modes trade feedback latency, not the bar, since CI runs the whole set either way.

Contributors have a one-click path to the same toolchain: .devcontainer/ (Codespaces or local Docker) carries current stable Rust, protoc, Python and a Docker daemon, and post-create.sh deliberately does not build the tree.

Toolchain gap (clippy): CI runs clippy on the current stable rustc, which can be newer than the toolchain in a dev sandbox. Newer clippy adds lints (e.g. collapsible_match) that the older one doesn't emit, so a local cargo lint can pass while CI fails with -D warnings. If CI's clippy step is red but local is green, reproduce with CI's version explicitly:

rustup toolchain install <ci-version>   # e.g. 1.97.0 — see the failing log's clippy URL
cargo +<ci-version> clippy --workspace --all-targets -- -D warnings
cargo +<ci-version> clippy --workspace --all-targets --all-features -- -D warnings

Dependency policy

wingfoil is a published library, so the manifests are a public contract in a way the lockfile is not. Cargo.lock does not travel: it ships inside the .crate, but cargo ignores a dependency's lockfile. Every version decision a consumer gets is made by the caret ranges in Cargo.toml alone. The lock pins CI and dev machines, nothing else.

Eight rules follow from that.

  1. Caret always. Never =, never a bare major, never a wildcard. An = pin makes this crate un-unifiable with anyone else's graph.
  2. The version string is a floor, and its only job is to be true. The caret already hands users the newest compatible release, so the only thing the number encodes is the minimum we actually compile against. tokio = "1.53", never tokio = "1" — the latter claims tokio 1.0.0 builds, and it does not.
  3. major.minor granularity. Add a .patch component only when a specific patch release introduced something we use, and say so in a comment.
  4. Set the floor from the lock when you add the dependency; raise it in the same commit that reaches for a newer API. Don't guess, and don't audit retroactively. The floors here are conservative — set to what the lock resolved rather than to a verified minimum. cargo minimal-versions check is what would let you lower one with confidence; without it, do not lower a floor, because "probably fine" is the same unverified claim in the other direction.
  5. Never an umbrella crate, never features = ["full"]. Take the sub-crate you use: crossbeam → crossbeam-channel, futures → futures-util. This is the rule that pays every consumer on every build — the default graph is 34 crates and umbrella deps were six of them.
  6. default-features = false unless you have checked what the defaults drag in. A dependency's default features are unmaskable by consumers: they cannot turn off what we switch on. reqwest's defaults put OpenSSL in a tree that is rustls everywhere else.
  7. One declaration per dependency. Anything named by two members — or by both [dependencies] and [dev-dependencies] — lives in [workspace.dependencies]. Version at the workspace, features at the member. This is invisible to consumers (cargo package flattens workspace = true to a literal) and exists purely so two manifests cannot drift to different floors, which is exactly what thiserror and serde had already done.
  8. Path deps keep { path = "…", version = "9.0.0" }. bump.yml moves them in lockstep via cargo set-version; don't hand-edit and don't hoist them into the workspace table.

Enforcement, because rules rot

  • scripts/check-dep-duplicates.py — the check that catches a stale floor growing a second copy of a stack. It fires on exactly the shape we can fix: a crate we name directly sitting behind a newer, semver-incompatible line that something else already pulled. It deliberately does not gate on duplicates between two third-party crates — an --all-features resolve here is ~700 crates with ~50 such pairs, and cargo deny's multiple-versions = "deny" would need a 50-entry skip list that would rot exactly the way the stale floors did. Exceptions live in the script's ALLOWED set with a reason; adding one is a decision, leaving one to rot is not.
  • cargo deny check — licences, sources and advisories. bans is left at warn there for browsing, per the above.
  • --locked in CI (rust-test.yml) — the committed lock is what gets tested, and a manifest edit that needs a new lock entry fails loudly instead of being papered over in the runner.
  • Renovate, weekly and grouped (.github/renovate.json), with rangeStrategy: "bump" so it raises the manifest floor and not just the lock. Floors go stale silently; nothing else fixes that.

Does a dependency change need a version bump?

A bump is needed to publish at all. Which kind:

  • Major only if a dependency's types are in our public API. They currently are not — there is no pub use of a dependency anywhere in src/. The one dep type on the public surface is pub type ReadyReceiver = crossbeam::channel::Receiver<usize> (runtime/kernel.rs), and crossbeam is a facade re-exporting crossbeam-channel, so depending on the sub-crate directly leaves the type identity unchanged.
  • Minor for raising a floor (it can force a consumer's cargo update, or fail to resolve against something else capping the range) and for narrowing a dependency's default features (anyone receiving, say, chrono/clock through feature unification with us loses it).
  • Patch for the hygiene half only — workspace-table consolidation, deny.toml, Renovate, comments. None of that changes the published manifest.

A breaking upstream release (sha2 0.11, reqwest 0.13) is a build task, not a semver event: it breaks our code, not our API.

Error Handling

Production code must not call .unwrap(). Replacement priority:

  1. ? — preferred. Every lifecycle function returns anyhow::Result, so propagate via ? and add .context("…") from anyhow::Context at I/O boundaries (file open, socket connect, codec decode).
  2. .expect("invariant: WHY") — only when a precondition makes the None/Err branch unreachable. The message must explain the invariant (e.g. expect("current_node_index set during cycle")).
  3. .unwrap() — allowed inside #[cfg(test)] modules and doc comments showing example usage; otherwise disallowed.

Mutex poisoning is not recovered: .lock().expect("<name> mutex poisoned") is the pattern. The expect documents intent — a poisoned lock means another thread panicked while holding it, and we propagate that panic deliberately.

Working conventions

  • Tests use RunMode::HistoricalFrom(NanoTime::ZERO) for determinism, and assert exact values and tick times (with_time() + accumulate()).
  • accumulate() / collapse_accumulate() are test instruments, not output edges. They grow a Vec by one entry per tick for the whole run and clone it on every tick, so they are unbounded in realtime and O(n²) copying in a backtest. Use them where a bounded run's whole sequence is needed in one value afterwards — asserting values and tick times, or tying two runs out against each other. Everywhere else — examples included — emit as the run progresses: print(), logged(..), for_each(..) / for_each_mut(..), inspect(..), or window(..) / buffer(..) for a bounded look-back. An example that collects into a Vec only to println! it after the run is the anti-pattern; it also stops the same wiring being pointed at a live feed.
  • Temp files in tests get unique names (pid + counter) so parallel tests never collide; see crates/wingfoil/tests/lines_adapter.rs.
  • Feature-gated tests start with #![cfg(feature = "...")] at file level.
  • Adapter and stats ops stay out of the prelude — users opt in with use wingfoil::adapters::<name>::...;.
  • No locks on the graph execution path (cycle / start etc.); use the channel layer to talk to background threads.

Skills — and they are living documents

Three skills carry the step-by-step recipes for the kinds of surface you add to wingfoil. Use them for their respective tasks:

  • /new-op (.claude/commands/new-op.md) — adding a node/op to the catalog (ops.rs / stats.rs): the Op shape, #[op(build = …)], the fluent extension-trait method, nitro!/compiled coverage, the #[pyop] / pyop_fn! Python bindings, and the parity + completeness tests.
  • /new-adapter (.claude/commands/new-adapter.md) — adding an I/O adapter under src/adapters/: source/sink shapes, feature gating, the parity obligation, and the adapter tests.
  • /bind-adapter (.claude/commands/bind-adapter.md) — adding the Python bindings for an adapter that is already ported: #[pyadapter] shapes, the feature/wheel roll-ups, dynamic payloads, the GIL rules, and the three test tiers. /new-adapter links here for its Python step, so a brand-new adapter runs both.

All three are living documents — keep them current. Every time you onboard a new op, adapter, or binding, or change an existing one, check whether the work surfaced something the matching skill doesn't yet capture — a recurring pitfall, a new invariant, a CI gate, a pattern worth codifying, a deviation that should become a rule — and fold it back into that skill (ideally in the same PR). The adapter skill already grew several of its rules exactly this way, and the binding skill was extracted out of it once the first real adapter binding landed. A skill that has drifted from how we actually build ops/adapters is a bug; treat updating it as part of "done", not an optional extra.

Pre-Commit Checklist

Before committing any changes, ALWAYS run:

cargo fmt --all
cargo lint        # default features
cargo lint-all    # all features — CI runs this and feature-gated code is easy to miss
cargo test -p wingfoil --all-features
cargo test -p wingfoil-derive   # a separate package: the line above never builds it
python3 scripts/check-dep-duplicates.py   # only if you touched a Cargo.toml
cargo test -p wingfoil --examples --features <the example's>   # only if you touched an example with tests

Two notes on what those last two lines do and don't reach:

  • -p wingfoil does not build the derive crate's tests. nitro!/#[op] reason about the wiring function they are handed, and that reasoning has unit tests of its own in crates/wingfoil-derive/src/lib.rs. They only run when you name the package. (CI ran nothing that named it until rust-test.yml grew a step for it — issue #835.)
  • -p wingfoil does not run example tests either. --lib --tests and a plain cargo test leave out example targets, so the #[cfg(test)] modules some examples carry (trading_e2e, the dynamism examples) only run under --examples. CI runs them in their own step (rust-test.yml, "Run example tests"); locally, name the example's features.
  • Doc fences are already covered here, and that is a property of cargo test, not of the flags. cargo test runs the --doc target alongside the unit and integration ones; --all-features matters because most of the doc fences live in adapters/, which is gated off by default — a default-feature run collects 12 of them, all-features 35. CI needs a separate step for this (cargo test --doc -p wingfoil --all-features in rust-test.yml), because the test leg runs cargo nextest, which drives libtest harnesses and cannot run doctests at all. So do not "simplify" the local checklist to cargo nextest run: the doc fences would stop being checked before commit.

All must pass without errors before creating a commit.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.