Performance

Measure first. A performance claim without a method behind it is not a result.

Measure

ls benchmarks/
python3 scripts/run_bench.py --help
bash scripts/bench.sh
flow test-matmul                 # matmul demo plus assembly

Report every time: the machine, the operating system, the compiler and its version, the Flow host, the backend, the optimisation flags, the input size, the number of repetitions, and the spread across runs. A single timing on a laptop that was also compiling something is noise.

The routes

FLOW_HOST=python flow mlir program.flow --optimize
FLOW_HOST=python flow mlir-run program.flow
FLOW_HOST=python flow fir-g program.flow --opts     # FIR-G graph and analyses

fir-g dumps the graph and its analyses, with --calibrate and --opts. Use it to see what the optimiser knows before guessing at a change.

Order of attack

1. Confirm the hot spot with a measurement. Do not optimise from intuition.
2. Fix the algorithm. A better method beats every flag.
3. Remove allocation and copying from the inner loop.
4. Check the generated C for something obviously silly, such as a bounds check that cannot fail or a struct copied by value in a loop.
5. Try the other backend and measure both.
6. Only then reach for SIMD or GPU, and measure the transfer cost as well as the kernel.

Read the generated C

FLOW_HOST=python flow compile program.flow
cat "$FLOW_HOME/build/program.c"

This is the honest account of what a construct costs. If a declarative form lowers badly, that is a compiler issue worth filing rather than a reason to hand-write the loop everywhere.

Guard against measuring nothing

• Make sure the result is used, or the compiler will delete the work.
• Warm up before timing.
• Time enough iterations that the clock resolution stops mattering.
• Run the baseline and the change on the same machine in the same session.
• Report the spread, not just the best run.

Reporting

Say what got faster, by how much, on what, and what did not change. Label estimates as estimates. If you only ran the benchmark once, say so.