Workload Tests

Use this skill for tests under examples/*/workloads, bb-tests/workloads, and chip regression manifests. Keep the test's layer, scale, backend, and name explicit.

Test layers

Do not call a test full_bank merely because it is a BankOp. Name the operation, layout, data pattern, or boundary it proves, for example im2col_k3_windows or toint8_i32_pack16.

Source and coverage rules

• One focused test per source file. No multiple mains, multiple RUN cases, or case-switching test files.
• A test source, including generated MLIR input, must not exceed 100 lines. Split by behavior instead of moving test logic into a large shared helper.
• Preserve distinct coverage when reorganizing. Do not delete an old test until a registered replacement covers the same behavior at the correct layer. Kernel-size, padding, layout, tail, accumulation, and sequencing variants are distinct coverage unless proven otherwise.
• Keep CTest and BallOp inputs small and hand-checkable. BankOp tests must exercise real computation over at least one configured bank, not a unit-scale, identity-table, or no-op data path dressed up as a bank test.
• Do not add standalone "contract tests." If a Ball cannot operate at the active geometry, make its CMake configuration fail with the Ball name, required geometry, and actual generated values.

Parameters and configuration

• Geometry and topology come from the active chip's chip.pb and generated headers (params.h, topology.h), never from fixed bank depths, widths, thread counts, or shared-bank bases in a test.
• Derive row/lanes/counts from generated parameters where the Ball ISA does not fix them. ISA-fixed dimensions such as a 16x16 array may remain explicit.
• A test must continue to mean the same thing after valid chip parameter changes. Reject unsupported geometries at configuration time; do not clamp, silently shrink, or provide a fallback.

Multicore chip tests

• Build one workload ELF. It dispatches with bb_get_thread_id(); the framework does not know role names or choose a leader after boot.
• bb_get_thread_id() returns {core, thread}. Cores are homogeneous; thread-level paths may differ within a core.
• Put heterogeneous paths in separate C files in one workload directory. The *-main.cpp file only owns shared state and dispatches by {core, thread}.
• Use the shared bare-metal/Linux multicore runtime. Thread 0 performs the existing level-one boot and reports completion; do not recreate a parallel boot protocol in individual tests.

Registration and regression

• Workload names always begin with <chip>-<thread>-. The two naming entry points are buckyball_ctest_name(thread, test_id), which produces <chip>-<thread>-ctest-<test_id>, and buckyball_mlirtest_name(thread, test_id), which produces <chip>-<thread>-mlirtest-<test_id>. Each test owns its test_id. For example, CTest IDs may be mvin_2d, thread-conv_pipeline, or chip-shared_transpose; MLIR IDs may be ball_transpose_i8 or bank_quant_i32_pack16. Do not add local name-formatting helpers.
• Every active test must be registered in its local CMake file and built by its layer's aggregate target. A source present on disk but absent from an aggregate is not coverage.

• Regression manifests list only the complete workload file name:

  [workloads]
  tests = ["pebble-pebble-ctest-example-baremetal"]

  A workload file name must be unique under bb-tests/output. The runner scans that output root and fails if the name is missing or has multiple matches. Do not use search_path, paths in the manifest, per-workload pk/trace flags. Backend selection and tracing belong to the runner.

• Add small CTests and BallOps to Verilator manifests. Add BankOps to BEMU manifests. Add thread/chip tests to every backend that actually supports that chip; do not claim coverage through stale or non-existent artifacts.
• After a rename or move, verify every manifest file name has exactly one match in the freshly synchronized output.

Completion checks

1. Enforce the line limit and confirm one test per source.
2. Build the relevant CTest and MLIR aggregate targets with -O2.
3. Run the appropriate BEMU and Verilator regressions. Use BEMU trace when diagnosing or when the batch configuration enables it; trace is a backend setting, not a workload attribute.
4. Report the tests added, migrated, and deliberately removed, plus any unsupported geometry that now fails explicitly.