agentleFS
lasplasp/adamant-skills/adamant-testing/SKILL.md

adamant-testing

Skill4 starsChanged 3 months agoApache-2.0View on GitHub

Comprehensive testing patterns and infrastructure for Adamant embedded software framework components. Use when writing unit tests, creating test harnesses, using the History API, testing commands/parameters/faults/data products, or debugging test failures.

The file

Show its rules and text

We split this skill into the rules it gives an agent

29 rules. About 0 are common, 9 are distinctive, and 20 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈9

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 20

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Send_Dropped histories are ALWAYS populated: When Connector_*_Recv_Sync_Status is set to Message_Dropped, the tester's invokee handlers …

    line 12
  2. Very specific:

    Unused Self in Validate_Parameters: Use pragma Unreferenced (Self).

    line 14
  3. Very specific:

    NEVER write tester files from scratch: Run redo templates in both component/ and test/ dirs, then cp build/template/*.ads …

    line 15
  4. Very specific:

    Test dirs use env.py, NOT .all_path (causes duplicate conflicts)

    line 30
  5. Very specific:

    Test spec (*_tests-implementation.ads) MUST come from admt templates (wraps redo templates)

    line 32
  6. Distinctive:

    Every test must verify what happened, how many times, and with what values.

    line 65
  7. Distinctive:

    If the component emits it, the test must verify count AND value.

    line 100
  8. Distinctive:

    If a field value cannot be predicted (e.g., a buffer address), document why and check it separately.

    line 118
  9. Very specific:

    Every packed record type has a generated .Assertion child package (with My_Type.Assertion.

    line 122
  10. Very specific:

    NEVER use Natural_Assert for non-Natural values: Use the typed assertion package.

    line 126
  11. Very specific:

    NEVER use E'Pos to compare enums: Use the typed enum assertion: My_Enum_Assert.Eq(Status, Expected) not Natural_Assert.Eq(E'Pos(Status), …

    line 128
  12. Very specific:

    Do NOT call Smart_Assert.Eq(...) directly — use pre-instantiated packages from Basic_Assertions, *.Assertion child packages, or instantiate …

    line 130
  13. Distinctive:

    Verify every item before declaring tests complete:

    line 134
  14. Very specific:

    Adding tests: Add to tests.yaml -> admt templates -> copy ONLY the *_tests-implementation.ads -> add implementation in .adb -> admt test.

    line 155
  15. Very specific:

    Discarding status: When a function returns a status you don't need, use pragma Warnings (Off, "unused").

    line 203
  16. Distinctive:

    History buffer limits: Default tester history buffers hold ~100 entries.

    line 250
  17. Very specific:

    When a component has arrayed connectors (count: N in YAML):

    line 373
  18. Very specific:

    Index type visibility: Import Component.<Name>_Reciprocal in test body for the tester's index type: with Component.<Name>_Reciprocal.

    line 376
  19. Very specific:

    Connector_Count_Type arithmetic: If using arithmetic on indices, add with Connector_Types.

    line 377
  20. Very specific:

    Use Command_Enums.Command_Response_Status.E for response status (NOT Command_Execution_Status).

    line 399
  21. Very specific:

    Add with for every type referenced in tests: Basic_Assertions, Packed_F32.Assertion, Command_Enums, Interfaces, custom types from …

    line 482
  22. Distinctive:

    Speculative assertions: Only assert on behavior you can verify from the YAML model + implementation.

    line 488
  23. Distinctive:

    Typed histories don't auto-clear: when raw history cleared

    line 491
  24. Very specific:

    When tests need different init params, skip Init/Set_Up in Set_Up_Test (keep only Init_Base + Connect), call Component_Instance.Init + …

    line 514
  25. Very specific:

    Use Expect_*_Dropped tester flag (hand-edit tester).

    line 558
  26. Distinctive:

    After Update_Parameters succeeds, the component reads new values in its next processing handler.

    line 599
  27. Very specific:

    When component fields are not reachable via commands or parameters, add setter procedures to the hand-written tester.

    line 644
  28. Very specific:

    Test file calls: T.Set_Pressure (501.0).

    line 657
  29. Distinctive:

    Use for: simulated sensor values, internal state flags, cooldown counters -- anything the component stores privately that must be set to …

    line 659

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Adamant Testing Framework

Auto-generated reciprocal testers with history capture and white-box component access.

⚠️ Common Pitfalls (Read First)

  1. Send_Dropped histories are ALWAYS populated. When Connector_*_Recv_Sync_Status is set to Message_Dropped, the tester's invokee handlers still push to history. Do NOT assert History.Get_Count = 0 under Message_Dropped. The status only affects the invoker's return value.
  2. Tester Validate_Parameters returns Parameter_Update_Status.E (values: Success, Id_Error, Validation_Error), NOT Parameter_Validation_Status.E (values: Valid, Invalid). The framework translates: your component's Invalid -> tester's Validation_Error.
  3. Unused Self in Validate_Parameters: Use pragma Unreferenced (Self); -- do NOT copy Self (Ignore : constant Instance := Self;) as Instance is a limited type and copying violates Ravenscar.
  4. NEVER write tester files from scratch. Run redo templates in both component/ and test/ dirs, then cp build/template/*.ads build/template/*.adb . in the test dir.
  5. Do NOT add with Interfaces; or with Command; to test bodies. The generated tester already provides visibility to Interfaces and command-related packages through the base class hierarchy. Adding redundant with clauses causes -gnatwr style failures. Only with packages that the test body uniquely needs (e.g., with Packed_F32; for assertion helpers).

Test Structure

component_name/test/
├── component_name.tests.yaml                              # Test model (REQUIRED)
├── test.adb                                               # AUnit runner (from template)
├── env.py                                                 # Build path (REQUIRED, NOT auto-generated)
├── component-component_name-implementation-tester.ads/adb # Generated tester (from template)
└── component_name_tests-implementation.ads/adb            # Handwritten tests

CRITICAL rules:

  • Test dirs use env.py, NOT .all_path (causes duplicate conflicts)
  • File naming: {component_name}.tests.yaml, NOT bare tests.yaml
  • Test spec (*_tests-implementation.ads) MUST come from admt templates (wraps redo templates)
  • tests.yaml MUST be in test/ dir, NOT component dir

Setup: bash scripts/mk_test_env.sh [test_names...] (from component directory)

Workflow

⚠️ CRITICAL -- NEVER write tester .ads/.adb or test.adb from scratch. These files are ~500 lines of generated code with complex reciprocal connector wiring, History API instantiation, and AUnit scaffolding. ALWAYS generate them:

⚠️ ALL test build commands MUST run from inside the test/ directory. Running from the project root uses the Linux target (no AUnit). The test env.py sets TARGET=Linux_Test which includes AUnit, but this only takes effect when the build is invoked from within test/.

# From the test/ directory (MANDATORY -- do NOT run from project root):
cd test/
admt templates                              # Generates build/template/*, prompts to copy
# If you need to cherry-pick specific files instead of the default copy,
# answer the prompt with ``n`` and use the container form:
admt env exec "cd /home/user/<project>/src/components/<name>/test && cp build/template/component-*-tester.ads ."
admt env exec "cd /home/user/<project>/src/components/<name>/test && cp build/template/component-*-tester.adb ."
admt env exec "cd /home/user/<project>/src/components/<name>/test && cp build/template/*_tests-implementation.ads ."
admt env exec "cd /home/user/<project>/src/components/<name>/test && cp build/template/test.adb ."
# Then write ONLY: *_tests-implementation.adb (test case bodies)
admt test                                   # Build and run (wraps redo test)
admt coverage                               # Coverage analysis via gcov (wraps redo coverage)

⚠️ Quality gates — tests are NOT done until:

  1. redo test — all tests pass, zero failures, zero unexpected errors
  2. redo style — zero warnings. Unused imports, unreferenced variables, and style violations are build failures.
  3. redo coverage — 100% line coverage on the implementation .adb file. If lines appear unreachable, ask the user what they would like to do about them. Coverage of generated code and test infrastructure is not required.

Test Assertion Completeness

Every test must verify what happened, how many times, and with what values. This applies uniformly to all Adamant IDed entities (events, data products, faults, packets) and all output connectors.

IDed Entity Verification

For EVERY IDed entity type the component produces (events, data products, faults, packets):

  1. Total raw history count — verify the exact total. This catches unexpected extra emissions:
    Natural_Assert.Eq (T.Event_T_Recv_Sync_History.Get_Count, 3);
    Natural_Assert.Eq (T.Data_Product_T_Recv_Sync_History.Get_Count, 5);
  2. Per-type history count — verify each specific typed history:
    Natural_Assert.Eq (T.My_Event_History.Get_Count, 1);
  3. Per-type history values — verify the FULL record value for every entry via the type's .Assertion package. Always assert the complete record — never check individual fields:
    -- CORRECT: Full record assertion
    Parameter_Table_Id_Assert.Eq (T.My_Event_History.Get (1), (Id => 10));
    Packed_U32_Assert.Eq (T.My_Dp_History.Get (1), (Value => 42));
    
    -- WRONG: Field-level extraction with type conversion
    Natural_Assert.Eq (Natural (T.My_Event_History.Get (1).Id), 10);
    
    -- WRONG: Boolean comparison on individual fields
    Boolean_Assert.Eq (T.My_History.Get (1).Operation = Set, True);
    
    -- WRONG: pragma Assert on field
    pragma Assert (T.My_History.Get (1).Status = Success);
  4. Empty histories — verify histories that SHOULD be empty:
    Boolean_Assert.Eq (T.My_Event_History.Is_Empty, True);

If the component emits it, the test must verify count AND value. No exceptions.

Connector Output Verification

For EVERY output connector the component sends on, verify:

  1. Send count — exact number of sends via the connector's history.
  2. Send values — use the connector type's .Assertion package for full record assertion. Assert the complete record in aggregate, not individual fields:
    -- CORRECT: Full record assertion of connector output
    Parameters_Memory_Region_Assert.Eq (T.My_Connector_History.Get (1), (
       Region => (Address => Expected_Addr, Length => Expected_Len),
       Operation => Parameter_Enums.Parameter_Table_Operation_Type.Set
    ));
    
    -- WRONG: Field-by-field checks
    Boolean_Assert.Eq (T.My_Connector_History.Get (1).Operation = Set, True);

If a field value cannot be predicted (e.g., a buffer address), document why and check it separately. Otherwise, always assert the full record.

Assertion Rules

Every packed record type has a generated .Assertion child package (with My_Type.Assertion; use My_Type.Assertion; gives My_Type_Assert.Eq). Array types generate _U_Assert (unpacked) and _Assert (packed .T) suffixes.

NEVER use pragma Assert or AUnit Assert(condition, "message"). Smart_Assert prints expected vs actual on failure. pragma Assert only says "failed."

NEVER use Natural_Assert for non-Natural values. Use the typed assertion package. Natural_Assert is ONLY for counts (Get_Count, loop indices).

NEVER use E'Pos to compare enums. Use the typed enum assertion: My_Enum_Assert.Eq(Status, Expected) not Natural_Assert.Eq(E'Pos(Status), E'Pos(Expected)).

Do NOT call Smart_Assert.Eq(...) directly — use pre-instantiated packages from Basic_Assertions, *.Assertion child packages, or instantiate your own.

Pre-Submission Checklist

Verify every item before declaring tests complete:

  • Every IDed entity history (events, DPs, faults, packets): total raw count checked
  • Every IDed entity history: per-type count checked
  • Every IDed entity history: per-type VALUES checked via full record .Assertion package
  • Empty histories verified with Is_Empty
  • Every output connector history: send count checked
  • Every output connector history: send values checked via full record .Assertion package
  • No Natural(...) type conversions on typed history values
  • No pragma Assert or Boolean_Assert for record field comparisons
  • No individual field checks where full record assertion is possible
  • redo test — all pass, zero failures, zero unexpected errors
  • redo style — zero warnings
  • redo coverage — 100% on implementation .adb

The ONLY file you write from scratch is *_tests-implementation.adb (the test case bodies). Everything else is generated. You also need env.py and *.tests.yaml.

⚠️ Test body package name is <Component_Name>_Tests.Implementation (e.g., Signal_Processor_Tests.Implementation), NOT Component.<Name>.Implementation.Tester.Tests.Implementation. The file name is <component_name>_tests-implementation.adb. Match what admt templates generates in *_tests-implementation.ads.

Adding tests: Add to tests.yaml -> admt templates -> copy ONLY the *_tests-implementation.ads -> add implementation in .adb -> admt test.

⚠️ NEVER cp build/template/*.adb . -- this overwrites your handwritten implementation body. Only copy specific files: tester .ads/.adb, test spec .ads, test.adb. Never glob-copy .adb files from templates after writing implementation.

Test Body with Clauses

The test body (*_tests-implementation.adb) does NOT inherit visibility from the generated tester. You must explicitly with every package you use. Common pattern:

-- Required for most test bodies:
with Basic_Assertions; use Basic_Assertions;  -- Natural_Assert, Boolean_Assert
-- Only add `with Interfaces; use Interfaces;` if your test code directly uses
-- Unsigned_16, Unsigned_32, etc. Do NOT add it speculatively -- it causes
-- `-gnatwu` style warnings if unused. The generated tester and base class
-- already have Interfaces visibility for their own code.

-- For typed assertion packages (match your data product/event param types):
with Packed_U32.Assertion; use Packed_U32.Assertion;  -- Packed_U32_Assert
with Packed_U16.Assertion; use Packed_U16.Assertion;

-- For command response checking:
with Command_Enums; use Command_Enums;

-- For framework types used directly in test logic:
with Tick;                                -- qualify as Tick.T in sends
-- with Command;                          -- ONLY if you construct Command.T values directly
--                                        -- (e.g., invalid command test). T.Commands.* and
--                                        -- T.Command_T_Send do NOT require this import.
--                                        -- Adding it without use causes -gnatwu style warning.
with Sys_Time;                            -- for timestamps
with Data_Product;                        -- if inspecting raw DPs

-- For command response/status assertions:
with Command_Enums.Assertion; use Command_Enums.Assertion;
-- Gives: Command_Response_Status_Assert, Command_Execution_Status_Assert

-- NEVER with these directly (they are child instantiations, not standalone):
-- Natural_Assert              -> with Basic_Assertions; use Basic_Assertions;
-- Boolean_Assert              -> with Basic_Assertions; use Basic_Assertions;
-- Packed_U16_Assert           -> with Packed_U16.Assertion; use Packed_U16.Assertion;
-- Packed_U32_Assert           -> with Packed_U32.Assertion; use Packed_U32.Assertion;
-- Command_Response_Status_Assert -> with Command_Enums.Assertion; use Command_Enums.Assertion;
-- Command_Response_Status     -> use Command_Enums.Command_Response_Status
-- Command_Execution_Status    -> use Command_Enums.Command_Execution_Status
```ada

**Assertion methods**: `Eq`, `Neq`, `Lt`, `Le`, `Gt`, `Ge` -- all available on `Natural_Assert`, `Packed_U16_Assert`, etc. Use `Ge` for "at least N" checks (common for accumulated counts).

**Discarding status**: When a function returns a status you don't need, use `pragma Warnings (Off, "unused"); Ignore := Some_Function; pragma Warnings (On, "unused");` or the `Ignore renames` pattern.

**Package-level `use type`**: Can be placed at package body level (not just inside procedures) for operators needed across multiple test methods. Applies to ALL types whose operators you need -- framework types (e.g. `Command_Enums.Command_Response_Status.E`) AND custom preamble types from your own records (e.g. `My_Header.Version_Type`, `My_Header.Packet_Kind_Type`). Sub-byte enum/mod fields generate their own types in the record package.

**Name collisions**: `Tick` can collide with `Ada.Real_Time.Tick`. Always qualify: `Tick.T`, not just `T` when ambiguous. Add `use Tick;` if you reference `Tick.T` frequently.

## Test Model

```yaml
tests:
  - name: Test_Nominal
    description: Test nominal behavior
  - name: Test_Error_Paths

env.py (Minimum)

from environments import test  # noqa: F401
```ada

## Tester Architecture

Testers are **reciprocal components** with inverted connectors:
- Component sends -> tester receives (captures in histories)
- Component receives -> tester sends (provides stimuli)
- White-box access via `T.Component_Instance`

**⚠️ History naming follows the TESTER's perspective (inverted from component):**
- Component has `Event_T_Send` -> tester history is `T.Event_T_Recv_Sync_History`
- Component has `Data_Product_T_Send` -> tester history is `T.Data_Product_T_Recv_Sync_History`
- Component has `Command_Response_T_Send` -> tester history is `T.Command_Response_T_Recv_Sync_History`
- Component has `Fault_T_Send` -> tester history is `T.Fault_T_Recv_Sync_History`
- Typed histories use event/DP/fault names: `T.My_Event_History`, `T.My_Data_Product_History`

**Sending stimuli to component (tester sends what component receives):**
- `T.Tick_T_Send(...)` -- sends tick TO component
- `T.Command_T_Send(...)` -- sends command TO component
- `T.Commands.My_Command(...)` -- constructs command with correct ID (ALWAYS use this)
- `T.Parameters.My_Param(...)` -- constructs parameter with correct ID
- **Named recv_async connectors**: For `name: Foo_T_Recv_Async` in component YAML, the tester send method is `T.Foo_T_Send()` (strips `_Recv_Async`, adds `_Send`). NOT `T.Foo_T_Recv_Async_Send()`.
- **Named recv_sync connectors**: Same pattern. For `name: Foo_T_Recv_Sync` in component YAML, the tester send method is `T.Foo_T_Send(Arg)` (strips `_Recv_Sync`, adds `_Send`). For unnamed typed connectors (e.g., `type: Interfaces.Unsigned_16`), the tester method is `T.Interfaces_Unsigned_16_Send(Arg)` -- NO `_T_` infix for raw Ada types. The `_T_` infix only appears for framework `.T` types (e.g., `Tick.T` -> `Tick_T_Send`).

**NEVER use `T.Command_T_Send_History`** -- that doesn't exist. The tester SENDS commands (no history for sends). It RECEIVES responses (history for receives).

⚠️ **Set_Up DP history pollution (recurring #1 test error)**: If component overrides `Set_Up` and sends initial data products (e.g., zeroed counters), typed DP histories will have entries BEFORE any test stimuli. **Always clear ALL typed histories at the start of each test procedure** (after Set_Up_Test), or account for the offset in `Get` indices. Example: `T.Current_State_History.Clear; T.Fault_Count_History.Clear;` -- do this for EVERY typed history you will assert on in that test.

**History buffer limits**: Default tester history buffers hold ~100 entries. Tests that send many ticks (e.g., 50+ in a loop) can overflow them, causing lost entries and wrong assertion indices. Keep tick counts per test under 50, or clear histories between test phases with `T.Event_T_Recv_Sync_History.Clear` etc.

## Test Lifecycle

```ada
overriding procedure Set_Up_Test (Self : in out Instance) is
begin
   Self.Tester.Init_Base;                          -- (Queue_Size => N for active)
   Self.Tester.Connect;
   Self.Tester.Component_Instance.Init (Param => 42);  -- IFF YAML has init:
   Self.Tester.Component_Instance.Set_Up;
end Set_Up_Test;

overriding procedure Tear_Down_Test (Self : in out Instance) is
begin
   Self.Tester.Final_Base;
end Tear_Down_Test;

overriding procedure Test_Name (Self : in out Instance) is
   T : Component.Name.Implementation.Tester.Instance_Access renames Self.Tester;
begin
   T.Tick_T_Send ((Time => (0, 0), Count => 1));
   Natural_Assert.Eq (T.Event_T_Recv_Sync_History.Get_Count, 1);
end Test_Name;
```ada

**Order:** `Init_Base` → `Connect` → `Component_Instance.Init` → `Set_Up`. Init params go to `Component_Instance.Init`, NOT `Init_Base`. Both Init and Set_Up are optional -- only call them if YAML declares `init:` or component overrides Set_Up. **If the component YAML has NO `init:` section, do NOT call `Component_Instance.Init` or `Set_Up` -- they won't exist and won't compile.** For components without init, Set_Up_Test is just `Init_Base` + `Connect`. For per-test init params, defer Init/Set_Up to each test body (see [setup-variants.md](references/setup-variants.md) for deferred init patterns).

**Re-init mid-test (full):** To fully reset, call `Final_Base` then repeat: `Final_Base` → `Init_Base` → `Connect` → `Component_Instance.Init(new_params)` → `Set_Up`. This resets all histories and reconnects.

**Re-init mid-test (simple):** For passive components, you can often just call `T.Component_Instance.Init(new_params)` directly without the full teardown cycle. This changes init config without resetting histories. Used in real code (e.g., bot_station PID controller tests).

**⚠️ History Depth**: Generated `Init_Base` initializes all histories with `Depth => 100`. If a test sends more than 100 events/data products/commands, the history overflows and the test fails with "History is full." Solutions:
- **Increase depth in tester .adb Init_Base**: After copying templates from `build/template/`, edit the tester `.adb` file and change any `Depth => 100` to whatever you need (500, 1000, even 10000 -- there is no practical upper limit). The Init calls are in `Init_Base` in the `.adb`. Canonical example: `adamant_example/src/components/oscillator/test/component-oscillator-implementation-tester.adb` -- shows both raw connector histories and typed event histories with Depth parameters.
- Clear histories mid-test: `Self.Tester.Event_T_Recv_Sync_History.Clear;`
- Design tests to use fewer iterations (preferred -- keep tests small and focused)

## Sending Stimuli

```ada
T.Tick_T_Send ((Time => (0, 0), Count => 1));
T.Command_T_Send (T.Commands.My_Command ((Field => Value)));   -- with args
T.Command_T_Send (T.Commands.My_Noop_Command);                 -- no args

-- Parameters (3-step + processing trigger):
Status := T.Stage_Parameter (T.Parameters.Param_Name ((Value => 42.0)));
Parameter_Update_Status_Assert.Eq (Status, Parameter_Enums.Parameter_Update_Status.Success);
Status := T.Validate_Parameters;
Parameter_Update_Status_Assert.Eq (Status, Parameter_Enums.Parameter_Update_Status.Success);
Status := T.Update_Parameters;
Parameter_Update_Status_Assert.Eq (Status, Parameter_Enums.Parameter_Update_Status.Success);
-- Active: T.Tick_T_Send (The_Tick);  -- triggers Self.Update_Parameters in tick handler
-- Passive: T.My_Input_T_Send (Data);  -- triggers Self.Update_Parameters in recv handler

ALWAYS use T.Commands / T.Parameters -- never create local instances (wrong ID bases). ALWAYS read the generated tester .ads (build/template/component-*-tester.ads) to discover available helper functions. Do NOT guess the API -- T.Stage_Parameter, T.Validate_Parameters, T.Update_Parameters, T.Parameters.*, typed history accessors, etc. are all defined there. For custom packed types on connectors: construct unpacked (.U) then Pack: T.Cmd_T_Send (My_Type.Pack (unpacked_val));

History Verification

Dual-level capture: raw connector history + individual typed histories.

-- Raw (all events/DPs/faults)
Natural_Assert.Eq (T.Event_T_Recv_Sync_History.Get_Count, 3);
-- Typed (specific event/DP/fault)
Natural_Assert.Eq (T.My_Event_History.Get_Count, 1);
-- Access (1-indexed) -- typed histories store PACKED type (.T), not unpacked (.U)
Packed_U32_Assert.Eq (T.Counter_History.Get (1), (Value => 42));
-- For custom packed record types, use the typed assert with a named aggregate:
-- My_Status_Assert.Eq (T.My_Status_History.Get (1), ((Field_1 => X, Field_2 => Y)));
-- No Pack/Unpack needed -- Ada resolves the .T type from context.
-- Clear between phases
T.Event_T_Recv_Sync_History.Clear;
```ada

**Named connectors:** `name: Spi_Data` on send → `Spi_Data_Reciprocal_History` in tester (NOT `_T_Recv_Sync_History`).

**CRITICAL:** Clearing raw history does NOT clear typed histories. Use cumulative counts or clear typed histories explicitly: `T.My_Event_History.Clear;`

## Active Component Testing (Async Dispatch)

```ada
-- Send to async connectors (queued, not processed yet)
T.Async_Data_Send (data1);
T.Async_Data_Send (data2);
Natural_Assert.Eq (T.Output_History.Get_Count, 0);  -- Nothing yet

-- Dispatch processes the queue (FUNCTION returning Natural)
Natural_Assert.Eq (T.Dispatch_All, 2);  -- Must capture return value
Natural_Assert.Eq (T.Output_History.Get_Count, 2);

-- Queue overflow testing
-- Use small Queue_Size in Init_Base (e.g., 100 bytes) for overflow tests
-- CRITICAL: Expect_*_Dropped auto-resets to False after each drop
for I in 1 .. N loop
   T.Expect_Async_Data_Send_Dropped := True;  -- Must set before EVERY send
   T.Async_Data_Send (overflow_data);
end loop;
Natural_Assert.Gt (T.Async_Data_Send_Dropped_Count, 0);

⚠️ CRITICAL -- Variable-Length Types Must Be Initialized Before Queue Push: Types with variable_length: fields (Packet.T, Command.T, etc.) MUST have their length field explicitly set before sending to an async queue. Ada does NOT zero-initialize record fields. An uninitialized Buffer_Length causes Serialization_Failure or Too_Full from the queue push, triggering *_Send_Dropped.

-- WRONG: Buffer_Length is uninitialized garbage
Pkt : Packet.T;
Pkt.Header.Sequence_Count := 42;
T.Packet_T_Send (Pkt);  -- FAILS: queue push error

-- CORRECT: Always set Buffer_Length (and other header fields)
Pkt : Packet.T;
Pkt.Header.Time := (0, 0);
Pkt.Header.Id := 0;
Pkt.Header.Sequence_Count := 42;
Pkt.Header.Buffer_Length := 0;  -- REQUIRED for queue serialization
T.Packet_T_Send (Pkt);  -- Works

Arrayed Connector Testing

When a component has arrayed connectors (count: N in YAML):

  • Tester send (arrayed recv_async): T.Packed_F32_T_Send (Index, Arg) -- pass index as first arg
  • Tester send (arrayed recv_sync): Same pattern -- T.Packed_F32_T_Send (Index, Arg) (tester sends to component's recv_sync)
  • Index type visibility: Import Component.<Name>_Reciprocal in test body for the tester's index type: with Component.<Name>_Reciprocal; use Component.<Name>_Reciprocal;
  • Connector_Count_Type arithmetic: If using arithmetic on indices, add with Connector_Types; use type Connector_Types.Connector_Count_Type;
  • Dispatch_All does NOT invoke Cycle: For active components, Dispatch_All only dispatches queued async messages. If periodic logic is in Cycle, it is not exercised by Dispatch_All. Prefer adding a Tick.T recv_sync connector for testable periodic computation.

Command Response Verification

⚠️ Dispatch_All is ONLY for recv_async connectors. Commands on recv_sync connectors execute immediately when T.Command_T_Send is called -- do NOT call Dispatch_All for them (it will return 0). Only call Dispatch_All after sending to recv_async connectors (e.g., Packet_T_Send for async packet input). Check the component YAML to determine which connectors are sync vs async.

-- Command test pattern (commands are recv_sync -- execute immediately):
T.Command_T_Send (cmd);
-- Response is available immediately -- no Dispatch_All needed:
Natural_Assert.Eq (T.Command_Response_T_Recv_Sync_History.Get_Count, 1);
Command_Response_Assert.Eq (T.Command_Response_T_Recv_Sync_History.Get (1), (
   Source_Id => 0,
   Registration_Id => expected_reg_id,
   Command_Id => T.Commands.Get_Set_Value_Id,  -- Use ID getter
   Status => Success));
```ada

Use `Command_Enums.Command_Response_Status.E` for response status (NOT `Command_Execution_Status`).
Need `use type Command_Enums.Command_Response_Status.E;` for `=` operator visibility.

## Assertion Import Patterns

```ada
with Basic_Assertions; use Basic_Assertions;  -- Natural_Assert, Boolean_Assert
with Packed_U32.Assertion; use Packed_U32.Assertion;  -- Packed_U32_Assert
with Command_Enums.Assertion; use Command_Enums.Assertion;  -- Command_Response_Status_Assert
with Parameter_Enums.Assertion; use Parameter_Enums.Assertion;  -- Parameter_Update_Status_Assert
with My_Custom_Record.Assertion; use My_Custom_Record.Assertion;  -- My_Custom_Record_Assert

Construct expected records as aggregates — no Pack/Unpack needed:

My_Status_Assert.Eq (T.My_Status_History.Get (1), (Field_1 => Value_1, Field_2 => Literal));
My_Array_Assert.Eq (T.My_Array_History.Get (1), [Val_0, Val_1, Val_2]);

Data Dependency Testing

Setting Mock Values and Ticking

The tester generates named fields for each data dependency. Set them directly:

-- Use non-zero timestamps when testing data dependency STALENESS.
-- (0,0) is fine for non-staleness tests (commands, events, basic DPs).
Test_Time : constant Sys_Time.T := (100, 0);
The_Tick : constant Tick.T := (Time => Test_Time, Count => 0);

-- In Set_Up_Test:
T.System_Time := Test_Time;
T.Data_Dependency_Timestamp_Override := Test_Time;

-- Set mock values (field names match data_dependencies.yaml names)
T.Sensor_Value := (Value => 50.0);
T.Tick_T_Send (The_Tick);
-- Component calls Self.Get_Sensor_Value(...) which reads T.Sensor_Value

Source: threshold_monitor. Field names (Sensor_Value, Setpoint, etc.) match the names in data_dependencies.yaml. The tester auto-generates these fields.

Multiple Data Dependencies

T.Setpoint := (Value => 10.0);          -- Packed_F32.T
T.Process_Value := (Value => 7.0);      -- Packed_F32.T
T.Tick_T_Send (The_Tick);
-- error = 10.0 - 7.0 = 3.0, output = Kp * 3.0 = 3.0
Packed_F32_Assert.Eq (T.Output_History.Get (1), (Value => 3.0));

Source: pid_controller. Set all dependency fields before ticking.

Simulating Fetch Failure

T.Data_Dependency_Return_Status_Override := Data_Product_Enums.Fetch_Status.Id_Out_Of_Range;
T.Tick_T_Send (The_Tick);
-- Component's error path fires a fault, no DPs sent
Natural_Assert.Eq (T.Sensor_Failure_History.Get_Count, 1);
Natural_Assert.Eq (T.Data_Product_T_Recv_Sync_History.Get_Count, 0);

Source: threshold_monitor. Override the return status to test the component's error handling for missing/failed data dependencies.

Simulating Stale Data

T.Data_Dependency_Timestamp_Override := (50, 0);  -- Old timestamp
T.Tick_T_Send ((Time => (100, 0), Count => 0));   -- Current time is newer

The framework detects that the data product timestamp is older than expected and reports staleness to the component.

Data_Dependency_Timestamp_Override is GLOBAL -- it applies the same timestamp to ALL data dependency fetches. You cannot selectively make one dep stale while keeping another fresh via this field alone. To test per-dependency staleness, set individual mock values with different staleness characteristics and rely on the component's internal logic to differentiate, or test each dependency in isolation.

Implementation needs: with Data_Product_Enums; use Data_Product_Enums; use Data_Product_Enums.Data_Dependency_Status; in the component body for status checks.

Side-effect events from Invalid_Data_Dependency: When a Get_* call returns a non-Success status, the framework automatically calls the component's Invalid_Data_Dependency override DURING the Get call (before control returns to the caller). If that override sends events, those events appear in the history BEFORE any events the caller sends afterward. Account for these extra events in assertion counts -- e.g., if testing staleness and the override sends a Sensor_Stale event, that event fires inside Get, not after it.

Test Body With-Clauses

Add with for every type referenced in tests: Basic_Assertions, Packed_F32.Assertion, Command_Enums, Interfaces, custom types from src/types/. Only with what you use -- admt style flags unused imports.

Common Errors

Top errors that waste time:

  1. Speculative assertions: Only assert on behavior you can verify from the YAML model + implementation. If you don't know whether a tick sends an event, don't assert on event counts. Write minimal, correct tests first -- you can always add assertions after observing actual behavior.
  2. Wrong stimulus API: T.*_T_Send to stimulate, NOT T.*_T_Recv_Sync
  3. Init params → Component_Instance.Init, NOT Init_Base
  4. Typed histories don't auto-clear when raw history cleared
  5. Dispatch_All is a FUNCTION returning Natural (active components only)
  6. History depth is 100 -- clear mid-test for heavy DP producers
  7. No Packed_U8 -- use Packed_Byte.T
  8. Tick.T.Count is Unsigned_32 -- cast loop vars
  9. Named Event.T send connectors crash on non-component event IDs
  10. Array aggregates use [], record aggregates use ()
  11. Copy tester files from build/template/, never hand-write

Invalid Command Testing

Invalid_Cmd : Command.T := T.Commands.Set_Value ((Value => 0));
Invalid_Cmd.Header.Arg_Buffer_Length := 0;  -- Wrong length
T.Command_T_Send (Invalid_Cmd);
-- Verify Length_Error response
Command_Response_Assert.Eq (T.Command_Response_T_Recv_Sync_History.Get (1), (
   Source_Id => 0, Registration_Id => expected_reg_id,
   Command_Id => T.Commands.Get_Set_Value_Id, Status => Length_Error));

Deferred Init (Per-Test Initialization)

When tests need different init params, skip Init/Set_Up in Set_Up_Test (keep only Init_Base + Connect), call Component_Instance.Init + Set_Up in each test body.

Multiple Command Connectors

Named command connectors generate _Send, _Send_2, etc.:

T.Command_T_Send (cmd);     -- First command connector
T.Command_T_Send_2 (cmd2);  -- Second command connector (named in YAML)

Error Injection

-- For any send connector, tester provides Expect_*_Dropped
T.Expect_Data_Product_T_Send_Dropped := True;
-- Component tries to send → tester captures drop instead of asserting failure
T.Expect_Data_Product_T_Send_Dropped := False;  -- Reset

Send_Dropped Testing

Send_Dropped fires when connector IS attached but receiver returns Message_Dropped -- NOT when unattached (Send_If_Connected skips entirely).

⚠️ CRITICAL -- Tester histories are ALWAYS populated: When Message_Dropped status is set, the tester's invokee handlers (Event_T_Recv_Sync, Data_Product_T_Recv_Sync, etc.) are still called and push to history. The Message_Dropped status only affects what the invoker (component under test) sees as a return value, which triggers the *_Send_Dropped handler. Do NOT assert History.Get_Count = 0 -- the tester receives the data regardless of the status value.

Sync sends (most common)

-- Set tester connector status to trigger Send_Dropped:
T.Connector_Event_T_Recv_Sync_Status := Connector_Types.Message_Dropped;
T.Connector_Data_Product_T_Recv_Sync_Status := Connector_Types.Message_Dropped;
T.Packed_F32_T_Send ((Value => 80.0));  -- Triggers component sends
-- Tester histories ARE populated (invokee always receives):
Natural_Assert.Eq (T.Event_T_Recv_Sync_History.Get_Count, 1);  -- NOT 0
-- The test verifies the *_Send_Dropped handler (null) doesn't crash.
-- Restore:
T.Connector_Event_T_Recv_Sync_Status := Connector_Types.Success;
T.Connector_Data_Product_T_Recv_Sync_Status := Connector_Types.Success;

Field names: Connector_<Type>_Recv_Sync_Status in generated reciprocal .ads.

Async sends

Use Expect_*_Dropped tester flag (hand-edit tester). See framework command_router tests.

Array Event.T connector tester crash

⚠️ When a component forwards Event.T through an array send connector (e.g., count: 4) AND has a separate Event.T send for component-generated events, the generated tester's Event_T_Recv_Sync handler calls Dispatch_Event on ALL received events. Forwarded events have IDs that don't match local event IDs, causing CONSTRAINT_ERROR (range check failure). Fix: After copying tester template, remove Self.Dispatch_Event(Arg) from the Event_T_Recv_Sync handler that receives forwarded (non-local) events.

Parameter Testing

Full Stage-Validate-Update Cycle

The complete parameter update flow requires three steps plus a tick:

Status := T.Stage_Parameter (T.Parameters.Output_Limit ((Value => 5.0)));
Parameter_Update_Status_Assert.Eq (Status, Parameter_Enums.Parameter_Update_Status.Success);
Status := T.Validate_Parameters;
Parameter_Update_Status_Assert.Eq (Status, Parameter_Enums.Parameter_Update_Status.Success);
Status := T.Update_Parameters;
Parameter_Update_Status_Assert.Eq (Status, Parameter_Enums.Parameter_Update_Status.Success);
T.Tick_T_Send (The_Tick);  -- New parameter values take effect

Declare Status as Parameter_Enums.Parameter_Update_Status.E. Requires:

with Parameter_Enums;
with Parameter_Enums.Assertion; use Parameter_Enums.Assertion;

Validation Rejection Testing

Status := T.Stage_Parameter (T.Parameters.Warning_Threshold ((Value => 95.0)));
Parameter_Update_Status_Assert.Eq (Status, Parameter_Enums.Parameter_Update_Status.Success);
Status := T.Validate_Parameters;
Parameter_Update_Status_Assert.Eq (Status, Parameter_Enums.Parameter_Update_Status.Validation_Error);

Source: threshold_monitor. Stage succeeds (just buffers), but Validate rejects because the component's Validate_Parameters override checks cross-parameter constraints (warning >= critical is invalid).

Parameter Takes Effect on Next Processing Call

After Update_Parameters succeeds, the component reads new values in its next processing handler. For active/ticked components, send a tick. For passive components (no tick -- only recv_sync handlers), send the next data input instead:

-- Active component: tick applies parameters
Status := T.Update_Parameters;
T.Tick_T_Send (The_Tick);
Packed_F32_Assert.Eq (T.Output_History.Get (1), (Value => 5.0));

-- Passive component: next recv_sync input applies parameters
Status := T.Update_Parameters;
T.My_Input_T_Send (Next_Input);  -- component calls Self.Update_Parameters internally
-- Now assert new behavior with updated parameters

Update_Parameters_Action Coverage

The tester's 3-step Stage/Validate/Update protocol does NOT call Update_Parameters_Action. The tester's Update_Parameters sends a Parameter_Update.T with Operation => Update via the modify connector, which calls Process_Parameter_Update. This sets Ready_To_Update but does NOT call Update_Parameters_Action.

To trigger Update_Parameters_Action, you must separately call Update_Parameters on the component instance. Since this is a non-overriding procedure on Base_Instance (not visible through the private type), add a tester helper:

-- In tester .ads (before end):
procedure Call_Update_Parameters (Self : in out Instance);

-- In tester .adb (before end):
procedure Call_Update_Parameters (Self : in out Instance) is
begin
   Self.Component_Instance.Update_Parameters;
end Call_Update_Parameters;

Then in the test body, after the 3-step protocol:

Stat := T.Stage_Parameter (T.Parameters.Threshold ((Value => 50)));
Stat := T.Validate_Parameters;
Stat := T.Update_Parameters;
T.Call_Update_Parameters;  -- Actually applies staged values

This pattern applies to ALL components with parameters.yaml and custom Update_Parameters_Action.

Tester Helpers for Private State

When component fields are not reachable via commands or parameters, add setter procedures to the hand-written tester. The tester IS a child of Component.X.Implementation, so it can access private record fields:

-- In tester .ads (before end):
procedure Set_Pressure (Self : in out Instance; Value : in Short_Float);

-- In tester .adb (before end):
procedure Set_Pressure (Self : in out Instance; Value : in Short_Float) is
begin
   Self.Component_Instance.Pressure := Value;
end Set_Pressure;
```ada

Test file calls: `T.Set_Pressure (501.0);`

Use for: simulated sensor values, internal state flags, cooldown counters -- anything the component stores privately that must be set to exercise specific branches.

## Coverage Patterns

```bash
admt coverage    # From the component's test/ directory (wraps redo coverage)

Common patterns for reaching 100%:

  • Invalid_Command: Corrupt Cmd.Header.Arg_Buffer_Length := 22; after constructing via T.Commands.*
  • Send_Dropped (sync): T.Connector_*_Recv_Sync_Status := Connector_Types.Message_Dropped;
  • Recv_Async_Dropped: Overflow the queue by sending more messages than capacity
  • Untested branches: Map Missing line numbers to source with cat -n

Full coverage workflow: coverage-guide.md

References

Load selectively. The main SKILL.md covers most testing patterns. Only load references when you need them.

ReferenceWhen to load
references/setup-variants.mdComponents with init parameters, custom Set_Up, or Tear_Down. Skip for simple components with no init.
references/assertion-patterns.mdComplex assertion chains, packed type field comparisons, or when tests need extended History API patterns. Skip for basic tests (count + simple value checks).
references/command-test-patterns.mdOnly when testing components with commands. Skip for passive components without commands.
references/adamant-example-patterns.mdOnly for task-based testing, memory regions, or advanced packet construction. Rarely needed.
references/coverage-guide.mdOnly when measuring or improving test coverage (admt coverage).

Related Skills

  • Component dev: adamant-component-dev
  • Assembly: adamant-assembly-dev
  • Style: adamant-style -- test code must pass admt style
  • Type system: adamant-type-system -- packed type assertions and comparisons

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.