agentleFS
vchelaruvchelaru/Gum/.claude/skills/gum-unit-tests/SKILL.md

gum-unit-tests

Skill659 starsChanged 2 months agoMITView on GitHub

Writing unit tests in the Gum repo. Triggers: tests in Gum.ProjectServices.Tests, Gum.Cli.Tests, or any other Gum test project.

The file

Show its rules and text

We split this skill into the rules it gives an agent

13 rules. About 0 are common, 1 is distinctive, and 12 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈1

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 12

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    When in doubt, put tests in MonoGameGum.Tests/: Only use V2/V3 projects for tests that exercise visual-version-specific behavior.

    line 25
  2. Very specific:

    Always use Shouldly — never xUnit Assert.

    line 35
  3. Very specific:

    If the assertion compares against ToolsUtilities.FilePath.FullPath, a bare leading-slash literal still isn't safe: route the literal …

    line 39
  4. Distinctive:

    Use named parameters for boolean literals.

    line 41
  5. Very specific:

    Before pushing, run only the test classes you added or changed (pwsh Tools/verify.ps1 does this from the diff).

    line 42
  6. Very specific:

    Never use the full suite as an iteration loop.

    line 44
  7. Very specific:

    Don't name a test namespace after an existing Gum type (e.g.

    line 47
  8. Very specific:

    Never dispose the shared graphics device in a test, and never block without a time limit on a call onto the headless UI thread.

    line 61
  9. Very specific:

    Read pixels only through HeadlessWindowDriver (PixelAt, PixelsAlong, SaveFrame), never window.CaptureRenderedFrame() directly: a single …

    line 77
  10. Very specific:

    Keep [AvaloniaFact] tests synchronous.

    line 86
  11. Very specific:

    Read BaseTestClass before adding setup — it handles singleton init, a ready-made GumProjectSave, and Dispose cleanup.

    line 111
  12. Very specific:

    Every StateSave must have ParentContainer set — GetValueRecursive traverses via that field and silently misbehaves or throws when it is …

    line 113
  13. Very specific:

    Keep the MEF batch an explicit mirror of LoadPlugins (not a catch-all export provider) so a plugin gaining an unbridged …

    line 146

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Gum Unit Test Reference

Test Projects

ProjectLocationWhat it tests
MonoGameGum.TestsMonoGameGum.Tests/Default project for new tests. MonoGame runtime, Forms controls, rendering, localization, data types — anything not specific to V3 visuals or integration
Gum.ProjectServices.TestsTests/Gum.ProjectServices.Tests/Headless services: error checking, codegen, font generation, project loading, save byte-parity (ProjectSaveParityTests over ParityCorpus/)
Gum.Presentation.TestsTests/Gum.Presentation.Tests/The tool's headless logic in Tools/Gum.Presentation (commands, view models, managers, plugin host)
Gum.Avalonia.TestsTests/Gum.Avalonia.Tests/The Avalonia tool head: composition, shell, plugin host, canvas host, tree control, theme resources, and an unattended run of the real executable
Gum.Cli.TestsTests/Gum.Cli.Tests/CLI command exit codes and output
Gum.Bundle.TestsTests/Gum.Bundle.Tests/GumCommon/Bundle/: GumProjectDependencyWalker, FontReferenceCollector, .gumpkg format and file providers. TestProjectBuilder builds the in-memory projects and temp-directory fixtures
MonoGameGum.Tests.V3Tests/MonoGameGum.Tests.V3/Tests specific to V3 default visuals
MonoGameGum.IntegrationTestsTests/MonoGameGum.IntegrationTests/Requires a real GraphicsDevice: content loading, renderer teardown, full GumService lifecycle
RaylibGum.TestsTests/RaylibGum.Tests/raylib runtime (incl. the #if RAYLIB branches of the source-shared GueDeriving/*Runtime.cs)
SkiaGum.TestsTests/SkiaGum.Tests/Skia runtime and shape runtimes
Gum.NetStandard.TestsTests/Gum.NetStandard.Tests/The netstandard2.1 builds of GumCommon and SkiaGum, run on .NET 10. Tests of netstandard2.1-only branches (Unity) go here; the csproj comment explains how it loads that build
Gum.ProjectServices.SkiaGum.TestsTests/Gum.ProjectServices.SkiaGum.Tests/SkiaGum-backed SVG export (SkiaGumSvgExportService, the service behind gumcli svg / tool File ▸ Export); drives SKSvgCanvas headlessly

When in doubt, put tests in MonoGameGum.Tests/. Only use V2/V3 projects for tests that exercise visual-version-specific behavior.

Tool logic tests go in Gum.Presentation.Tests and head tests in Gum.Avalonia.Tests. Tool/Tests/GumToolUnitTests is the frozen WPF head's suite (controls, WPF plugins, the WPF editor canvas): only a test that must construct a WPF control belongs there, and no new tool logic test does.

SkiaGum.Tests runs in CI as a blocking suite (#3233) — it renders into an in-memory CPU raster SKSurface, so it is fully headless despite the name. A red Skia test now fails the job like any other Bucket-A suite.

RaylibGum.Tests runs in CI as a blocking Windows suite (#3250). raylib's InitWindow needs an OpenGL 3.3 context the GPU-less runners lack; the Windows job supplies it via Mesa's llvmpipe software GL (dropped in next to the test binaries before the suite runs), so the tests run headless and a red raylib test now fails the job like any other Bucket-A suite. The same project also runs locally on macOS (#6987): GLFW/Cocoa creates a window only on the process main thread, so it is xunit v3 with the linked MonoGameGum.IntegrationTests/MainThreadTestHost.cs (its own Main), and the window is opened by the assembly fixture TestAssemblyInitialize, not a framework constructor, because that is what runs on the main thread. Before that move a test host on macOS hung forever in InitWindow; keep new raylib tests off Task.Run/other threads that touch raylib. A green CI run now does cover raylib — including the #if RAYLIB branches of a source-shared GueDeriving/*Runtime.cs — so the old mandatory local pre-merge run is no longer required; still update any assertion that pins old behavior when you change raylib-covered code. (Issue #3234: #3183 changed the raylib stroke-width PreRender but left the suite asserting the pre-#3183 value; back then CI didn't run raylib, so it shipped red — now it would be caught.)

Key Rules

  • Always use Shouldly — never xUnit Assert.
  • Disable parallel execution in every test project ([assembly: CollectionBehavior(DisableTestParallelization = true)]) — Gum uses global singletons.
  • A test asserting on an absolute path must not use a Windows-style "C:\..." literal — Path.IsPathRooted doesn't recognize a drive letter as rooted on Unix, so macOS/Linux CI treats it as relative and silently prepends the runner's real working directory, corrupting the path. Use a leading-slash literal (e.g. "/game/Content/") instead — rooted on both platforms.
    • A test that creates files under a temp directory has the mirror-image trap: \ is a legal file name character on macOS/Linux, so Path.Combine(root, "Folder\\File.cs") makes one oddly named file in root rather than a nested one — green on Windows, red on CI. Write the relative path with / and Replace('/', Path.DirectorySeparatorChar) it.
    • If the assertion compares against ToolsUtilities.FilePath.FullPath, a bare leading-slash literal still isn't safe: route the literal through new FilePath(...).FullPath on both sides of the comparison instead. See the gum-file-paths skill for why, and for the other FilePath comparison traps.
  • MessageDialogStyle.YesNo is a static property returning a new instance per get, so a Moq setup matching it by value never matches. The unmatched call returns the default MessageDialogResult (0, negative), so the code under test takes the user-declined branch and the test fails somewhere unrelated. Match on the dialog title or It.IsAny<MessageDialogStyle?>().
  • Use named parameters for boolean literals.
  • Before pushing, run only the test classes you added or changed (pwsh Tools/verify.ps1 does this from the diff); CI runs every suite on every push. A full local run is not a routine step.
    • dotnet test <csproj> --filter X rebuilds the whole project graph each time. When iterating, use pwsh Tools/fast-test.ps1 ClassA,ClassB, which builds once and runs --no-build for exactly the named classes.
    • Never use the full suite as an iteration loop. On a failure that only shows up with other classes, trace the mechanism once (in Gum.Avalonia.Tests, PluginFailureGuardAttribute names the test that disabled a plugin) instead of rerunning the suite after each fix.
    • Shared singletons still make tests order-dependent: a static a test leaves set (e.g. ObjectFinder.Self.GumProjectSave) breaks whichever class happens to run after it, and only in some orders. BaseTestClass resets the known ones on the way in and out; a class that doesn't derive from it must clear every static it sets in its own Dispose. Architecture/StaticStateBaseClassGuardTests (in both Gum.Presentation.Tests and MonoGameGum.Tests) fails for a test class whose file names ObjectFinder or StandardElementsManager and that does not derive from BaseTestClass. If CI fails on an unrelated class after your push, that cleanup is the first suspect. The weekly shuffled-test-order-weekly.yaml workflow runs Gum.Presentation.Tests, MonoGameGum.Tests and MonoGameGum.IntegrationTests in random class and method order; setting GUM_TEST_SHUFFLE_SEED to a failed run's seed replays that order locally (ShuffledTestCollectionOrderer, ShuffledTestCaseOrderer; the xunit v3 integration project uses Tests/TestOrdering/Xunit3 plus the class shuffle in MainThreadTestFrameworkExecutor). A class-level [TestCaseOrderer] would override the shuffle for that class, so don't add one.
  • A large differential or invariant sweep takes its case count from SweepScale.Cases(full) for random seeds or SweepScale.Includes(index) for an exhaustive enumeration (MonoGameGum.Tests/SweepScale.cs): a tenth locally, all of them on GitHub Actions or with GUM_FULL_SWEEPS=1. Golden-file comparisons keep their full count.
  • Don't name a test namespace after an existing Gum type (e.g. MonoGameGum.Tests.Binding collides with Gum.Forms.Data.Binding) — an unrelated file elsewhere in the same test project that references the type unqualified can suddenly fail to compile (CS0118: '...' is a namespace but is used like a type).
  • GumService.InitializeForTesting skips the runtime-type registrations a real app gets from SystemManagers, so a file-loaded Text is a plain GraphicalUiElement that raises no PropertyChanged, and a TextBox built on it never sees its text change. A test that loads elements from files calls StandardRuntimeRegistrations.Register() (MonoGameGum.Tests) first.

Avalonia head tests (Gum.Avalonia.Tests)

  • Anything that creates an Avalonia object (controls, ResourceDictionary, geometry) must be [AvaloniaFact], which runs on the headless UI thread; a plain [Fact] throws "Call from invalid thread".
  • Drive input with the Avalonia.Headless window helpers (MouseDown/MouseUp/KeyPress with a PhysicalKey) and call Dispatcher.UIThread.RunJobs() before asserting on anything the control updates on a later dispatcher pass.
  • Tests that compose plugins need the container registered with Locator (see HeadTestServices). A hung test host blocks the next run until it is killed; run with --blame-hang --blame-hang-timeout 120s.
  • Never dispose the shared graphics device in a test, and never block without a time limit on a call onto the headless UI thread; run device code through DeviceTestThread.Run. Either mistake hangs the whole test run.
  • MainWindow is a container singleton that HeadCompositionTests shows and closes. A test that needs it must not Show() it again (a closed window cannot be re-shown) and must not build a second one through ActivatorUtilities (it re-parents the singleton plugin tab controls and breaks unrelated tests). Read its state through window.Content without showing it.
  • HeadProcessTests launches the built head (Tool/Gum.Avalonia/bin/<Config>/net10.0) on a copied fixture; it skips without a display and on CI.
  • To drive a whole plugin tab with real input, build on the shared pieces in Harness/README.md: ToolProjectFixture (temp project, scripted dialogs, editor tab plugin kept out) and HeadlessWindowDriver (input, pixel reads). Animations/AnimationEditorHarness and VariableGrid/VariableGridHarness are the examples. A plugin added by hand must go in both PluginManager.Plugins and PluginManager.PluginContainers (events are dispatched only to plugins with a container). PluginFailureGuardAttribute fails the test that disabled a guarded plugin and re-enables it afterwards, so every test starts with the same plugin set.
  • A list's own arrow-key navigation runs before a bubbling KeyDown handler and marks the event handled whatever the modifiers, so an Alt+arrow hotkey on a ListBox needs AddHandler(KeyDownEvent, ..., RoutingStrategies.Tunnel).
  • Read pixels only through HeadlessWindowDriver (PixelAt, PixelsAlong, SaveFrame), never window.CaptureRenderedFrame() directly: a single render tick can return the previous frame.
  • Gum.Avalonia.Tests runs one Avalonia app for the whole assembly (AvaloniaTestIsolation(PerAssembly) in TestAppBuilder.cs); keep it. A test that changes app-level state (Application.Current.Resources, theme, accent) restores it. A window a test builds itself needs render tick, RunJobs(), render tick before it hit-tests anything; HeadlessWindowDriver does this. A window that hit-tests nothing is a regression, not a flake. Details: Tests/Gum.Avalonia.Tests/Animations/README.md. Never add thread-pool work that reaches into Avalonia (timers, continuations) to a test or a plugin's StartUp.
  • Keep [AvaloniaFact] tests synchronous. An async Task one needs a nested dispatcher frame, and the headless session sometimes throws PlatformNotSupportedException from PushFrame. Playback-style timing goes through an injected TimeProvider plus a manual timer the test advances, never a wall-clock sleep loop (TestAnimationPlugin.Clock/Timers, AnimationEditorHarness.Wait).
  • HeadTestServices replaces IFontFileGenerator with NoOpFontFileGenerator, so no Avalonia test writes .fnt/.png files; a test that needs real fonts must register the real generator and await it.

Scripted Forms input (Gum.Automation)

Tests/Gum.Automation drives Forms in-process without the OS mouse or keyboard: GumDriver.ForGumService(GumService.Default) then Install() gives click/drag/scroll/type/key/read by element name, and GumScriptRunner.Run(driver, ndjson) plays an NDJSON script with per-line pass/fail (commands are documented on GumScriptCommand). Use it instead of a hand-written ICursor/keyboard fake. A desktop app plays a script off-screen via GumAutomationHost (Samples/MonoGameGumInCode, launch command in its Automation/smoke.ndjson).

Save parity corpus

ParityCorpus/ holds whole project folders that must re-save byte for byte in every culture and on every OS. A red parity test is a regression; only an intended format change regenerates the baselines (GUM_UPDATE_PARITY_BASELINES=1, then review the diff). See ParityCorpus/README.md.

Test at production defaults

When a feature's tests disable a production default for isolation (e.g. LoaderManager.Self.CacheTextures = false), remember that default is on in real apps — so any code path that only runs with it on is left untested. Treat "this test turns a production default off" as a smell: keep at least one test that exercises the path at the production default. (A raylib font regression survived review because every font test ran with caching off, which made a new cache-hit branch dead code.)

Headless Tests (ProjectServices, MonoGameGum.Tests.V3)

Read BaseTestClass before adding setup — it handles singleton init, a ready-made GumProjectSave, and Dispose cleanup. Don't repeat that in subclasses.

Every StateSave must have ParentContainer set — GetValueRecursive traverses via that field and silently misbehaves or throws when it is null. Use ScreenSave for standalone state tests (no base type, no StandardElementsManager fallback).

InternalsVisibleTo is set up in Gum.ProjectServices.csproj for Gum.ProjectServices.Tests — internal members are directly accessible.

Headless Forms allocation tests: install a real Cursor

BaseTestClass installs a Moq mock as FrameworkElement.MainCursor, and each proxied member access allocates (~296 B). Any control that reads MainCursor (e.g. a ScrollBar value setter) then pollutes an AllocationMeasurer result with a pure test artifact. Assign a real Gum.Input.Cursor(null) before measuring — production always uses a real cursor. See ListBoxScrollAllocationTests.

Skia allocation guards (Tests/SkiaGum.Tests/Performance)

SkiaSceneHost wires an in-memory surface to GumService.Default; DrawAllocationTests and LayoutAllocationTests measure Update + Draw with AllocationMeasurer (managed bytes only, not native Skia memory). LayoutAllocationTests and DrawAllocationTests also exist in the MonoGame projects, so fast-test.ps1 needs SkiaGum.Tests:<Class>.

To find what allocates, enable an in-process EventListener on Microsoft-Windows-DotNETRuntime (keyword 0x1) and count TypeName on GCAllocationTick events over about 40,000 frames. That names the type directly, which reading the code did not.

A property setter that clears a cache unconditionally allocates every frame once a runtime wrapper re-pushes its values each frame (SkiaShapeRuntime.ApplyDropshadow, the TextRuntime font bridge in SystemManagers); guard it with != value. RichTextKit's TextBlock.Paint allocates on every call, so Text replays a recorded SKPicture while its block is unchanged.

SKImageFilter, SKShader and SKColorFilter cannot change after creation, so an animated dropshadow, gradient or sprite tint allocates one wrapper (104 bytes) per frame (UpdateAndDraw_AnimatedProperty_AllocationBaseline). Arrays passed to their factories can be kept on the renderable and reused, because Skia copies them (GradientGoldenImageTests).

WPF-touching tool code (GumToolUnitTests, frozen WPF head) needs an STA thread

xUnit's runner is MTA, but WPF FrameworkElements (MenuItem, Menu, ComboBox, …) throw InvalidOperationException: The calling thread must be STA when constructed. If a tool class news up a WPF control — often a ViewModel building right-click MenuItems in its constructor, or a plugin's StartUp() — mark the test [StaFact] (Xunit.StaFact), which runs it on an ApartmentState.STA thread. See MenuStripManagerTests.

Verify the real construction blocker empirically before designing around an assumed one. A quick throwaway probe (construct the object, see what actually throws) beats reasoning: e.g. BitmapFrame PNG decode and most non-control VM constructors run fine on MTA, so the blocker is usually the WPF control, not the singleton/resource you suspected.

[StaFact] alone isn't enough for a control that pulls StaticResources from an App-level merged ResourceDictionary — those only exist inside the real running Application, so construction throws XamlParseException ("Cannot find resource named '...'") even under STA. Don't construct the real control to test a plugin's event-driven show/hide logic; extract that logic into a small class taking ISelectedState/IPluginTab via the constructor (mirrors VariableGridSelectionCoordinator) and test it without touching the control.

Plugin/DI composition tests (GumToolUnitTests for the WPF head, PluginHostTests for the Avalonia head)

AllPluginsCompositionTests composes every tool plugin through MEF the way PluginManager.LoadPlugins does, and ServiceProviderCompositionSpikeTests resolves the bridged services from the real Builder.cs container. Two reusable techniques live there:

  • Stub anything headlessly without running its constructor. RuntimeHelpers.GetUninitializedObject(type) fabricates a concrete instance (even a heavy WinForms/WPF host singleton) with no ctor call; a Moq proxy covers interfaces/abstract types. Composition/DI only needs the dependency to exist as the right type, so this avoids STA/graphics setup entirely. (Run the composition on RunOnSta regardless — some plugin constructors still touch WPF.)
  • Satisfy not-yet-drained plugins' direct Locator.GetRequiredService<T>() ctor calls with a catch-all IServiceProvider registered via Locator.Register(...); remove it in Dispose (Locator has no Unregister — RenameManagerTests shows the reflection teardown).

Keep the MEF batch an explicit mirror of LoadPlugins (not a catch-all export provider) so a plugin gaining an unbridged [ImportingConstructor] dependency turns the test red — that regression signal is the whole point. See the gum-tool-plugins skill for keeping PluginBridgedServiceTypes.All in sync during drains.

Golden-image pixel-diff tests (SkiaGum.Tests)

Tests/SkiaGum.Tests/GoldenImages/ covers rendering behavior that has no Style/paint-parameter to assert on (e.g. geometric per-glyph transforms) — the rest of SkiaGum.Tests' visual tests assert on paint/style objects instead of pixels. PixelComparer is a pure per-pixel/per-channel diff with tolerance (unit-tested in-memory, no files); GoldenImageAssert.Matches(surface, name) loads a checked-in baseline PNG from GoldenImages/Baselines/<name>.png and diffs it against a rendered SKSurface.

Baselines are approved snapshots, not derived from spec — same convention as Jest's --updateSnapshot. If the baseline is missing or the render regresses, the assertion fails and writes the actual render to GoldenImages/Actual/<name>.actual.png; review that PNG, then copy it into the source GoldenImages/Baselines/ folder to approve it. Add the new <None Include="GoldenImages\Baselines\**\*.png"> csproj entry's CopyToOutputDirectory pattern for any new baseline subfolder.

Golden-image tests are not currently viable for text. Pixel-exact comparison assumes identical rasterization on every CI runner, which text breaks even with every obvious source of drift eliminated. Attempted on #3692 (TextCustomizationGoldenImageTests, since removed): (1) a system font family (FontName = "Arial") — the macOS Actions image has no Arial and silently substitutes a different typeface, blowing the pixel tolerance; fixed by loading a bundled TTF directly via SKTypeface.FromFile and a custom Topten.RichTextKit.FontMapper assigned to the static FontMapper.Default. (2) MathF.Sin-derived glyph offsets/colors — Math.Sin/MathF.Sin call into the OS math library (ucrt/libSystem/glibc), not guaranteed bit-identical cross-platform, so a last-bit difference nudges a glyph by a sub-pixel amount and flips an antialiased edge pixel; fixed by replacing them with a fixed lookup table of exact integers/bytes. Windows still passed and macOS still failed after both fixes — with the identical bundled font and zero floating-point math, Skia itself rasterizes/hints the same glyph outline differently per platform. There is no known fix within this harness's current design (PixelComparer's strict per-pixel-position/channel diff). Until a per-OS-baseline or fuzzy/structural comparison strategy exists, keep golden-image tests restricted to non-text, geometric content (shapes, colors, alpha — see RectangleGoldenImageTests) and cover per-glyph geometry (position/color from a [Custom]-style callback) with deterministic assertions against RichTextKit's own layout data instead (TextBlock.FontRuns[i].GlyphPositions/.Style, as in TextCustomizationTests — no rasterization involved, so it's exact and OS-independent).

Integration Tests (MonoGameGum.IntegrationTests)

Use this project for anything requiring a real GraphicsDevice. The whole assembly shares one Game (SharedGame in SharedGame.cs); a test class is marked [Collection(SharedGame.CollectionName)] and takes a lease on it by deriving a nested class from SharedGameLease (OnInitialize runs on the first RunOneFrame, OnDispose on dispose). ShapeGameLease (Apos.Shapes) and MultiServiceGameLease (1024x768 back buffer) are the other leases. See GumServiceUnitTests.cs for the pattern.

Never declare a Game subclass or a second collection in this project: a real Game created and disposed beside the shared one leaves the shared device without a GL context, so later draws render nothing. SharedGameCollectionGuardTests fails on either. A lease's Dispose presents two frames because MonoGame frees a disposed GL object two Presents later, and first calls HostGame.InvalidateVertexAttributeCache(): MonoGame skips re-pointing vertex attributes for a vertex buffer whose GL name matches the last one drawn, so a new buffer that reuses a freed name (Apos.Shapes' after its predecessor was disposed) is drawn from nothing and the process dies with an access violation. Keep that call before the frames.

The project uses xunit v3 with its own Main (MainThreadTestHost.cs) so tests run on the main thread, which macOS requires. Don't reference MonoGameGum.TestsCommon from it, because that project brings in xunit v2.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.