agentleFS
korchard333korchard333/claude-power-platform-community/.claude/skills/test-engine/SKILL.md

test-engine

Skill46 starsChanged 8 months agoMITView on GitHub

Power Apps Test Engine + Playwright. Use when: E2E testing of Canvas Apps, Model-Driven Apps, or Power Pages; automated UI verification; regression testing in CI/CD pipelines; visual QA of deployed solutions.

1 other project uses a version of this.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

3 rules. About 0 are common, 3 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈3

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Rule of thumb: Start with Test Engine Power Fx for standard apps.

    line 20
  2. Distinctive:

    Test Engine uses a provider-based architecture.

    line 39
  3. Distinctive:

    Not recording video on CI/CD runs (impossible to debug failures without visual evidence)

    line 78

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Skill: Power Apps Test Engine + Playwright

When to Use

Trigger when testing Power Platform apps end-to-end, writing automated test plans, setting up test pipelines, or verifying UI after deployment.

Decision Matrix: Which Testing Approach?

ApproachWhenLanguageSelectorsAuth
Test Engine (Power Fx YAML)Standard Canvas/MDA testing, low-code teamsPower Fx in YAMLApp-level control namesBuilt-in Entra handling
Raw Playwright (TypeScript)Complex UI, custom controls, third-party components, visual QATypeScript or C#CSS/XPath DOM selectorsManual auth scripting
Test Engine + Playwright hybridStandard tests with occasional DOM accessPower Fx + Preview.PlaywrightAction()BothBuilt-in + direct
Manual testingOne-off verification, exploratory testingN/AN/AN/A

Rule of thumb: Start with Test Engine Power Fx for standard apps. Drop to raw Playwright only when Test Engine abstractions can't reach what you need (custom controls, third-party embedded UIs, visual regression).

AI Accuracy Warning

LLMs are less accurate with Power Fx than mainstream languages (JavaScript, C#, Python) due to smaller training datasets. Always verify Power Fx test steps manually before trusting them in CI/CD. This applies to both test authoring and formula column definitions.

⚠️ REQUIRED: Load Sub-Files Before Implementation

SKILL.md is a summary only — it is NOT sufficient for implementation.

The detailed content (complete payloads, XML templates, working examples, edge-case handling) lives in sub-files in the same directory as this SKILL.md. Before writing any code, you MUST use read_file on the sub-files relevant to your task:

  • Test Plans (YAML Format) — YAML schema, Power Fx test steps, assertions, credentials, recording video
  • Canvas App Testing — Canvas provider, control name selectors, gallery navigation, delegation testing
  • Model-Driven App Testing — MDA provider, form verification, view checks, BPF testing, Dataverse integration
  • Playwright Advanced — Direct Playwright functions, custom C# scripts, raw TypeScript tests, Sean Astrakhan's AI-generated test pattern
  • CI/CD Integration — ADO pipeline YAML, GitHub Actions, publishing .trx results, regression testing on every deploy
  • Power Pages Testing — Portal provider, auth testing, table permission verification, Web API testing

Providers Overview

Test Engine uses a provider-based architecture. Each provider is optimized for a specific application type:

ProviderFlagApplication TypeKey Capabilities
canvas--provider canvasCanvas AppsControl-name selectors, gallery navigation, connector mocking
mda--provider mdaModel-Driven AppsForm interaction, view navigation, BPF testing
powerfx--provider powerfxDataverse (no UI)Direct Power Fx execution, backend logic testing
portal--provider portalPower Apps portalPortal automation operations

Quick Start

# 1. Install prerequisites
# PAC CLI: https://learn.microsoft.com/power-platform/developer/cli/introduction
# Azure CLI: https://learn.microsoft.com/cli/azure/install-azure-cli

# 2. Authenticate
az login --allow-no-subscriptions
pac auth create --name TestAuth --url "https://contoso.crm.dynamics.com"

# 3. Set test user credentials (env vars — never in YAML)
$env:user1Email = "testuser@contoso.onmicrosoft.com"

# 4. Run a canvas app test
pac test run `
  --provider canvas `
  --test-plan-file "tests/smoke-test.te.yaml" `
  --tenant "your-tenant-id" `
  --environment-id "your-env-id"

# 5. View results
# Output: .trx file + screenshots/videos in output directory

Anti-Patterns

  • Using raw Playwright when Test Engine control-name selectors would work (fragile DOM selectors break on platform updates)
  • Storing credentials in test plan YAML files (use environment variables)
  • Skipping test setup/teardown (test data left behind corrupts other tests)
  • Running tests against production data without isolation
  • Not recording video on CI/CD runs (impossible to debug failures without visual evidence)
  • Trusting AI-generated Power Fx test steps without manual verification
  • Using headless: true for initial test development (use headless: false to see what's happening)

Key References

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.