Foundry E2E Testing

End-to-end testing for Falcon Foundry apps using Playwright and the @crowdstrike/foundry-playwright library.

The library provides authentication, app install/uninstall, page objects, and configuration so each app only writes its app-specific tests.

Part of a suite. If development-workflow has not already run, and this is a new app or its first capability, load the development-workflow skill first — it owns the CLI prerequisite check, scaffolding order, and manifest coordination.

Quick Start

1. Create the e2e/ directory

my-foundry-app/
├── e2e/
│   ├── .env                    # Local credentials (git-ignored)
│   ├── .env.sample             # Template for other developers
│   ├── .gitignore
│   ├── package.json
│   ├── playwright.config.ts
│   └── tests/
│       └── foundry.spec.ts
├── manifest.yml
└── ...

2. package.json

Node.js LTS is recommended.

{
  "name": "playwright-foundry",
  "version": "1.0.0",
  "scripts": {
    "test": "npx playwright test",
    "test:ui": "npx playwright test --ui",
    "test:debug": "npx playwright test --debug",
    "test:verbose": "DEBUG=true npx playwright test --reporter=list"
  },
  "type": "commonjs",
  "devDependencies": {
    "@crowdstrike/foundry-playwright": "0.5.6",
    "@types/node": "25.6.0"
  }
}

Always pin exact versions — never use "latest", "^", or "~". Check npm for the current version of each package.

The library brings @playwright/test, @dotenvx/dotenvx, and otpauth as transitive dependencies; don't install them separately.

3. .env

FALCON_USERNAME=your.email@company.com
FALCON_PASSWORD=your-password
FALCON_AUTH_SECRET=your-totp-secret
FALCON_BASE_URL=https://falcon.us-2.crowdstrike.com
APP_NAME=your-app-name

Convention for sample apps: Set APP_NAME to match the manifest name field, which should match the repo name (e.g., foundry-sample-functions-python). This avoids spaces in names and simplifies CI. This is a convention, not a hard requirement.

4. playwright.config.ts

import { defineFoundryConfig } from '@crowdstrike/foundry-playwright';

export default defineFoundryConfig();

This gives you the standard 4-project pipeline automatically:

1. setup: authenticate and save session state
2. app-install: install the app via App Catalog
3. chromium: run your tests
4. app-uninstall: clean up after tests

5. .gitignore

node_modules/
playwright/.auth/
playwright-report/
test-results/
.env

6. Install and run

cd e2e
npm install
npx playwright install chromium --with-deps
npm test

Writing Tests

Available page objects

The library provides these page objects:

Fixtures pattern

Create src/fixtures.ts to wire up page objects as Playwright fixtures. Only import what your tests actually use — don't define unused fixtures:

import { test as baseTest } from '@playwright/test';
import { DetectionExtensionPage, WorkflowsPage, HostManagementPage } from '@crowdstrike/foundry-playwright';

type FoundryFixtures = {
  detectionExtensionPage: DetectionExtensionPage;
  workflowsPage: WorkflowsPage;
  hostManagementPage: HostManagementPage;
};

export const test = baseTest.extend<FoundryFixtures>({
  detectionExtensionPage: async ({ page }, use) => { await use(new DetectionExtensionPage(page)); },
  workflowsPage: async ({ page }, use) => { await use(new WorkflowsPage(page)); },
  hostManagementPage: async ({ page }, use) => { await use(new HostManagementPage(page)); },
});

export { expect } from '@playwright/test';

Playwright fixtures are lazy, so unused ones cost nothing at runtime, but they're dead code. Add fixtures as tests need them.

Example test: workflows

import { test } from '../src/fixtures';

test.describe.configure({ mode: 'serial' });

test('should execute workflow', async ({ workflowsPage }) => {
  test.setTimeout(180000);
  await workflowsPage.navigateToWorkflows();
  await workflowsPage.executeAndVerifyWorkflow('My Workflow Name');
  await workflowsPage.verifyWorkflowExecutionCompleted();
});

test('should execute workflow with input', async ({ workflowsPage, hostManagementPage }) => {
  test.setTimeout(180000);
  const hostId = await hostManagementPage.getFirstHostId();
  if (!hostId) { test.skip(true, 'No hosts available'); return; }

  await workflowsPage.navigateToWorkflows();
  await workflowsPage.executeAndVerifyWorkflow('Host Details Workflow', {
    inputs: { 'Host ID': hostId },
  });
  await workflowsPage.verifyWorkflowExecutionCompleted();
});

executeAndVerifyWorkflow() handles search, execution trigger, and initial verification. verifyWorkflowExecutionCompleted() opens the execution detail view in a new tab and polls until the status leaves "In Progress" — it fails the test if the execution reports "Failed" and times out after 120s by default. For render-only checks (a workflow without credentials), use verifyWorkflowRenders().

Example test: UI extensions

import { test, expect } from '../src/fixtures';

test('should render extension', async ({ detectionExtensionPage }) => {
  const frame = await detectionExtensionPage.openExtension('hello');
  await expect(frame.getByText(/My App Title/i)).toBeVisible({ timeout: 10000 });
});

openExtension() navigates to Endpoint Detections, opens the first detection, scrolls to the named extension button, expands it, and returns the iframe FrameLocator.

Apps with Configuration Screens

If your app has API integration settings during install (e.g., ServiceNow credentials), the default install will fail because the Install button stays disabled until fields are filled.

1. Add integration credentials to .env and .env.sample

# .env.sample — commit this as a template
SERVICENOW_INSTANCE_URL=https://dev123456.service-now.com
SERVICENOW_USERNAME=your-servicenow-username
SERVICENOW_PASSWORD=your-servicenow-password

# .env — local values, git-ignored
SERVICENOW_INSTANCE_URL=https://dev99999.service-now.com
SERVICENOW_USERNAME=admin
SERVICENOW_PASSWORD=s3cret

2. Create a custom tests/app-install.setup.ts

import { test as setup } from '@playwright/test';
import { AppCatalogPage, config } from '@crowdstrike/foundry-playwright';

setup('install app', async ({ page }) => {
  const catalog = new AppCatalogPage(page);

  const instanceUrl = process.env.SERVICENOW_INSTANCE_URL;
  const username = process.env.SERVICENOW_USERNAME;
  const password = process.env.SERVICENOW_PASSWORD;
  if (!instanceUrl || !username || !password) {
    throw new Error('Missing required ServiceNow env vars: SERVICENOW_INSTANCE_URL, SERVICENOW_USERNAME, SERVICENOW_PASSWORD');
  }

  await catalog.installApp(config.appName, {
    configureSettings: async (page) => {
      await page.getByRole('textbox', { name: 'Name', exact: true }).fill('ServiceNow Integration');
      await page.getByRole('textbox', { name: 'Instance' }).fill(instanceUrl);
      await page.getByRole('textbox', { name: 'Username' }).fill(username);
      await page.getByRole('textbox', { name: 'Password' }).fill(password);
    },
  });
});

The library loads .env automatically (via @dotenvx/dotenvx) so process.env values are available without extra setup. In CI, set these as GitHub Actions secrets instead.

3. Point the config at the custom install

export default defineFoundryConfig({
  appInstallDir: './tests',
});

How to discover field names: Use Playwright MCP to take a snapshot of the install page and inspect the form fields. See debugging-with-mcp.md.

Disabling Workflow Provisioning

Apps with workflows that require valid API credentials will fail during install if you provide fake credentials for the configuration screen. Some workflows are also long-running (e.g., Anomali ThreatStream ingestion) and shouldn't be provisioned during testing because they'll start automatically. Use AppBuilderPage.disableWorkflowProvisioning() before install to skip provisioning:

import { test as setup } from '@playwright/test';
import { AppBuilderPage, AppCatalogPage, config } from '@crowdstrike/foundry-playwright';

setup('install app', async ({ page }) => {
  setup.setTimeout(300000);
  const appBuilder = new AppBuilderPage(page);
  await appBuilder.disableWorkflowProvisioning(config.appName);

  const catalog = new AppCatalogPage(page);
  await catalog.installApp(config.appName);
});

This navigates to the App Builder, finds the app, toggles off "Provision on install" for each workflow template, and saves. When it changed anything, it also deploys and releases a new version of the app from the App Builder; it skips all of that if a previous release already has provisioning disabled. Call it before installApp() in your custom app-install.setup.ts.

Multi-Screen Configuration Wizards

Some apps have settings spread across multiple screens (e.g., Workday and SailPoint, with 3 each). The screens don't always appear in the same order, so loop over them, detect the current screen by a field only it has, fill it, and click "Next setting" when it's visible:

import { test as setup } from '@playwright/test';
import { AppBuilderPage, AppCatalogPage, config } from '@crowdstrike/foundry-playwright';

setup('install app', async ({ page }) => {
  setup.setTimeout(300000);
  const appBuilder = new AppBuilderPage(page);
  await appBuilder.disableWorkflowProvisioning(config.appName);

  const catalog = new AppCatalogPage(page);

  await catalog.installApp(config.appName, {
    configureSettings: async (page) => {
      const nextButton = page.getByRole('button', { name: 'Next setting' });

      for (let screen = 0; screen < 3; screen++) {
        if (await page.getByRole('textbox', { name: 'Client ID' }).isVisible({ timeout: 3000 }).catch(() => false)) {
          await page.getByRole('textbox', { name: 'Name' }).fill('My Integration');
          await page.getByRole('textbox', { name: 'Host' }).fill('https://api.example.com');
          await page.getByRole('textbox', { name: 'Client ID' }).fill(process.env.CLIENT_ID!);
          await page.getByRole('textbox', { name: 'Client Secret' }).fill(process.env.CLIENT_SECRET!);
        } else if (await page.getByRole('textbox', { name: 'Tenant ID' }).isVisible({ timeout: 3000 }).catch(() => false)) {
          await page.getByRole('textbox', { name: 'Tenant ID' }).fill(process.env.TENANT_ID!);
        } else {
          await page.getByRole('textbox', { name: 'Name' }).fill('My Second Integration');
          await page.getByRole('textbox', { name: 'Host' }).fill('https://api.example.com');
        }

        // The last screen shows "Install app" instead of "Next setting"
        if (await nextButton.isVisible({ timeout: 2000 }).catch(() => false)) {
          await nextButton.click();
          await page.waitForLoadState('domcontentloaded').catch(() => {});
        }
      }
    },
  });
});

Set the loop count to your app's screen count. The .catch() calls keep a missing field or an already-loaded page from failing setup.

Custom Page Objects

For app-specific UI that the library doesn't cover, extend BasePage:

import { Page, expect } from '@playwright/test';
import { BasePage, AppCatalogPage, config } from '@crowdstrike/foundry-playwright';

export class MyAppPage extends BasePage {
  constructor(page: Page) {
    super(page, 'MyAppPage'); // display name for logging only
  }

  protected getPagePath(): string {
    throw new Error('Direct path navigation not supported. Use navigateToInstalledApp() instead.');
  }

  protected async verifyPageLoaded(): Promise<void> {
    await this.page.locator('h1').filter({ hasText: /My App/i }).waitFor();
  }

  async navigateToInstalledApp(): Promise<void> {
    return this.withTiming(async () => {
      const catalog = new AppCatalogPage(this.page);
      await catalog.navigateToInstalledApp(config.appName);
      await this.verifyPageLoaded();
    }, 'Navigate to installed app');
  }

  async verifyAppContent(): Promise<void> {
    return this.withTiming(async () => {
      const iframe = this.page.frameLocator('iframe[name="portal"]');
      const heading = iframe.getByRole('heading', { name: /My App/i });
      await expect(heading).toBeVisible({ timeout: 10000 });
    }, 'Verify app content');
  }
}

Foundry app page URLs contain deployment-specific IDs that change on every deploy, so never hardcode paths. Use AppCatalogPage.navigateToInstalledApp() to navigate via the App Catalog menu instead.

BasePage gives you smartClick(), withTiming(), navigateToPath(), elementExists(), a logger, and a waiter (SmartWaiter instance).

Add the custom page to your fixtures alongside library page objects.

Debugging with Playwright MCP

Playwright MCP is invaluable for writing and debugging e2e tests. See references/debugging-with-mcp.md for detailed patterns.

Key principle: When using Playwright MCP interactively, authentication is manual. Navigate to the Falcon login page, then tell the user: "Please log in to Falcon in the browser, then let me know when you're ready." Do not attempt automated TOTP login through MCP.

CI with GitHub Actions

The sample apps use a shared e2e.yml workflow pattern. Rather than duplicating it here (where it would go stale), reference the canonical implementation:

Reference: foundry-sample-functions-python/.github/workflows/e2e.yml

Key CI concepts

1. Concurrency serialization: Only one e2e run per repo at a time (prevents deployment collisions)
2. Unique app names: CI generates a unique name from repo + actor + timestamp to avoid conflicts
3. Manifest ID stripping: yq -i 'del(.. | select(has("id")).id) | del(.. | select(has("app_id")).app_id)' manifest.yml
4. Deploy → wait → release → test → cleanup: The workflow deploys, polls for success, releases, runs tests, then always deletes the app
5. App cleanup: foundry apps delete -f runs in an always() step so apps are cleaned up even on failure

Required GitHub secrets

CI-specific behavior

The library detects CI=true and adjusts:

• Higher timeouts (60s default vs 45s local)
• Retries enabled (2 retries vs 0 local)
• .env loading skipped (credentials come from environment)

defineFoundryConfig() Options

Sidebar navigation flakiness

The Falcon sidebar menu re-renders during navigation, which can detach DOM elements mid-click and cause intermittent timeouts. This affects any test that navigates through the sidebar (detections, workflows, host management, etc.). The library's navigation methods include retry logic for this, but with retries: 0 locally, a single re-render failure will fail the test.

For apps that navigate through the sidebar, set retries: 2 to handle this reliably:

export default defineFoundryConfig({ retries: 2 });

Common Pitfalls

Reference Implementations

All CrowdStrike/foundry-sample-* apps have e2e tests using this library:

Integration with Other Skills

• development-workflow: E2E testing is delegated from the orchestrator when users request it
• ui-development: Tests verify UI pages and extensions render correctly in the Falcon console
• workflows-development: Tests verify workflow execution and completion
• debugging-workflows: Use for deploy/release issues that block testing