agentleFS
final-runfinal-run/finalrun-agent/skills/finalrun-use-cli/SKILL.md

finalrun-use-cli

Skill310 starsChanged 8 months agoApache-2.0View on GitHub

Use the published FinalRun CLI to install, configure, validate, run, troubleshoot, and inspect reports for repo-local mobile test workspaces. Trigger this skill for requests involving finalrun, check, test, suite, doctor, runs, start-server.

  • Reads credentials
  • Installs packages

The file

Show its rules and text

We split this skill into the rules it gives an agent

33 rules. About 0 are common, 13 are distinctive, and 20 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈13

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 20

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Prefer local CLI help such as finalrun --help, finalrun test --help, and finalrun suite --help.

    line 27
  2. Very specific:

    Check .finalrun/config.yaml, .finalrun/env/, .finalrun/tests/, and .finalrun/suites/ before asking questions that the repo can answer.

    line 29
  3. Very specific:

    Do not invent unsupported commands such as finalrun init.

    line 30
  4. Distinctive:

    Treat validation failures as blockers.

    line 37
  5. Very specific:

    Use finalrun runs to inspect recent run artifacts after validation or execution.

    line 38
  6. Very specific:

    Ask before starting finalrun start-server, because those commands launch background UI behavior.

    line 39
  7. Very specific:

    If the user wants to create or edit YAML tests, suites, or env bindings, route that work to finalrun-generate-test.

    line 40
  8. Very specific:

    If .finalrun/ is missing:

    line 44
  9. Distinctive:

    do not claim that the CLI can bootstrap the workspace automatically

    line 50
  10. Very specific:

    If finalrun test or finalrun suite fails during validation, report the CLI error directly in plain language, explain what it means, and …

    line 67
  11. Very specific:

    If .finalrun/env/<env>.yaml is missing, malformed, or missing referenced bindings, explain the exact file or binding problem and tell the …

    line 72
  12. Very specific:

    If a secret placeholder is malformed, explain that FinalRun requires the exact ${ENV_VAR} form.

    line 73
  13. Distinctive:

    If a secret references a missing shell environment variable, state the missing variable name and tell the user they must export or set it …

    line 74
  14. Very specific:

    If a spec references an unknown ${secrets.*} or ${variables.*} binding, point to the unresolved binding name and tell the user the env file …

    line 75
  15. Distinctive:

    Never invent, infer, write, or silently substitute secret values.

    line 77
  16. Distinctive:

    test execution can take time and may consume provider credits

    line 95
  17. Very specific:

    After finalrun test or finalrun suite completes, the CLI prints a structured summary.

    line 100
  18. Distinctive:

    Stop. Do not suggest follow-up runs, suite escalation, or additional checks.

    line 108
  19. Distinctive:

    Read the failure output and identify which tests failed and why.

    line 114
  20. Very specific:

    Use openai/... models from the GPT-5 family and above.

    line 201
  21. Very specific:

    Use google/... models from the Gemini 3 family and above.

    line 202
  22. Very specific:

    Use anthropic/... models from the Claude Sonnet 4 / Opus 4 families and above.

    line 203
  23. Very specific:

    Verify the exact provider/model value with finalrun test --help before using sample run commands.

    line 204
  24. Distinctive:

    Run one spec after validation:

    line 214
  25. Distinctive:

    Run one suite after validation:

    line 221
  26. Very specific:

    verify whether .finalrun/env/ exists

    line 257
  27. Very specific:

    if multiple env files exist and the selection is ambiguous, tell the user to choose --env <name> explicitly

    line 258
  28. Distinctive:

    do not write or infer secret values

    line 265
  29. Very specific:

    use finalrun doctor as the default local environment diagnostic

    line 285
  30. Distinctive:

    If the user asks to:

    line 291
  31. Distinctive:

    create a new YAML test

    line 293
  32. Distinctive:

    update a suite manifest

    line 294
  33. Very specific:

    add or change .finalrun/env/*.yaml bindings

    line 295

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

FinalRun CLI Guide and Operator

You help repo users safely use the published finalrun CLI inside repositories that contain .finalrun/. Default to published CLI usage in app repositories, not monorepo contributor workflows, unless the user explicitly asks for contributor setup.

Prerequisites

Install the published package and confirm the CLI is available:

npm install -g @finalrun/finalrun-agent
finalrun --help

For local Android or iOS execution, use finalrun doctor as the first host-readiness diagnostic.

finalrun doctor

Source of Truth

  • Prefer local CLI help such as finalrun --help, finalrun test --help, and finalrun suite --help.
  • Inspect the actual workspace files under .finalrun/ before giving execution advice.
  • Check .finalrun/config.yaml, .finalrun/env/, .finalrun/tests/, and .finalrun/suites/ before asking questions that the repo can answer.
  • Do not invent unsupported commands such as finalrun init.

Core Workflow

  1. Inspect the workspace and confirm finalrun is available on PATH when command execution matters.
  2. Validate first with finalrun check so selectors, suites, config, env bindings, and app overrides are verified before the user relies on a run result.
  3. Explain the exact finalrun test or finalrun suite command you intend to use, including the selected --env, --platform, and --model when relevant.
  4. Treat validation failures as blockers. Explain the CLI error in plain language and do not pretend the test run happened.
  5. Use finalrun runs to inspect recent run artifacts after validation or execution.
  6. Ask before starting finalrun start-server, because those commands launch background UI behavior.
  7. If the user wants to create or edit YAML tests, suites, or env bindings, route that work to finalrun-generate-test.

Missing Workspace

If .finalrun/ is missing:

  • explain that FinalRun must run from a repository containing .finalrun/
  • clarify that only .finalrun/tests is always required; suites/ is only required for suite-based workflows
  • show the expected structure
  • offer a scaffold plan or route the user to finalrun-generate-test
  • do not claim that the CLI can bootstrap the workspace automatically

Expected structure:

.finalrun/
  tests/      # required
  suites/     # optional unless using `finalrun suite` or `--suite`
  env/        # optional
  config.yaml # optional

Validation and Error Handling

  • finalrun check is the default validation command.
  • Validation is mandatory before the user should trust a test run result.
  • finalrun test and finalrun suite are validation-gated. The CLI validates selectors, suites, config, env bindings, and app overrides before execution starts.
  • If finalrun test or finalrun suite fails during validation, report the CLI error directly in plain language, explain what it means, and stop short of describing the run as executed.
  • Diagnose selector, suite, config, env, and binding problems from CLI output first. Do not guess.

Secret and credential errors require explicit user action:

  • If .finalrun/env/<env>.yaml is missing, malformed, or missing referenced bindings, explain the exact file or binding problem and tell the user to fix the env file or choose the correct --env.
  • If a secret placeholder is malformed, explain that FinalRun requires the exact ${ENV_VAR} form.
  • If a secret references a missing shell environment variable, state the missing variable name and tell the user they must export or set it before validation or execution can succeed.
  • If a spec references an unknown ${secrets.*} or ${variables.*} binding, point to the unresolved binding name and tell the user the env file must declare it.
  • If the provider API key is missing, report the exact required variable for the active model (OPENAI_API_KEY, GOOGLE_API_KEY, or ANTHROPIC_API_KEY). When no default model is configured, recommend a repeat-use setup: add .finalrun/config.yaml with model: google/gemini-3-flash-preview, add export GOOGLE_API_KEY=<your-key> to ~/.zshrc, reload the shell with source ~/.zshrc, and rerun. Mention --api-key only if the user asks for a one-off run.
  • Never invent, infer, write, or silently substitute secret values.

Safety Policy

Safe to run without extra approval when the user asks:

  • finalrun check
  • finalrun doctor
  • finalrun runs

Ask before executing:

  • finalrun test
  • finalrun suite
  • finalrun start-server

Why ask first:

  • test execution can take time and may consume provider credits
  • server commands can start background processes and open the browser

Post-Execution

After finalrun test or finalrun suite completes, the CLI prints a structured summary. The agent's response depends on the outcome.

On success

The CLI prints a short confirmation with the artifact directory and report URL. The agent should:

  • Confirm success in plain language.
  • Include the report URL so the user can review the run.
  • Stop. Do not suggest follow-up runs, suite escalation, or additional checks.

On failure

The CLI prints a failure summary listing each failed test with paths to its artifacts. The agent should:

  1. Read the failure output and identify which tests failed and why.
  2. For each failed test, read the artifact files the CLI listed to build context for the user:
    • result.json — the test outcome, failure message, and step-level results.
    • actions/ — individual agent action files (JSON) showing what the agent tried at each step, including its reasoning, the action it took, and whether it succeeded.
    • screenshots/ — per-step screenshots showing the device screen at each action.
    • recording.mp4 or recording.mov — screen recording of the full test (if available).
    • device.log — device-level logs (logcat on Android, log stream on iOS) captured during the test (if available).
    • runner.log — the CLI's own log for the entire run (at the run directory root).
  3. Summarize the failure in plain language: what the test was trying to do, where it failed, and what the device screen showed at that point.
  4. Suggest concrete next steps: whether the test spec needs updating, whether the app has a bug, or whether the environment was misconfigured.

Do not ask the user to go find artifact files. The CLI prints the full paths — read them directly and present the findings.

Command Reference

Validation

  • finalrun check [selectors...]
    • validates the .finalrun workspace, selectors, suite manifests, env bindings, and app overrides
    • supports --env, --platform, --app, and --suite
    • uses .finalrun/config.yaml env when --env is omitted

Execution

  • finalrun test [selectors...]
    • runs one or more specs from .finalrun/tests
    • supports --env, --platform, --app, --suite, --api-key, --model, --debug, and --max-iterations
    • uses .finalrun/config.yaml env and model when those flags are omitted
    • validates before execution starts
  • finalrun suite <suitePath>
    • runs one suite manifest from .finalrun/suites
    • supports --env, --platform, --app, --api-key, --model, --debug, and --max-iterations
    • uses .finalrun/config.yaml env and model when those flags are omitted
    • validates before execution starts

Diagnostics

  • finalrun doctor
    • checks local Android and iOS host readiness
    • supports --platform

Reports

  • finalrun runs
    • lists local run artifacts for the current workspace
    • supports --json
  • finalrun start-server
    • starts or reuses the local report UI for the current workspace
    • supports --port and --dev

Explicit CLI flags override .finalrun/config.yaml defaults.

Common CLI Flags

  • --env <name>
    • used by check, test, and suite
    • selects .finalrun/env/<name>.yaml
  • --platform <android|ios>
    • used by check, test, suite, and doctor
    • required when platform cannot be inferred from context or app override
  • --app <path>
    • used by check, test, and suite
    • .apk implies Android and .app implies iOS
  • --model <provider/model>
    • used by test and suite
    • required unless .finalrun/config.yaml already defines model
  • --api-key <key>
    • used by test and suite
    • overrides provider-specific environment variables
  • --debug
    • used by test and suite
    • enables debug logging
  • --max-iterations <n>
    • used by test and suite
    • caps execution iterations before the run aborts
  • --suite <path>
    • used by check and test
    • on test, remains a compatibility path; prefer finalrun suite <path>
  • --port <n>
    • used by start-server
  • --dev
    • used by start-server
    • runs the report UI in development mode

Supported Providers and Models

  • Supported providers are openai, google, and anthropic.
  • Use openai/... models from the GPT-5 family and above.
  • Use google/... models from the Gemini 3 family and above.
  • Use anthropic/... models from the Claude Sonnet 4 / Opus 4 families and above.
  • Verify the exact provider/model value with finalrun test --help before using sample run commands.

Common Workflows

Validate an existing workspace:

finalrun check --env dev

Run one spec after validation:

finalrun check auth/login.yaml --env dev --platform android
finalrun test auth/login.yaml --env dev --platform android --model google/gemini-3-flash-preview

Run one suite after validation:

finalrun check --suite smoke.yaml --env dev --platform ios
finalrun suite smoke.yaml --env dev --platform ios --model anthropic/claude-3-7-sonnet

Inspect recent runs:

finalrun runs

Open the report UI:

finalrun start-server

Discover current command and model help:

finalrun --help
finalrun test --help
finalrun suite --help

Troubleshooting

Diagnose from the actual CLI output first. Do not guess.

  • Missing .finalrun/
    • explain that FinalRun must run from a repository containing .finalrun/
  • Missing .finalrun/tests/
    • explain that this directory is required by the CLI
  • Missing or ambiguous env files
    • verify whether .finalrun/env/ exists
    • if multiple env files exist and the selection is ambiguous, tell the user to choose --env <name> explicitly
  • Unknown env bindings
    • explain which ${variables.*} or ${secrets.*} bindings are unresolved
    • tell the user the env file must declare them
  • Malformed or missing secrets
    • malformed placeholders must use the exact ${ENV_VAR} form
    • missing shell environment variables must be exported by the user before validation or execution
    • do not write or infer secret values
  • Missing provider API keys
    • openai/... uses OPENAI_API_KEY
    • google/... uses GOOGLE_API_KEY
    • anthropic/... uses ANTHROPIC_API_KEY
    • recommend .finalrun/config.yaml with model: google/gemini-3-flash-preview when no default model is configured
    • recommend export GOOGLE_API_KEY=<your-key> in ~/.zshrc for the default setup path
    • tell the user to reload the shell with source ~/.zshrc and rerun
    • mention --api-key only if they want a one-off run
  • Invalid .finalrun/config.yaml
    • point to the YAML or config error directly
  • Selector and suite resolution failures
    • finalrun test auth/login.yaml resolves from .finalrun/tests/auth/login.yaml
    • finalrun suite smoke.yaml resolves from .finalrun/suites/smoke.yaml
    • explicit .finalrun/tests/... and .finalrun/suites/... paths still work
  • App override and platform mismatch
    • .apk overrides require Android
    • .app overrides require iOS
    • tell the user to fix the path or --platform mismatch instead of ignoring it
  • Host readiness issues
    • use finalrun doctor as the default local environment diagnostic

Coordination with finalrun-generate-test

This skill is for CLI usage, validation, execution, reporting, and troubleshooting.

If the user asks to:

  • create a new YAML test
  • update a suite manifest
  • add or change .finalrun/env/*.yaml bindings
  • plan a new test flow

route that work to finalrun-generate-test.

This skill may explain where the CLI expects those files and how selectors resolve, but it should not duplicate the YAML authoring workflow.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.