agentleFS
jesspigjesspig/Godot-Autopilot/skills/godot-autopilot-runtime/SKILL.md

godot-autopilot-runtime

Skill1 starsChanged 17 days agoMITView on GitHub

Godot runtime and debugging guide: launch and stop the game, input injection with frame-accurate timing, pause semantics, three log and error paths, the error watermark confirmation loop and runtime inspection. Use when running the game, simulating input or diagnosing problems.

The file

We split this skill into the rules it gives an agent

3 rules. About 0 are common, 1 is distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈1

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Every game call has a three-layer timeout budget: the host waits timeout_ms + 2000 ms (response grace) for the game answer, and that host …

    line 57
  2. Distinctive:

    Always gate text conditions on visibility: hiding a label does not reset its text, so a text-only condition stays true forever once the …

    line 229
  3. Very specific:

    When the game stops at a debugger breakpoint, get_debugger_session_info reports the session as breaked.

    line 448

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Runtime and Debugging with godot-autopilot

Invocation: use the native MCP tools first; when that channel is down, follow the calling ladder in the godot-autopilot-tools skill (the gda CLI, via bunx, a global install, or the standalone gda binary); with no CLI available, work through its references/manual-fallback.md.

This book covers the game process life cycle: launching and stopping the game, the game_* runtime channel, input injection, viewport capture and the visual verification loop, UI reconnaissance, the four log and error paths, the error watermark confirmation loop and inline GDScript tests. Deep dives live in references/input-injection.md, references/debug-paths.md and references/runtime-inspection.md.

Start and stop the game

  • play_editor_current_scene launches the currently edited scene as a game. Save the scene first (save_editor_scene) or playback may fail.
  • It returns result set to ok or already_playing plus an is_playing flag. While a game is already running you get already_playing — the tool does not restart the game. Call stop_editor_playing first if you want a fresh run.
  • stop_editor_playing stops the running game and returns the editor to editing mode. It returns ok even when no game is running, so it is always safe to call at the end of a play-testing session or before modifying the scene again.

Prerequisites of the game_* runtime channel

All game_* tools (and capture_editor_viewport with target set to game) send requests to the running game process through the editor debug session. Two prerequisites must hold:

  1. The game was launched from the editor with play_editor_current_scene.
  2. The game project loads the godot-autopilot extension (plugin enabled in that project).

Without an attached game, tools fail with game not running — start the game from the editor first, then retry. Once attached but not yet ready, the default wait_ready: true queues the request and replays it as soon as the game reports gda:ready, so the call waits instead of failing; pass wait_ready: false to fail immediately with:

game started but not ready: the game process has not reported gda ready yet — wait a moment
after play, or verify the game project loads the godot-autopilot extension

The bridge reports ready asynchronously right after launch. A queued call times out if gda:ready never arrives (the timeout error says the request was queued for readiness); if wait_ready: false calls keep failing while a game process is attached, the game project does not load the extension — enable the plugin there.

Every game call has a three-layer timeout budget: the host waits timeout_ms + 2000 ms (response grace) for the game answer, and that host wait must stay below the HTTP transport's hard 30 s limit. Accepted timeout_ms for game ops is therefore capped at 25000 ms (some tools reject, some clamp, and the low-level sender clamps every request), while editor-only tools keep a 30000 ms maximum (code_execute is the exception: its ceiling is whatever the server reports in the code_execute schema). Budget long work as a chain of shorter calls and poll with get_game_status instead of raising timeout_ms.

Check liveness with get_game_status

get_game_status takes no parameters and reports the engine version, current scene, node count, FPS, paused and physics_frame, plus derived liveness fields:

  • healthy — the game reported activity within the last 3000 ms (see last_activity_ms); read it together with physics_stalled.
  • physics_stalled — physics_frame has not advanced while FPS is positive, i.e. a frozen physics loop.

physics_frame is the engine-global physics frame counter: it keeps advancing while paused is true, so it cannot by itself prove that a pause froze gameplay. Prove the freeze with a gameplay-owned value instead — a game node's position, a tick counter owned by gameplay code, or a dedicated gameplay probe — and read it together with the paused field.

A failed scene reload shows up as an empty scene with a sharp node_count drop. Do not debug a half-dead process: stop with stop_editor_playing and rerun play_editor_current_scene. Full scenario readings live in references/runtime-inspection.md.

The editor and the game are two processes

A running game is a separate OS process from the editor. Anything you do to the editor process does not reach the game:

  • The editor-side input tools — press_input_action, release_input_action, is_input_action_pressed, is_input_action_just_pressed, press_input_key, release_input_key, move_input_mouse, press_input_mouse_button, release_input_mouse_button, start_input_gamepad_vibration, stop_input_gamepad_vibration — inject events into the editor process; a running game never receives them.
  • Script code executed in the editor process likewise cannot touch the game's Input or SceneTree singletons.

To drive the running game use queue_game_input, wait_game_input, sequence_game_inputs or execute_game_script (all of which travel over the debug channel into the game process).

Inject inputs into the running game

  • queue_game_input simulates one input: type is key, mouse_button or action; mode is event (default, parsed input event), api (immediate Input.action_press/action_release) or hold (event-style, held for duration_ms). Parameters outside the whitelist are rejected with an error — a typo fails the call. api only writes the action state: it creates no InputEvent and dispatches none, so _input, _unhandled_input, _gui_input and is_action_just_pressed_by_event never see it. Use event mode (or a real key event) for UI and other event-driven code; api suits only code that polls Input action state (is_action_pressed / is_action_just_pressed).
  • wait_game_input waits until an action reaches a transient state (just_pressed default, just_released or pressed) in a physics frame, or timeout_ms elapses (default 2000, max 25000). Optional inject sends the input first in the same call; just_pressed/just_released require inject — without it the one-frame transient window has already expired.
  • sequence_game_inputs fires a timeline of inputs on exact physics frame offsets (max 256 items, each with kind and at_frame) — the tool for playtest automation.
  • get_game_input_status reports an action's pressed, just_pressed, just_released and the current physics_frame. Engine errors are not attached to this response — when input appears ignored, read get_debugger_errors (running game) or get_debugger_log (editor process) separately.

Two engine facts govern the timing of everything above:

  • just_pressed has two counters: a press records physics_frame + 1 for the physics context and the current frame for the process context. Right after injection, checking from the same _process returns true via the process branch, while the same physics tick returns false — the earliest a physics-context check agrees is the next tick. Transient states are tracked against physics frames only: poll in _physics_process, not _process.
  • Injected events land on the next frame: parsed input events go through a buffer flushed at the end of the display server's event pass, so an event injected during _process is dispatched at the start of the next frame. Use sequence_game_inputs offsets, not round-trip timing.

Frame-accuracy recipe: script the whole interaction as one sequence_game_inputs timeline with explicit at_frame offsets, then verify with get_game_input_status or a conditioned capture - never assume an inject-then-read round trip lands on the same frame, because injected events dispatch at the next frame start.

Full semantics — buffering and flush points, api mode forgery versus real key events, action matching and per-device state — are in references/input-injection.md.

Screenshots and visual verification

Screenshots are the observation step of the change/observe/verify loop: look at the rendered result instead of trusting the values you set. Capture after UI or scene appearance changes (editor and in-game), at key moments of a playtest, and before and after click automation.

Three tools, three targets:

  • capture_game_viewport — the running game's viewport over the runtime channel (data, format, width, height). Optional timeout_ms bounds the wait (default 5000, max 25000; see the budget chain above).
  • capture_editor_viewport — the default target editor grabs the editor 2D viewport (falling back to the 3D viewport), so it works while you are still building a scene; target game captures the running game instead. save=true (or giving save_dir) applies to both targets: the PNG is written into the GDA per-project capture folder by default (under the GDA home directory, not the Godot user data folder), or into the given res:// directory inside the project root, and the result gains saved_path; save_dir outside the project root / GDA captures folder is rejected with path rejected. Saving requires the write capability; both targets keep only the 20 most recent capture files. On the plain capture_game_viewport the same save/save_dir parameters write a copy after the editor reads the transport PNG back from the game.
  • capture_display_screen — a whole physical screen by screen index, for desktop-level checks such as window placement; no running game required.

Reading small details: scale

scale (integer 1-8, default 1) nearest-neighbour upscales the output image, which is how you read small HUD text: capture a region around the text and scale it up 4-8x. Order is region crop, then scale, then max_dimension — the cap still wins when both are given, and the result's width/height always report the final size. Nearest-neighbour keeps pixel edges crisp; the max_dimension downscale stays bilinear and comes last.

Capturing short-lived events: after_frames and when

Sampling a 2 s banner or a 0.7 s death screen by polling capture_game_viewport races the event window and engine time-scale slow-motion corrupts the gameplay being verified. Instead hand the condition to the game and let it decide when to shoot:

  • after_frames (integer >= 0) — wait this many rendered frames after the request before capturing (0 = immediately).
  • when — a GDScript expression evaluated once per rendered frame inside the game; the capture fires on the first frame where it is true. An empty string means no condition.

The condition frame is reported as when_satisfied_frame; the image itself is read on the next rendered frame and reported as captured_frame (captured_frame > when_satisfied_frame), so the picture shows the state after the condition became true rather than the frame that evaluated it.

Give both and the frame budget must be spent before the condition is even evaluated. when runs with the current scene as the base instance (the tree root when no scene is set), so conditions are plain node lookups, e.g. a HUD label whose text becomes non-empty, a Visible panel, or a health value crossing a threshold. A malformed expression fails immediately with a structured error carrying the Expression error text; an expression that keeps raising at runtime (a missing node, a wrong path) does not fail the call — its last error is attached to the timeout error instead. timeout_ms bounds the entire wait, and on expiry you get a structured error (code when_timeout) with the frames waited, never a silent hang.

The forensics pattern is act first, then condition, then capture:

{"name": "call_tool", "arguments": {"name": "capture_game_viewport", "arguments": {"when": "get_node(\"HUD/MessageLabel\").visible and get_node(\"HUD/MessageLabel\").text != \"\"", "timeout_ms": 4000}}}

Always gate text conditions on visibility: hiding a label does not reset its text, so a text-only condition stays true forever once the text was set. The full deterministic-verification recipe (single-script teleport-await-read, no cross-call timing) is in references/runtime-inspection.md.

Both parameters exist on capture_game_viewport and on capture_editor_viewport with target game. The editor target rejects them: its capture runs synchronously on the editor main thread and cannot yield frames.

Delivery: through call_tool the base64 PNG arrives as an MCP image content block, so a multimodal model sees the picture directly. The text JSON keeps format, width and height, and its data field reads "<attached-as-image-content>" with image_attached: true. Inside batch_execute and code_execute no image block is attached — the JSON keeps the full base64.

Limits: captures are capped at 4096 pixels per side and an 8 MiB PNG, and a capture whose base64 payload would exceed the 4 MiB JSON response cap is rejected with a response_too_large-style error (base64 inflates by about one third), so prefer moderate resolutions.

Pair screenshots with get_game_ui_elements: enumerate the controls first (path, global_rect), inject the click with sequence_game_inputs or queue_game_input, then capture again and re-enumerate — screens change under fixed coordinates.

Enumerate the running game's UI

get_game_ui_elements walks the running game's scene tree and returns every Control-derived node with:

  • path (absolute node path), type (class name), visible (including ancestors), text (when present), global_rect (position and size in viewport coordinates), mouse_filter (STOP/PASS/IGNORE) and z_index. The last two are the hit-testing and draw-order facts: a full-screen PASS control can swallow clicks meant for controls below it, while IGNORE passes them through.

max_elements defaults to 100 (max 1000); truncated set to true signals more elements remain — raise the cap or target a subtree via execute_game_script. This is the reconnaissance step before click automation: locate buttons and panels here, then drive them with sequence_game_inputs or queue_game_input using global_rect coordinates, or with execute_game_script using path.

  • validate_game_ui_layout scans the running game's Control tree for layout defects (zero-size visible controls, off-viewport controls, possibly-occluded controls, naming the occluding control) and reports each as a triage item with no automatic verdict. It complements get_game_ui_elements reconnaissance before click automation.

UI change acceptance: run validate_game_ui_layout first, then click with click_game_ui_element and check the hit result against the target. click_game_ui_element returns hit_path (the control actually under the injected point) and hit_mismatch (true when it is not the requested target); by default a mismatch is reported without failing the call, while strict_hit: true turns it into an error. A hit_mismatch together with a possibly_occluded issue is the signature of a full-screen PASS container eating the click.

Pause, reload and run code in the game

  • set_scene_tree_pause (paused boolean) pauses or resumes the running scene tree; is_scene_tree_paused confirms the state. Pause semantics are a matrix, not a switch: nodes keep processing according to their process mode (INHERIT resolves up the tree, the root is fixed PAUSABLE), and pausing also deactivates the physics servers — the simulation itself stops, not just node callbacks. Transient input states are lost while paused. physics_frame keeps advancing while paused because it is the engine-global counter, so combine paused with a gameplay value (node position or a gameplay-owned tick) when you need to prove that gameplay froze. The full matrix is in references/input-injection.md.
  • reload_scene_tree_current_scene reloads the currently running scene from disk, recreating the scene root — script state, connections and runtime modifications reset to the saved file. Without a running scene it returns ERR_UNCONFIGURED (3).
  • execute_game_script runs code inside the game process: action script takes either a complete GDScript defining func _run() (used as-is) or bare statements, auto-wrapped into extends Node + func _run() — the same bare-statement wrapping as the editor channels, so a value needs an explicit return and wrapped calls report wrapped: true; get_property, set_property or call_method inspect or modify node state instead. persist (default false) keeps the temporary node alive under /root/__gda_runtime.
  • reload_game_scripts reloads GDScript files in the running game without restarting it (pass a paths array of res:// script paths). The request is applied during the game's next idle poll and no confirmation is returned — verify with get_game_log_entries or get_game_status.

Four log and error paths

PathToolsSourceNeeds a running game
Editor engine logget_debugger_logengine log buffer of the editor process: script errors and messages routed through the engine loggerno — always contains data
Debugger session captureget_debugger_errors, get_debugger_output, get_debugger_scene_treewith an active debug session: the running game over the runtime channel; without one: an empty result plus a note — no editor-side fallbacklive game data needs a session
On-disk game logget_game_log_entriestail window of the game process log file user://logs/godot.logthe log file must exist — start the game once with play_editor_current_scene
Plugin LogSystemget_plugin_logthe plugin's own in-process diagnostic buffer (authorization denials, timeout bookkeeping, dropped late game responses) — not routed through any engine loggerno — always available
  • get_debugger_log (optional limit, default 50) reads the editor engine log — script errors and print output from the editor process. It never requires a running game; read it first after editing any .gd, .tscn, .tres or .cs file, before changing code again.
  • get_plugin_log (optional limit, default 100, max 1000; level, category, filter, since_index) reads the plugin's own in-process diagnostics and returns the entries array with a count and next_index. Unlike get_debugger_log it does not read the engine log buffer; this is where authorization denials, timeout bookkeeping and dropped late game responses appear.
  • get_debugger_errors (optional limit, default 20) returns, with an active session, a structured list under result with time, file, func, line, error, descr, is_warning and stack per entry.
  • get_debugger_output reads stdout/stderr captured from the game process (optional limit, default 50); get_debugger_scene_tree takes no parameters and returns the game's tree as formatted text.
  • Without an active debug session, get_debugger_errors, get_debugger_output and get_debugger_scene_tree return an empty result plus a note suggesting play_editor_current_scene or get_game_log_entries — there is no editor-process fallback.
  • get_game_log_entries reads the on-disk log tail (optional limit, default 50, max 500; returns path, entries, total_lines). Pass an optional filter to keep only lines containing that case-sensitive substring: the scan then covers the last 2000 lines and the result adds matched_lines with the total matches found (an empty filter means no filtering). It falls back to the archived godot.log.1 (reported as from_archive); if the file does not exist the error includes directory diagnostics and a hint to start the game.

The on-disk log matters more than it looks: the debugger transport silently drops messages under load (see below), but the log file does not.

  • collect_game_evidence collects status, screenshot and recent errors from the running game in one read-only round trip so the three observations belong to the same moment; each section fails in isolation, and triage is left to you. Prefer it over separate get_game_status, capture_game_viewport and get_debugger_errors calls when moment-consistency matters.

Confirm a fix with the error watermark

Errors raised while the game runs are recorded by the plugin, and the next response of any tool carries a top-level new_errors_since_last_call field. The counter is one-shot: it is consumed when read, so each response reports exactly the errors recorded since your previous call (a failed call counts its own error too). The field is always present, including when it is 0.

The correct "I fixed it — confirm nothing new broke" loop:

  1. Reproduce and read the errors with get_debugger_errors or get_game_log_entries.
  2. Apply the fix.
  3. Re-run the game and exercise the broken code path.
  4. Make any next tool call and read new_errors_since_last_call: 0 means nothing new was recorded; more means new errors appeared — read them before moving on.

One caveat keeps this loop honest: why you may read no new errors even when errors happened. The engine's debug transport has three drop gates — messages sent before a debugger peer connects are lost entirely (early startup errors never surface), error/warning floods are silently discarded for up to a second, and a message-queue overflow drops messages wholesale. A watermark of 0 therefore means "nothing recorded since last call", not "nothing happened": for error-storm code paths, cross-check with get_game_log_entries, which bypasses all three gates. Details in references/debug-paths.md.

Close the loop with inline GDScript tests

run_gdscript_tests runs inline test cases with a built-in assertion framework. Each item of tests is a {name, source} object; the source must be plain sequential statements (top-level func/static/class declarations are rejected), is wrapped into a temporary @tool Node script, executed serially and removed afterwards so the tree stays clean. The edited scene root is exposed as SceneRoot.

Assertions available inside a source:

  • check(condition, msg="") — records a failure when false, keeps executing.
  • check_equal(actual, expected) — Variant equality.
  • check_almost_equal(actual, expected, epsilon=0.001) — numeric absolute difference; non-numeric values fall back to check_equal semantics.
  • fail(msg) — records a failure and keeps executing.
  • fatal(msg) — records a failure and immediately stops this case.

A failed assertion does not abort the case; every check appends one row to checks. Optional timeout_ms (default 10000, hard maximum 30000) budgets the whole suite: once exhausted, remaining cases get status skip with reason timeout budget exhausted, and the running case is flagged running_over_budget (synchronous GDScript cannot be interrupted).

run_gdscript_test_files runs the same framework over files in directory (default res://tests, non-recursive) filtered by pattern (default *.gda_test.gd); each file is one case named by its res:// path.

{"name": "run_gdscript_tests", "arguments": {"tests": [
  {"name": "math", "source": "check_equal(1 + 1, 2)\ncheck_almost_equal(sqrt(2.0), 1.414, 0.01)\ncheck(SceneRoot != null, \"scene must be open\")"}
]}}

The response contains a summary object (total, passed, failed, skipped) and a cases array (name, status, checks, error, elapsed_ms), status one of pass, fail or skip. Verification loop: modify code, run run_gdscript_tests, re-run the game, then confirm via get_game_status, get_game_log_entries and new_errors_since_last_call.

Debug helpers while the game runs

  • Monitors: get_debug_monitor_catalog lists every built-in monitor with id, name and type — the authoritative source of ids, never guess; get_debug_monitor reads one by integer id; get_debug_monitors returns a full snapshot of all 58 monitors. Custom monitors via get_debug_custom_monitor_names, get_debug_custom_monitor (JSON Variant value) and remove_debug_custom_monitor.
  • sample_game_property samples one game node property across N process frames without a caller-side polling loop, returning the serialized value per sample; the property must exist and the run is bounded by a timeout. Use it to watch a value evolve instead of hammering single reads.
  • Leaks and stacks: get_debug_object_count and get_debug_node_count — compare before/after an operation; a rising node count suggests leaks. get_debug_memory_usage is static engine memory only. get_debug_stack returns editor-process backtraces without a debug session.
  • Visualization: set_debug_collision_visual, set_debug_navigation_visual and set_debug_performance_visual toggle overlays and persist in project.godot; set_scene_tree_debug_collisions_hint affects the current run only; set_debug_physics_fps speeds or slows physics — restore 60.

Breakpoints and auto-resume

When the game stops at a debugger breakpoint, get_debugger_session_info reports the session as breaked. By default the plugin automatically continues breaked sessions so game_* calls keep working: auto-resume is controlled by the GDA_AUTO_CONTINUE environment variable (0 or false disables it), with at most 3 auto-continues per session. What a breakpoint does to the game process is in references/debug-paths.md.

See also

  • references/input-injection.md — frame-accurate injection, action matching, per-device state and the full pause matrix.
  • references/debug-paths.md — the four log paths, protocol drop gates and breakpoint semantics.
  • references/runtime-inspection.md — reading liveness, UI and scene state of the running game.
  • godot-autopilot-scripting — the four script execution channels, including execute_game_script.
  • godot-autopilot-tools — tool discovery protocol, watermark details and retryable soft errors.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.