agentleFS
eduardoghieduardoghi/reverse-engineering-skills/skills/wine-debugging/SKILL.md

wine-debugging

Skill2 starsChanged 12 days agoMITView on GitHub

Find out why a Windows program is slow, behaves differently or dies when it runs under Wine on Linux. Use to split wall time between CPU, file access and waiting, to measure what a network share costs, to reproduce Windows text widths, and to capture a silent crash.

The file

We split this skill into the rules it gives an agent

7 rules. About 0 are common, 0 are distinctive, and 7 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈0

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 7

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    All the commands below work on the process of the program, so find its Linux PID first.

    line 14
  2. Very specific:

    Add the option to the mount options (pam_mount.conf.xml, /etc/fstab) to keep it after the next login.

    line 98
  3. Very specific:

    Use coordinates relative to the window (mousemove --window) and read them from a capture of that window, where pixel (x, y) of the image is …

    line 130
  4. Very specific:

    After every action that can open a message box, list the windows again with wmctrl -l and look for a new one before typing anything.

    line 134
  5. Very specific:

    Don't rely on a fixed sleep after an action that loads data.

    line 138
  6. Very specific:

    Run the program from wine in the same prefix as the application while it is open.

    line 233
  7. Very specific:

    When a patched program dies, bisect with a variant that follows the same path but uses a constant where the real data goes.

    line 257

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Debugging a Windows program under Wine

The usual question is whether the slowness or the failure belongs to the program or to the environment Wine builds around it. Measure first. The same binary can take 7 seconds on one machine and 80 on another because of how files are listed, how fonts are measured or how a share is mounted.

All the commands below work on the process of the program, so find its Linux PID first. Wine names the process by the Windows command line, which /proc/PID/cmdline shows:

for p in /proc/[0-9]*; do tr '\0' ' ' < $p/cmdline 2>/dev/null | grep -q '^C:.Dir.app.exe' && basename $p; done

Where the time goes

perf and the wall clock answer different questions.

  • perf record -e cpu-clock:u -F 499 -g -p PID -- sleep 120 counts CPU time only, and it works on the process of your own user without extra permission. A program that waits for files or the network shows a few seconds of CPU while the user waits a minute. With kernel.perf_event_paranoid at 2 on a hybrid Intel CPU, -e cpu-clock:u and -g failed to open, while the default event and --call-graph=dwarf,8192 --user-callchains worked.
  • strace -f -c -w -p PID -o resumo.txt totals the wall time spent in each system call, for all threads. Idle threads (a timer thread in pselect6, a waiting thread in ioctl or futex) dominate the totals, so look at what the busy thread does. Stop it with Ctrl+C or kill -INT so it prints.
  • To see the main thread over time without attaching anything, read /proc/PID/task/PID/stat (field 3 is the state, R running, S sleeping, D waiting on I/O) and /proc/PID/task/PID/syscall every 20 ms. A thread blocked in read, getxattr or newfstatat in state D is waiting for files. Group the samples by 10 seconds to see when the slow part starts and ends. Reading syscall needs ptrace permission (see kernel.yama.ptrace_scope in the delphi-bpl skill). stat, wchan and /proc/PID/io don't, so without it sample the state, the wchan of each thread and the rchar and syscr counters of the process. A main thread in ntsync_schedule with flat counters is waiting on a Windows object, which for a program with a message loop usually means it is idle, so check this before profiling a screen the user calls slow.
  • To measure the cost of each item of a list (a model, a record, a file the program opens per item), start the program as a child of strace and log the opens with time stamps: strace -f -tt -e trace=openat,open -o opens.txt wine app.exe. With kernel.yama.ptrace_scope at 1 you can't attach to a running program, but a child is allowed. Group the opens by file name. The same file opened in two separate bursts usually means two passes over the list (building it, then checking each item), so take the first open of each file for one pass and the second for the other. The interval between consecutive items of a pass is the cost per item. Leave out intervals longer than a second or two, which are the user doing something else, and report how many you left out. In one case this showed 9 seconds for the pass that built a list and 382 seconds for a second pass that loaded every item again to read one field.
  • Many sockets show up as sendmsg and recvmsg to the wine server. Count the ones on the database socket and the gap between a send and the next read to get the round trip time. In one case 1281 round trips cost 0.7 seconds in total, which cleared the database of any blame.

Stacks of 32-bit code

With the new wow64 mode of Wine a 32-bit program runs inside a 64-bit Linux process, and perf doesn't know its code. The PE images appear as [JIT] tid PID (backed by /tmp/perf-PID.map) or as Wine DLLs without symbols, and --call-graph=fp unwinds the 32-bit frames as 64-bit ones, which gives garbage addresses. The dwarf mode still copies the raw user stack of every sample, and bin/pe-stacks reads it:

perf record --call-graph=dwarf,8192 --user-callchains -F 299 -p PID -o perf.data -- sleep 90
cp /proc/PID/maps maps.txt                       # the bases of the images, while the program runs
pe-stacks --maps maps.txt perf.data                          # whole recording
pe-stacks --maps maps.txt --from 5012.3 --to 5017 perf.data   # perf timestamps, in seconds
pe-stacks --maps maps.txt --module app.dll --depth 1 perf.data

Pick the busy window for --from and --to from the samples per second it prints. --module starts each path at the innermost frame of one image, which shows the function of the program that spends the time when most of the CPU is in the runtime library. A private function shows as the export before it plus an offset, so read it with bpl-xref or rizin. The scan also picks up stale return addresses left by calls that already returned. The real chain repeats in hundreds of samples, so read the counts and not one sample.

Files on a network share

Wine asks for two extended attributes of every file it lists, user.DOSATTRIB and user.WINEREPARSE, with getxattr. On a CIFS mount each call goes to the server, even when the file has no such attribute and the answer is ENODATA. For a folder of 6000 files that was 8700 calls of about 2 ms, over 20 seconds of waiting in a single directory listing.

python3 - <<'PY'
import os, time
files = sorted(os.listdir('.'))[:300]
t = time.time()
for f in files:
    try: os.getxattr(f, 'user.DOSATTRIB')
    except OSError: pass
print('%.3f s' % (time.time() - t))
PY
sudo mount -o remount,nouser_xattr /path/to/mount     # try it, and user_xattr undoes it

With nouser_xattr the kernel answers EOPNOTSUPP at once and the 300 calls took under a millisecond. Add the option to the mount options (pam_mount.conf.xml, /etc/fstab) to keep it after the next login. The price is that Wine can no longer store DOS attributes (hidden, read only, system) on that share. The contents of the files are not affected.

Reading the start of each file is slow on a cold cache as well. Time 300 files with a plain open and read of a few hundred bytes. On the share used here it was 18 ms per file the first time and 3 ms the second. A fix has to avoid the per file work, because mount options don't help.

Directory listing order

A program that lists a directory with FindFirstFile/FindNextFile (Delphi FindFirst) got the entries in alphabetical order under Wine, even when the folder was on btrfs and its subfolders had been created in a different order. ls -U on the same folder showed the creation order. Wine appears to sort the entries the way NTFS returns them, so the order in which you create files or folders can't be used to steer a program that processes them in listing order. Rename them instead.

Driving the user interface

To reproduce a bug or run a test through the screens of a program, xdotool sends clicks and keys and ImageMagick's import captures a window. Wine windows are X11 windows, so this also works in a Wayland session through XWayland. Both act on the real mouse and keyboard, so ask the person at the machine to leave them alone while a script runs.

wmctrl -lG                                       # window ids, titles and geometry
import -window 0x01a00159 shot.png               # capture one window, without decorations
xdotool windowactivate --sync 0x01a00159
xdotool mousemove --window 0x01a00159 205 39 click 1
xdotool type --delay 40 '1234'; xdotool key Tab
magick shot.png -crop 900x60+0+500 part.png      # zoom into a region before reading it
  • Use coordinates relative to the window (mousemove --window) and read them from a capture of that window, where pixel (x, y) of the image is the same point. With display scaling, wmctrl -lG and xdotool getwindowgeometry report the same window at different absolute positions (4688 against 2344 at 200%), so absolute coordinates are easy to get wrong.
  • After every action that can open a message box, list the windows again with wmctrl -l and look for a new one before typing anything. A modal dialog takes the keys, and xdotool getactivewindow doesn't always return it, so a script that captures "the active window" can miss the dialog and keep typing into the wrong place. Capture the dialog by its id, answer it, and only then go on.
  • Don't rely on a fixed sleep after an action that loads data. A record that took 10 seconds to load swallowed the keys typed after 3 seconds, and they ended up in another field. Capture again until the screen shows the loaded data.
  • A grid can keep showing an edited value after it was saved or reverted. Check the result in the data the program writes (a database, a file) rather than on the screen, and reload the record before editing it again. Some screens only write when a save button is pressed, not when the field loses focus, and a child dialog's "OK" may only take effect after the save button of the parent screen.
  • Clicking a VCL edit box doesn't select its text, and ctrl+a doesn't select all in it, so typing appends to the old value. A record number field kept its digits and the new ones went after them, and the program asked whether to create a record out of sequence. Clear the field with End and enough BackSpace, or reach it with Tab, which selects the whole text of the field it lands on.
  • A click can land on the neighbouring field and take the typing with it, for example a code that ended up inside a date field and came back as an invalid date. After clicking a field, capture it and check the focus before typing.
  • A form with a horizontal scroll bar scrolls when a field near its edge gets the focus, so every coordinate read from the previous capture moves. Capture again after each click on such a form.
  • In a data grid (TDBGrid), Down on the last row usually posts it and appends a new row (once it appended nothing, and the row came only with Insert), and Escape discards an appended row that is still empty. Insert, or a click on a cell of the empty row, makes the grid validate that row at once and pops up a "required field" message. To reach a column, press Tab a counted number of times from a column you know instead of clicking, because the grid scrolls sideways as the focus moves.
  • Message boxes often share one title ("Confirmation", "Warning") and a progress window can stay open under them. Tell them apart by size with xdotool getwindowgeometry and wait in a loop for the next window that isn't the progress one:

    for i in $(seq 1 30); do sleep 2
      for d in $(wmctrl -l | awk 'NR>2{print $1}'); do        # skip the main windows
        g=$(xdotool getwindowgeometry $d | grep Geometry)
        case "$g" in *260x92*) ;;                             # size of the progress window
          *) import -window $d next.png; echo "$d $g"; break 2;; esac
      done
    done

    Click the buttons by coordinates inside the dialog (mousemove --window $d), read from its capture.

  • To put a long text into an editor, set the clipboard and paste instead of typing. It takes no time, and auto-indent and accented letters don't change what arrives. On KDE, Klipper takes and returns the text over D-Bus, and the paste reaches the Wine window through XWayland:

    mine=$(qdbus6 org.kde.klipper /klipper getClipboardContents)        # the person's clipboard
    qdbus6 org.kde.klipper /klipper setClipboardContents "$(sed 's/$/\r/' case.txt)"   # CRLF
    xdotool key ctrl+Home; xdotool key ctrl+shift+End; sleep 0.3; xdotool key ctrl+v

    Select with ctrl+Home and ctrl+shift+End. ctrl+a can be the shortcut of a menu item, and in one editor it opened a dialog that Escape didn't close. Reading back the same way (ctrl+c, then getClipboardContents) saves the original text before the first test and shows with cmp that it was put back after the last one. Restore the person's clipboard at the end.

  • An open menu, a drop-down list or a hint is a window of its own and isn't in the capture of the main window. Capture the whole screen with the session's own tool (spectacle -b -n -f -o full.png on KDE, since import -window root fails under Wayland) and crop the area. The menu shows the shortcut of each item, and the shortcut is more reliable than clicking the menu again.
  • A capture taken right after a window opens can come out black, and a panel can still show the result of the previous run. A pause between the action and the capture fixed both (0.6 s before capturing a dialog, 1.3 s for a message panel that redraws after the dialog closes). When two cases in a row show the same result, suspect the capture before the program.
  • For a batch of cases, crop from each capture only the strip that holds the result, put a label above it and append the strips, so that thirty runs are read as one image:

    magick -background '#ddeeff' -fill black -pointsize 13 label:"$name" label.png
    magick shot.png -crop 1150x34+0+681 +repage strip.png
    magick label.png strip.png -append "case-$n.png"
    magick case-*.png -append all.png

Text widths that differ from Windows

A program that measures text and compares the width with a stored value, for example to check that the fonts of a document are the ones it was made with, fails by one or two pixels under Wine. FreeType's TrueType interpreter version 40 hints only vertically. Windows GDI uses the full hinting of version 35, so the advance widths differ.

FREETYPE_PROPERTIES=truetype:interpreter-version=35 wine app.exe

Measure before and after with a small program, built with i686-w64-mingw32-gcc medir.c -o medir.exe -lgdi32 -luser32, that calls GetTextExtentPoint32A for a few strings and heights:

HDC dc = GetDC(NULL);
HFONT f = CreateFontA(-12, 0, 0, 0, FW_NORMAL, 0, 0, 0, DEFAULT_CHARSET, 0, 0, DEFAULT_QUALITY, 0, "Arial");
SelectObject(dc, f);
SIZE s; GetTextExtentPoint32A(dc, text, lstrlenA(text), &s);   // s.cx is the width in pixels

For Arial, a 54 character line at 12 px measured 314 px by default and 312 px with version 35, and at 16 px 409 against 396. That removed a 2 pixel mismatch that failed a document check. The glyphs look different with version 35, which is the price of the same widths. Set the variable only for the runs that need it, in a second launcher. The metric compatible fonts of the original (for example the corefonts package) have to be installed in the prefix too.

Run the program from wine in the same prefix as the application while it is open. It shares the wine server and doesn't disturb it.

A program that closes or crashes

Wine writes nothing by default when a program disappears. Start it from a shell with the logging you need, and redirect stderr to a file.

  • WINEDEBUG=err+all prints the unhandled exceptions Wine reports, usually nothing for a clean exit.
  • WINEDEBUG=err+all,+seh logs every exception the program raises, including the ones it handles on purpose. A Delphi program raises many, so the file grows to tens of megabytes. The last lines show which handler was running when it died. An access violation with ecx=cafebabe is a Java thread that the JVM handles by itself and is not your crash.
  • One exception takes several lines of a +seh log (dispatch_exception, the registers, the handlers called, RtlUnwind), so grep -c c0000005 counts lines, not exceptions. Count the distinct addr= values instead. An exception that a program handles is not necessarily caused by your change. Note the module it happened in, run the previous file and the new one again, more than once, and compare. A handled access violation during a login that talks to a network service came and went between runs of the same files.
  • WINEDEBUG=err+all,+process is small and logs process creation and exit.
  • strace -f -qq -o trace.txt -e trace=exit_group,kill,tgkill -e signal=SIGSEGV,SIGKILL,SIGABRT wine app.exe shows how each process ended. exit_group(0) is a normal exit. +++ killed by SIGSEGV (core dumped) +++ is a native crash, and the SIGSEGV lines before it give si_addr. A first fault at a null address followed by one at a protected address is a crash while Wine was already handling the first, so look at what the program was drawing or writing at that moment.
  • When a patched program dies, bisect with a variant that follows the same path but uses a constant where the real data goes. See the binary-patch skill.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.