agentleFS
SensorsIotSensorsIot/Embedded-AI-Harness/.claude/skills/testbench-logging/SKILL.md

testbench-logging

Skill235 starsChanged 3 months agoMITView on GitHub

Use this skill whenever you need to read serial output or debug logs from ESP32 devices on the testbench. Covers serial monitor with pattern matching (wait for boot messages, WiFi connected, crash dumps), UDP debug log retrieval when USB is occupied (e.g. HID keyboard), boot capture, and crash analysis. Use for verifying firmware started correctly, checking WiFi connection status, or diagnosing boot loops. Triggers on "serial monitor", "log", "debug log", "UDP log", "boot output", "crash", "monitor", "pattern", "serial output".

  • Sends data out

The file

We split this skill into the rules it gives an agent

5 rules. About 0 are common, 3 are distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈3

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Stop it before capturing boot output:

    line 79
  2. Very specific:

    Check first with debugging in /api/devices — a slot reads idle either way.

    line 86
  3. Distinctive:

    Serial is the lifeline: Never decide whether a device is alive by pinging it or calling its HTTP endpoint — a device that boots fine but …

    line 88
  4. Very specific:

    Check for expected boot messages (e.g., boot:0x28, firmware version)

    line 210
  5. Distinctive:

    If empty, device may not have UDP logging — fall back to serial monitor

    line 222

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

ESP32 Debug Logging

Base URL: $TESTBENCH_URL — see Step 0

Step 0: Point at a bench

There are several benches and their addresses move, so nothing here writes one down. $BENCH is not usable either — a container cannot resolve mDNS. Discover the bench and export its URL:

export TESTBENCH_URL=$(sudo python3 .claude/skills/esp-idf-handling/discover-testbench.py \
                         --url --name <bench-hostname>)
curl -s "$TESTBENCH_URL/api/info"        # confirm before anything else

--url refuses to guess when more than one bench answers, so --name is required whenever a second bench is powered on. TESTBENCH_URL is the same variable pytest --wt-url falls back to.

Two logging methods are available. Choose based on your situation:

Serial MonitorUDP Logs
Works without WiFiYesNo
Boot/crash outputYesNo
Pattern matchingBuilt-in (regex + timeout)Manual (poll + grep)
Blocks serial portYes (one session per slot)No
Multiple devicesOne slot at a timeAll devices simultaneously
Long-runningLimited by timeoutContinuous (buffer persists)

Endpoints

Request and response shapes: serial in FSD Appendix D.2, UDP log in D.7, the portal's own activity log in D.14.

Three different logs, and picking the wrong one wastes a test run: /api/serial/* is what the device printed over USB, /api/udplog is what it sent over the network, and /api/log is what the portal did — never device output.

Serial Monitor

Reads serial output via RFC2217 proxy. Optionally waits for a regex pattern.

# Wait up to 10s for a pattern match
curl -X POST $TESTBENCH_URL/api/serial/monitor \
  -H 'Content-Type: application/json' \
  -d '{"slot": "SLOT1", "pattern": "WiFi connected", "timeout": 10}'

# Just capture output for 5s (no pattern)
curl -X POST $TESTBENCH_URL/api/serial/monitor \
  -H 'Content-Type: application/json' \
  -d '{"slot": "SLOT1", "timeout": 5}'

Response: {"ok": true, "matched": true, "line": "WiFi connected to MyAP", "output": [...]}

A debug session silently changes what reset returns

POST /api/serial/reset picks its method from the slot. With no debug session it pulses DTR/RTS and returns the boot log as a list of lines. With OpenOCD attached it issues a JTAG reset run instead and returns a single string of OpenOCD's reply — JTAG tap: esp32c3.tap0 ... — which is not the device's output at all. Same endpoint, same request, two response shapes and two meanings.

Nothing in the response announces this except a "method": "jtag" key that is absent on the serial path. Client code that iterates output gets characters instead of lines and quietly finds no boot markers.

OpenOCD attaches by itself when a device is plugged in, so this is the default state of a board on a JTAG-capable slot, not something you opted into. Stop it before capturing boot output:

curl -X POST $TESTBENCH_URL/api/debug/stop \
  -H 'Content-Type: application/json' -d '{"slot": "SLOT3"}'

Check first with debugging in /api/devices — a slot reads idle either way.

Serial is the lifeline. Never decide whether a device is alive by pinging it or calling its HTTP endpoint — a device that boots fine but never joins WiFi looks identical to a dead one. Reading what it printed tells you which. Boot-marker patterns for running / download-mode / unknown are in references/state-detection.md.

Serial cannot count restarts on a native-USB part

On a chip whose console is USB-CDC (ESP32-C3/S3/C6), esp_restart() resets the USB-JTAG peripheral, the devnode re-enumerates, and the RFC2217 proxy's file descriptor breaks. The capture socket stays open and simply goes quiet — and on reconnect it can replay stale scrollback from before the reset. A test that counted "boots seen" from serial once read 25 lines then nothing for 30 minutes while the device restarted six times, and a filtered view showed a clean pass.

Serial is authoritative only within one boot. For anything that counts restarts — a 5-minute-rule reboot, a watchdog reset, a crash loop — read a monotonic boot counter the firmware publishes (e.g. an MQTT status payload or UDP telemetry), not the console. A silent device and a dead probe look identical in any filtered serial view, so make the harness exit loud (non-zero with the raw captured-line count) rather than reporting boots: 0.

Use serial monitor when:

  • You need boot messages (before WiFi is up)
  • You need to wait for a specific log line (pattern matching with timeout)
  • Device has no WiFi or UDP logging is not compiled in
  • You want crash/panic output from the UART

Dual-USB hub boards

  • Reset via the JTAG slot (triggers DTR/RTS auto-download circuit)
  • Monitor via the UART slot (where ESP_LOGI output appears)
  • Boot output on the JTAG slot will be empty or minimal — the actual boot log appears on the UART slot
# Reset via JTAG slot
curl -X POST $TESTBENCH_URL/api/serial/reset \
  -H 'Content-Type: application/json' \
  -d '{"slot": "<JTAG-slot>"}'

# Capture boot output from UART slot
curl -X POST $TESTBENCH_URL/api/serial/monitor \
  -H 'Content-Type: application/json' \
  -d '{"slot": "<UART-slot>", "timeout": 10}'

UDP Logs

ESP32 firmware sends debug logs as UDP datagrams to the testbench on port 5555. The testbench buffers up to 2000 lines.

# Get recent UDP logs (default limit: 200)
curl -s $TESTBENCH_URL/api/udplog | jq .

# Filter by source device IP
curl -s "$TESTBENCH_URL/api/udplog?source=192.168.4.2" | jq .

# Get logs since a timestamp, limited to 50 lines
curl -s "$TESTBENCH_URL/api/udplog?since=1700000000.0&limit=50" | jq .

# Clear the buffer before starting a test
curl -X DELETE $TESTBENCH_URL/api/udplog

Response format: {"ok": true, "lines": [{"ts": 1700000001.23, "source": "192.168.4.2", "line": "OTA progress: 45%"}, ...]}

Use UDP logs when:

  • Device is on WiFi and firmware sends UDP log packets
  • You want non-blocking log collection (doesn't tie up the serial port)
  • You're monitoring multiple devices simultaneously (filter by source IP)
  • You need logs during OTA updates (serial may be unavailable)

Do NOT use UDP logs when:

  • Device has no WiFi yet (pre-provisioning, boot phase) — use serial monitor
  • Firmware doesn't include UDP logging — use serial monitor
  • You need boot/crash output — only serial captures UART output from early boot
  • You need to wait for a specific pattern with a timeout — serial monitor has built-in pattern matching

How ESP32 Sends UDP Logs

The testbench listens on UDP port 5555. ESP32 firmware sends plain text lines:

/* inet_aton() parses dotted-quad only — it does NOT resolve names, and on
   failure it leaves the address at 0.0.0.0, so the logs vanish silently.
   Always check the return value. */
struct sockaddr_in testbench = { .sin_family = AF_INET, .sin_port = htons(5555) };
if (inet_aton(CONFIG_TESTBENCH_IP, &testbench.sin_addr) == 0) {   /* dotted-quad */
    ESP_LOGE(TAG, "bad testbench address");
    return ESP_ERR_INVALID_ARG;
}
sendto(sock, msg, strlen(msg), 0, (struct sockaddr *)&testbench, sizeof(testbench));

To use a name instead of an IP, resolve it first — inet_aton will not do it for you:

struct addrinfo hints = { .ai_family = AF_INET, .ai_socktype = SOCK_DGRAM };
struct addrinfo *res;
if (getaddrinfo("testbench", "5555", &hints, &res) == 0) {
    sendto(sock, msg, strlen(msg), 0, res->ai_addr, res->ai_addrlen);
    freeaddrinfo(res);
}

Resolving the .local form needs mDNS, which is not bundled in ESP-IDF 6 — add the espressif/mdns managed component if you want it. Passing an IP literal avoids the dependency entirely, which is why the test firmware does that.

Activity Log

The activity log tracks testbench actions (resets, WiFi changes, firmware uploads) — not device output.

curl -s $TESTBENCH_URL/api/log | jq .
curl -s "$TESTBENCH_URL/api/log?since=2025-01-01T00:00:00Z" | jq .

Common Workflows

  1. Verify boot after flash:
    • POST /api/serial/reset — returns boot output, but only with debug stopped (see below)
    • Check for expected boot messages (e.g., boot:0x28, firmware version)
  2. Wait for WiFi connection:
    • POST /api/serial/monitor with pattern: "WiFi connected" and timeout: 15
  3. Monitor OTA progress:
    • DELETE /api/udplog — clear buffer
    • Trigger OTA (see esp-idf-handling)
    • Poll: GET /api/udplog?since=<last_ts>&limit=50
  4. Debug a running device:
    • GET /api/udplog?source=<device_ip> — see what it's logging
    • If empty, device may not have UDP logging — fall back to serial monitor

Troubleshooting

ProblemFix
Monitor timeout, no outputBaud rate is fixed at 115200; ensure device matches. For dual-USB boards: make sure you're monitoring the UART slot, not the JTAG slot
No UDP logs appearingEnsure firmware sends UDP to testbench IP:5555; check WiFi connectivity
Logs from wrong deviceUse source query param to filter by IP
Old/stale logsClear with DELETE /api/udplog before starting a test
Need boot outputUDP logs don't capture boot — use serial monitor. For dual-USB boards, monitor the UART slot
Slot shows monitoringAnother monitor session is active — wait for it to finish

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.