Debugging LuisaCompute

1. Interpreting Stack-Traces

When a crash or LUISA_ERROR is emitted, capture the full console output first.

What to look for:

• Top frames — the actual fault (null dereference, assertion, backend error).
• LuisaCompute frames — functions prefixed with luisa::, especially luisa::compute:: and the DSL inline namespace luisa::compute::dsl::.
• Backend frames — plugins load as luisa-backend-<name>; the installed backends are cuda, dx, fallback, hip, metal, metal4, simd, vk. There is no cpu backend — the CPU paths are fallback (native C++/JIT) and simd.
• Last log line — often the preceding LUISA_INFO/LUISA_VERBOSE shows the dispatch or shader name that triggered the bug.

What the trace itself looks like (src/core/platform.cpp, include/luisa/core/stl/format.h):

• LUISA_ERROR → luisa::log_error appends the frames to the error message itself ( NN [0xADDR]: module :: symbol + offset), then flushes and calls std::abort() — so expect a second dump from the abort handler.
• Windows installs SetUnhandledExceptionFilter, std::set_terminate and a SIGABRT handler (static StackTracerInit), which print !!! Unhandled structured exception !!!, !!! std::terminate called (uncaught exception) !!! or !!! SIGABRT / std::abort() called !!!, followed by ----- Stack Trace (N frames) ----- rows of [ii] symbol at module+0xoff.
• Frames come from luisa::backtrace() — dbghelp (CaptureStackBackTrace + SymFromAddr) on Windows, ::backtrace/backtrace_symbols on Unix. On Windows this whole path is #ifndef NDEBUG: a Release build yields no frames at all, so reproduce under -m debug or -m releasedbg first.

Action:

1. Read the innermost frame (first after the crash header). This is the immediate cause.
2. Walk upward until you hit a recognizable LuisaCompute API call (e.g., Device::compile, Buffer::copy_to/copy_from, Stream::synchronize, or the shader(args...).dispatch(...) command built by ShaderInvoke<N>::dispatch). That is the call-site.
3. If the trace ends inside a driver/shared library, suspect (a) invalid resource usage (out-of-bounds buffer/image access), or (b) backend-specific limitation.

2. Plan Before Fixing

Once the stack-trace points to a file/line or API call, write a debug plan in this order:

1. Hypothesis — state what you believe caused the failure in one sentence.
2. Verification — describe the smallest code change or log addition that can confirm/disprove the hypothesis.
3. Fix strategy — if verified, what exactly will you change.
4. Rollback marker — note the original state so you can undo cleanly.

If the fix fails:

• Save the failed attempt with memory.
• Re-read the stack-trace and the saved steps. Do not repeat a failed hypothesis.
• Pick the next most likely cause and repeat from step 1.

3. When There Is No Stack-Trace

Silent failures (hang, wrong result, test timeout) provide no trace.

Distinguish hang types by CPU usage first:

• High-CPU "hang" (process burns a core indefinitely) is usually a spin on corrupted state, e.g. a spin_mutex inside a freed object — a UAF that looks like a hang. Treat it as a memory bug: audit recent lifetime changes and destructors that unlink cross-object references.
• Zero-CPU hang is a genuine wait (GPU fence, deadlocked mutex).

For multi-test binaries, isolate by running each test alone (Boost.UT takes positional name patterns after the backend arg: bin/debug/test_basic_types.exe vk test_vector_construction, i.e. xmake run test_basic_types vk test_vector_construction). When a fix candidate emerges, A/B it with git stash — a failure that persists with the fix stashed is pre-existing and out of scope; do not chase it.

Cheapest ways to make a silent failure talk:

• Run with LUISA_ENABLE_VALIDATION=1: Context::create_device wraps the backend device in the luisa-validation-layer plugin, so API misuse turns into a logged LUISA_ERROR (with trace) instead of driver-side corruption.
• Build with safe mode — lc_safe_mode (xmake) or LUISA_COMPUTE_ENABLE_SAFE_MODE=ON (CMake), which defines LUISA_ENABLE_SAFE_MODE. Resource constructors then LUISA_ERROR on an invalid handle at the creation call (e.g. include/luisa/runtime/buffer.h, src/runtime/stream.cpp) instead of failing later.
• Remember that a Release (NDEBUG) Windows binary cannot print frames at all (Section 1) — 'no stack trace' may just be the build mode.

Find the entry point:

• Read CMakeLists.txt or xmake.lua near the failing target to locate the executable source file and its main().
• Identify the test harness (e.g., test_device.h, boost::ut) and how the device is created.

Add host-side logging:

#include <luisa/core/logging.h>

// In host code (C++ runtime)
LUISA_VERBOSE("Entering {}::{}", __FILE__, __func__);
LUISA_INFO("Buffer size = {}", buf.size());
LUISA_VERBOSE_WITH_LOCATION("Dispatching kernel X");

Set log level early (before Context creation if possible):

luisa::log_level_verbose();  // or log_level_info()

Progressive narrowing:

1. Log at the start of main() and at every major phase (context → device → stream → compile → dispatch).
2. If the failure happens during a kernel dispatch, move to device-side logging (Section 4).
3. If the failure is a wrong numerical result, move to buffer read-back (Section 5).

4. DSL / Device-Side Logging

Inside kernels, use device_log (include/luisa/dsl/stmt.h, lowered to FunctionBuilder::print_) to emit per-thread messages. They are collected by the stream and flushed to the host callback (Stream::set_log_callback); with no callback installed the backends print them through the host logger as LUISA_INFO("[DEVICE] {}", ...).

Basic usage:

#include <luisa/dsl/syntax.h>
#include <luisa/dsl/sugar.h>

Kernel2D k = [&]() noexcept {
    UInt2 coord = dispatch_id().xy();
    $if (coord.x == 1) {
        device_log("hello {} {}", coord, make_float3x3());
    };
};

Custom log callback on the stream:

Stream stream = device.create_stream();
stream.set_log_callback([](luisa::string_view message) {
    LUISA_INFO("device: {}", message);
});
stream << shader().dispatch(128u, 128u) << synchronize();

Structured severity prefixes (for custom routing):

// Severity prefixes + dispatch-id pattern: src/tests/unit/runtime/test_printer_custom_callback.cpp
// (it also emits a 'V' prefix and appends `$dispatch_id` to every message)
#define DEVICE_INFO(FMT, ...) \
    device_log(luisa::format("I" FMT) __VA_OPT__(, ) __VA_ARGS__)
#define DEVICE_WARNING(FMT, ...) \
    device_log(luisa::format("W" FMT) __VA_OPT__(, ) __VA_ARGS__)
#define DEVICE_ERROR(FMT, ...) \
    device_log(luisa::format("E" FMT) __VA_OPT__(, ) __VA_ARGS__)

stream.set_log_callback([](luisa::string_view msg) {
    if (!msg.empty()) {
        switch (msg.front()) {
            case 'I': luisa::log_info("{}", msg.substr(1)); break;
            case 'W': luisa::log_warning("{}", msg.substr(1)); break;
            case 'E': luisa::log_warning("device error: {}", msg.substr(1)); break;
            default:  luisa::log_verbose("{}", msg); break;
        }
    }
});

Never route a device message to luisa::log_error/LUISA_ERROR from the callback: it is [[noreturn]], prints a host backtrace and calls std::abort() (include/luisa/core/logging.h), so one device-side 'E' line kills the host process mid-flush.

No build switch or environment variable is needed for device logging: device_log is lowered to FunctionBuilder::print_, each backend ships the format strings with the compiled shader (printers), and the host decodes the log buffer in format_shader_print (src/backends/common/shader_print_formatter.h). There is no enable_logging, logging_buffer, or buffer_printf API in this codebase.

Important: Device logs are asynchronous. Always synchronize() the stream before assuming all logs have arrived. If a kernel hangs, the callback may never fire for logs buffered inside the failing dispatch.

5. Using Buffer for DSL Debug

When you need to inspect many values or avoid per-thread log flooding, write results into a Buffer and read back on the host.

Buffer-based inspection:

#include <luisa/core/stl/vector.h>
#include <luisa/dsl/syntax.h>
#include <luisa/dsl/sugar.h>

Buffer<float4> debug_buf = device.create_buffer<float4>(1024);

Kernel1D k = [](BufferVar<float4> out) noexcept {
    UInt idx = dispatch_id().x;
    Float4 v = make_float4(cast<float>(idx),
                           cast<float>(idx) * 2.0f,
                           cast<float>(idx) * 3.0f,
                           0.0f);
    out.write(idx, v);
};

auto shader = device.compile(k);
stream << shader(debug_buf).dispatch(1024)
       << synchronize();

// Read back
luisa::vector<float4> host(1024);
stream << debug_buf.copy_to(luisa::span{host}) << synchronize();
for (size_t i = 0; i < 8; ++i) {
    LUISA_INFO("host[{}] = {}", i, host[i]);
}

Counter + slot pattern for conditional values:

• Allocate a Buffer<uint> (e.g. counter_buf) plus a payload Buffer<T> of the size you want to capture.
• In the kernel, take a slot atomically and write through it: auto slot = counter_buf.atomic(0u).fetch_add(1u); debug_buf.write(slot, payload); — BufferVar exposes read(i) / write(i, v) / atomic(i), not operator[] (include/luisa/dsl/resource.h, include/luisa/dsl/atomic.h; live example: src/tests/unit/coro/test_coro_wavefront.cpp:1071).
• Guard the slot (if (slot < N)) so a flooded kernel cannot write past the buffer.
• This captures the first N interesting threads without over-allocating.

6. Kernel Restrict Mode: add_debug_checks

TEMPORARY DEBUG SCAFFOLDING — DELETE IT BEFORE SHIPPING. add_debug_checks rebuilds the whole function (and every custom callable it uses) with a guard around every operation, and a guard is a printed diagnostic, never a free assertion. Expect an extremely huge performance cost, far beyond a few percent:

• each guarded op adds comparisons/branches plus size-query and diagnostic plumbing, and forces device printing on (requires_printing()), which defeats vectorization, unrolling and dead-code elimination on the instrumented path;
• error propagation adds a trailing reference<uint> out-parameter to every transformed callable and a check at every call site (propagate_callable_errors);
• a faulting thread additionally pays for building and flushing a device log line (asynchronous, so a flooded kernel also floods the log buffer). Real dispatches can slow down by orders of magnitude, so an instrumented build is unusable for timing or throughput work. Therefore: instrument only to localize a bug, then remove every add_debug_checks(...) call (plus LUISA_AST_DEBUG_KERNEL=1, the debug kernel type, and its extra shader/dispatch) as soon as the bug is found — never commit it, never measure performance with it, and re-run the normal uninstrumented kernel to confirm the fix. The GPU-side cost is identical in Debug and Release, so do not expect a release build to make it cheap.

add_debug_checks(kernel) (include/luisa/dsl/func.h:736-749, pulled in by luisa-compute.h) rewrites a kernel and, transitively, every custom callable it uses so that every operation that can fail at runtime is guarded: on failure the guard prints a diagnostic through the device printer and the thread stops early (return_). A failure inside a callable is reported to its caller through a generated trailing reference<uint> error out-parameter (0 = ok, 1 = failed), so one bad value stops the whole kernel thread. This is the kernel restrict mode — use it to turn a silently corrupted, out-of-bounds or NaN-producing dispatch into an explicit device-side message plus a provable early stop. Implementation: src/ast/function_builder_debugger.cpp (its header comment is the definitive operator-by-operator check table, kept in sync with include/luisa/ast/op.h); options: include/luisa/ast/function_builder_debugger.h:15-48.

Canonical usage (from src/tests/integration/runtime/test_function_debugger.cpp):

#include <luisa/luisa-compute.h>

auto kernel = add_debug_checks(Kernel1D{[&](BufferVar<float> in, BufferVar<uint> out) noexcept {
    auto i = dispatch_id().x;
    auto v = in->read(i * 100u);   // BUFFER_READ bounds guard
    auto y = v / 0.0f;             // float-div-by-zero + NaN/Inf guard
    out->write(i, 1u);             // sentinel: must not run for a faulting thread
    static_cast<void>(y);
}});
// Guards call BUFFER_SIZE/BYTE_BUFFER_SIZE/ACCEL_SIZE, so compile with debug
// info (required on dx, harmless elsewhere).
auto shader = device.compile(kernel, {.enable_debug_info = true});

// Guards print through the device printer -> capture them on the stream.
stream.set_log_callback([](luisa::string_view message) { LUISA_INFO("device: {}", message); });
stream << shader(input, output).dispatch(4u) << synchronize();  // input/output are the host buffers

Guards emit lc-debug: <what> in <kernel> (...) lines. The callback receives the raw print payload (no severity prefix is added by the backends); match on a substring, e.g. "buffer index out of range" or "NaN/Inf result". (The reference test's helper drops the first byte of every message, a leftover from test_printer_custom_callback.cpp where the kernel prepends its own severity letter — so it deliberately never matches lc-debug: itself.)

What is checked by default (detail::DebugKernelOptions):

Opt-in / not checked:

• check_texture_bounds{false} — TEXTURE_READ/WRITE coordinates vs TEXTURE_SIZE (writes are clamped on some backends); check_ray_validity{false} — NaN/Inf ray origin/direction.
• Never guarded: atomics (the AtomicRef sits on an already-bound buffer), size queries (BUFFER_SIZE/TEXTURE_SIZE/ACCEL_SIZE), texture sampling uv/coord (hardware-clamped), int overflow (MINUS/ADD/SUB/MUL, off in v1), accel motion keys, and loop for-init/cond/update + $while provenance expressions (the AST has no per-iteration statement position — documented v1 limitation).
• Rejected outright with a clear error: autodiff (BACKWARD/GRADIENT), raster-stage RASTER_*, and cooperative/async/expert ops (COOPERATIVE_*, ASYNC_COPY, PIPELINE_*, MBARRIER_*, …).
• Zero overhead when there is nothing to check: a function whose ops are all safe is rebuilt statement-for-statement identical (the test asserts equal statement counts for a plain shared-memory kernel and for acc->intersect without instance access).

Tune the checks. add_debug_checks always uses the default DebugKernelOptions; for per-check control call the detail entry point and rebuild the kernel type:

namespace detail = luisa::compute::detail;
detail::DebugKernelOptions opts;
opts.check_accel_instance_index = false;  // backend without CallOp::ACCEL_SIZE
opts.check_texture_bounds = true;         // opt in
Kernel1D<BufferVar<float>, BufferVar<uint>> checked{
    detail::debug_function(*plain_kernel.function(), opts)};

Zero-touch variant: LUISA_AST_DEBUG_KERNEL=1 (also true/on; read once per process, src/ast/function_builder_debugger.cpp:1404-1408) makes FunctionBuilder::define_kernel run the generator on every kernel defined afterwards — non-kernel builders pass through unchanged (include/luisa/ast/function_builder.h:289-297). Fastest way to check a whole app without touching source — but it is process-wide, so unset it (or restart the shell) when done: as long as it is set, every kernel defined afterwards carries the guards and pays the huge cost described above.

Pre-flight sanity: a guarded kernel sets requires_printing() and advertises the size queries it needs via propagated_builtin_callables() (CallOp::BUFFER_SIZE / BYTE_BUFFER_SIZE / ACCEL_SIZE) — assert both when wiring a new backend, and skip the check whose query that backend does not implement.

Reference test: src/tests/integration/runtime/test_function_debugger.cpp (xmake test_proj, CMake luisa_compute_add_test) is the end-to-end spec of every guard; run xmake run test_function_debugger <backend> (the accel-structure cases need cuda/dx/vk/metal). It checks both the message and that faulting threads stop before their sentinel write — copy that pattern to reproduce a suspected OOB/NaN kernel.

7. Environment Variables for Backend Diagnosis

177 distinct LUISA_* names are read from the environment in src/ outside src/tests (114 of them LUISA_SIMD_* codegen toggles), plus the system MTL_*/METAL_* variables; enumerate them with grep -rn '"LUISA_' src/. The table lists the diagnosis-relevant subset. Booleans are read through luisa::compute::detail::env_flag (src/backends/common/env_flag.h:11-17: 1, true, TRUE, on, ON), while a few checks compare against exactly "1" (noted below). Set flags to 1 and you are always safe.

(luisa::log_level_verbose() etc. are the programmatic equivalents, include/luisa/core/logging.h:136-142. Build-time switches are not env vars: CMake LUISA_COMPUTE_* options and xmake lc_* options.)

Use LUISA_DUMP_SOURCE=1 when you suspect a code-generation bug (wrong instruction, missing binding, incorrect type).

Where to find the dumps:

• DirectX: hlsl_output_<name>.hlsl in the current working directory.
• Vulkan user compute (XIR→SPIR-V path): spv_code_<name>.spvasm in the current working directory.
• Vulkan user compute (LLVM→SPIR-V path): spv_code_llvm_<name>.spvasm.
• Vulkan internal HLSL consumers: backend builtins/raster may dump hlsl_output_<name>.hlsl; ordinary Device::compile(Function) compute shaders must not.
• CUDA: .cu source in the runtime .cache directory (write_shader_source); PTX + .metadata also land in .cache (write_shader_cache) — only explicitly named/AOT bytecode goes to the data directory itself (write_shader_bytecode). The .data directory holds internal shaders, not user PTX.
• Metal: .metal source in the runtime .cache directory; Metal4 dumps .metallib there.
• Fallback: does not read LUISA_DUMP_SOURCE; set LUISA_DUMP_XIR=1 / LUISA_DUMP_LLVM_IR=1 / LUISA_DUMP_ASM=1 to get kernel.<hash>.xir, .opt.xir, .ll, .opt.ll, .s in the current working directory.

The runtime directories are printed by LUISA_INFO at context creation; they default to the executable directory. When running under xmake run, dumps written directly to the current working directory will appear in the project root.

8. Decision Checklist

9. Windows Crash Debugging with scripts/debugger.py

A lightweight Python debugger using Windows Debug API + DbgHelp.dll (pure ctypes, no third-party packages) to launch an x64 executable under the debugger, print a symbolic stack trace for every exception event, and stop at the second-chance one. Output: [!] Caught exception: header (code/address/FirstChance), then === C++ stack trace (with PDB symbols) === rows #NN 0xADDR func followed by file:line when line info exists.

Usage:

python scripts/debugger.py <path_to_exe> [pdb_search_path] [-- <args>...]

• Arguments after -- are forwarded to the target executable.
• Symbols come from SRV*C:\Symbols*https://msdl.microsoft.com/download/symbols;<pdb_search_path>, so the PDB must sit next to the EXE or in pdb_search_path (system DLLs are fetched from the Microsoft symbol server).
• Prints a trace for first-chance exceptions too (breakpoints, MSVC C++ EH 0xE06D7363) and keeps going; only a second-chance exception ends the session.
• Works on Windows x64 with Python 3.x (64-bit recommended).

Example:

# tests take the backend as argv[1] (Boost.UT name patterns follow it) — no gtest here
python scripts/debugger.py bin/debug/test_runtime.exe -- dx

10. Tracking Memory Growth with scripts/mem_monitor.py

A cross-platform (Windows/Linux) Python wrapper that launches a process, samples its memory at a fixed interval, writes a timeline log, and kills the process tree when private memory exceeds a threshold — essential when a runaway test could exhaust machine RAM before you can observe it.

• Windows: Private Bytes / Working Set via GetProcessMemoryInfo; kills the tree with taskkill /F /T.
• Linux: private memory = Private_Clean + Private_Dirty from /proc/<pid>/smaps_rollup (fallback RssAnon + VmSwap), working set = VmRSS; kills the child's whole process group with SIGKILL (the child is started in its own session).

Usage:

python scripts/mem_monitor.py [--kill-gb N] [--interval S] [--log FILE] -- <exe> [args...]

• --kill-gb N (default 20): hard kill threshold in GiB of private memory (Private Bytes on Windows, smaps_rollup private on Linux). Pick a value that leaves the machine usable (e.g. total free RAM minus headroom).
• --interval S (default 0.25): sampling interval in seconds.
• --log FILE (default mem_monitor.log): timeline output, one time_s private_mb working_set_mb row per sample.
• Arguments after -- are forwarded to the target executable.
• On exit it prints a summary: exit code, whether it killed the process, peak private memory, and the first/last samples.

Example (run a device test, kill if it exceeds 6 GiB):

python scripts/mem_monitor.py --kill-gb 6 --interval 0.25 --log mem.log -- bin/debug/test_runtime.exe dx

Workflow for a suspected leak / memory blow-up:

1. Run the full repro under the monitor with a safe --kill-gb. A straight-line, ever-growing timeline (constant MiB/s) usually means an unbounded accumulation loop (e.g. appending commands to a CommandList that never terminates), not a classic leak; stepwise growth synchronized with test phases suggests per-phase leaks.
2. Isolate the phase: run individual tests (Boost.UT: positional test-name patterns after the backend arg, e.g. test.exe <backend> my_test) and compare peak memory and timeline shape.
3. A phase that terminates with flat memory is innocent; a phase whose memory grows without plateau owns the bug. Then read that phase's code for loops whose trip count can make zero progress (e.g. n = min(remaining, k) with k == 0).
4. After the fix, re-run under the monitor: peak private memory should be bounded (hundreds of MiB for small device tests) and the timeline flat.

Note: GPU/device memory is mostly invisible to host-side private memory; this tool tracks host-side growth. Pair with backend allocator logging if device-side exhaustion is suspected.

Summary

• Stack-traces → innermost frame = cause; upward walk = call-site.
• Always plan before editing; record each failed attempt so it is not repeated.
• No trace → read CMakeLists.txt/xmake.lua, add LUISA_INFO/LUISA_VERBOSE, then device_log.
• DSL values → prefer Buffer write + host read-back for bulk inspection; use device_log for targeted per-thread messages.
• Kernel OOB / NaN / div-by-zero / bad shift → add_debug_checks(kernel) + enable_debug_info (Section 6), or LUISA_AST_DEBUG_KERNEL=1 for the whole app: guards print lc-debug: messages and stop the faulting thread.
• Backend/codegen issues → set LUISA_DUMP_SOURCE=1 to inspect generated shaders and LUISA_ENABLE_VALIDATION=1 to catch API/resource misuse.
• Memory blow-up / suspected leak → run under scripts/mem_monitor.py with --kill-gb (Windows/Linux); constant-rate growth = unbounded accumulation loop, stepwise growth = per-phase leak.