/observability: bootstrap an APM stack, then instrument named methods

Two modes. Decide which from the argument before doing anything:

• No specific method(s) named, and the argument talks about setting up, wiring in, adding, or bootstrapping observability/APM/tracing infrastructure → Bootstrap mode.
• Specific method(s) named to trace, span, or instrument → Instrument mode.
• Ambiguous (e.g. "add observability to doStuff") → ask which is meant; bootstrapping and instrumenting are different-sized changes and guessing wrong wastes a diff.

Both modes touch only what's asked. Neither adds logging, metrics, or any dependency beyond the two co.elastic.apm artifacts.

Bootstrap mode

Copies a working, previously-verified-end-to-end (real transaction + child spans indexed, including from a private method, across Maven, Gradle — both DSLs — and JBang) Elastic APM stack into the current project, wires the two co.elastic.apm dependencies, and adds the one static block that turns tracing on.

Step 1: detect the build tool

• pom.xml at the project root → Maven.
• build.gradle or build.gradle.kts (± a gradlew wrapper) → Gradle.
• A standalone .java file with a //DEPS line or a ///usr/bin/env jbang "$0" "$@" ; exit $? shebang, and no pom.xml/build.gradle* nearby → JBang. For JBang there's usually no separate "project root" — the script file is both the build config and the entry point.
• Anything else (npm, a bare Gradle project with neither file recognized, ambiguous mix) → stop and say so. Don't improvise wiring for a tool this skill hasn't verified.

Step 2: copy the stack

Copy every file from this skill's templates/ directory into <project-root>/observability/ (for JBang, alongside the script file, since there's usually no other root), with one rename: templates/env → observability/.env (must be dot-prefixed for Docker Compose to auto-load it; kept without the dot in the skill bundle only so it isn't treated as a hidden file there).

If <project-root>/observability/ already exists, don't overwrite silently — diff against the template and ask before replacing anything, since it may hold project-specific edits (custom ports, a real (non-dev) password, etc.).

Step 3: wire the dependency — build-tool specific

Maven (pom.xml):

• Add (or reuse, if a <properties> block already sets an APM version) <elastic-apm.version> set to the APM_AGENT_VERSION value from the .env you just copied.
• Add two dependencies if not already present — check for any existing co.elastic.apm groupId first to avoid duplicates:

<dependency>
    <groupId>co.elastic.apm</groupId>
    <artifactId>apm-agent-attach</artifactId>
    <version>${elastic-apm.version}</version>
</dependency>
<dependency>
    <groupId>co.elastic.apm</groupId>
    <artifactId>apm-agent-api</artifactId>
    <version>${elastic-apm.version}</version>
</dependency>

Gradle (build.gradle — Groovy DSL, verified):

ext {
    elasticApmVersion = '<APM_AGENT_VERSION from .env>'
}

dependencies {
    implementation "co.elastic.apm:apm-agent-attach:${elasticApmVersion}"
    implementation "co.elastic.apm:apm-agent-api:${elasticApmVersion}"
}

Gradle (build.gradle.kts — Kotlin DSL, verified):

val elasticApmVersion = "<APM_AGENT_VERSION from .env>"

dependencies {
    implementation("co.elastic.apm:apm-agent-attach:$elasticApmVersion")
    implementation("co.elastic.apm:apm-agent-api:$elasticApmVersion")
}

Match whatever configuration name (implementation, api, the legacy compile) the project's existing dependencies {} block already uses instead of assuming implementation, if it has other entries to go by. If the project uses a version catalog (libs.versions.toml), don't guess how to slot into it — ask, since getting that format wrong silently breaks the catalog for everyone else using it. Prefer ./gradlew over a bare gradle invocation everywhere below if a wrapper is present.

JBang (single script file): add two //DEPS lines near the top, after any existing directives/shebang and before the imports:

///usr/bin/env jbang "$0" "$@" ; exit $?
//DEPS co.elastic.apm:apm-agent-attach:<APM_AGENT_VERSION from .env>
//DEPS co.elastic.apm:apm-agent-api:<APM_AGENT_VERSION from .env>

No separate build file to edit — Steps 3 and 4 land in the same script.

Step 4: attach the agent from the entry point

Maven/Gradle: find the class with public static void main(String[] args). If there's more than one (multi-module, multiple entry points) or none at all, ask which class should host the attach call rather than guessing or inventing a new Main.

JBang: the entry point is the script's own public class (same name as the file).

Add a static initializer block as the first member of that class — before any fields or other static blocks, so it runs before anything else in the class can touch tracing:

import co.elastic.apm.attach.ElasticApmAttacher;

import java.util.Map;

public class Main {
    static {
        ElasticApmAttacher.attach(Map.of(
                "service_name", "<pom.xml artifactId / Gradle settings.gradle rootProject.name / JBang script filename>",
                "environment", "dev",
                "server_url", "http://localhost:8200",
                "secret_token", "<APM_SECRET_TOKEN from the copied .env>",
                "application_packages", "<the entry-point class's package>",
                // log correlation: reformats existing log output (JUL/Log4j/Logback,
                // whatever's in use) to ECS JSON with trace.id/transaction.id baked in,
                // and ships it to APM Server directly - no Filebeat, no log4j2.xml edits.
                // log_sending is experimental and buffers in memory: fine for local dev,
                // not a durable delivery guarantee.
                "enable_log_correlation", "true",
                "log_ecs_reformatting", "OVERRIDE",
                "log_sending", "true"
        ));
    }

    public static void main(String[] args) {
        // existing body, unchanged
    }
}

application_packages must cover every package you might later instrument (see Instrument mode below) — if the app spans multiple unrelated base packages, use a comma-separated list rather than guessing one.

JBang default-package gotcha (verified, not obvious from the docs): a JBang script's class usually has no package declaration, so its fully-qualified name is just its simple class name. application_packages matches as a prefix against that FQN — so for a default-package script, set application_packages to the class's simple name, not a package guess. Confirmed with a negative control: the same script with a mismatched application_packages value produced zero @CaptureSpan/@CaptureTransaction weaving (silently — it still compiled and ran), while setting it to the class's own name wove both annotations and produced real spans.

Log correlation and an existing logging framework — don't stack configs. If the target project already has Log4j2 or Logback wired up with its own ECS layout / pattern that injects trace.id/transaction.id (someone hand-configured it before), do not also enable log_ecs_reformatting=OVERRIDE on top of that — the two together double up the correlation fields in every log line (trace.id appears twice, once from each mechanism). Pick one lane: either leave their existing manual config alone and skip these three attach keys, or use OVERRIDE/log_sending here and don't touch their logging config file. When in doubt, ask rather than guess which lane the project is already in — check for an existing EcsLayout or a %X{trace.id} pattern before adding these keys.

log_ecs_reformatting=OVERRIDE and log_sending=true need no application dependency and no logging-config file edit — they work by having the agent reformat/ship whatever the app already writes through JUL, Log4j (1.x/2.x), or Logback. Don't add a logging framework or an ECS-layout dependency to make this work; if the project has no logging at all yet, these keys are just inert until it does.

Step 5: verify

• Maven: mvn -q compile (or ./mvnw if wrapped).
• Gradle: ./gradlew compileJava if a wrapper exists, else gradle compileJava.
• JBang: jbang <script>.java runs it directly (that is the compile step, effectively); jbang build <script>.java if you want compile-only without running.

If it's a runnable app, running it once (even with the stack not up yet) should show Starting Elastic APM ... as <service_name> in stderr and then connection-refused/backoff noise, not a crash — that's expected and confirms attach worked. Report what you did: files added, the build-file diff, which class got the static block, and the service name / packages you used — don't just say "done."

Instrument mode

Parse the argument for the root method (becomes a transaction — usually the one called out as "a new transaction," or the outermost of the named methods) and any child method(s) (become spans, nested under whichever named method calls them). Touch only the methods named — don't walk the call graph and instrument neighbors "for completeness."

Prerequisites — check before writing any code

1. co.elastic.apm:apm-agent-api must be on the classpath. If Bootstrap mode hasn't run in this project, it won't be — check the build file (pom.xml, build.gradle*, or the script's //DEPS lines) first, and if it's missing, say so before adding it (or offer to run Bootstrap mode).
2. An agent must actually be attached at runtime (Bootstrap mode's static block, or an equivalent -javaagent setup). The manual API degrades to a harmless no-op without this and never breaks the build — but annotations only take effect through the agent's bytecode weaving, so with no agent attached they're silently inert.
3. application_packages must cover the target class's package. If it doesn't, flag it before adding @Capture* annotations — they'll compile and do nothing, which is a worse failure mode than an error.

Default: annotations

import co.elastic.apm.api.CaptureTransaction;
import co.elastic.apm.api.CaptureSpan;

@CaptureTransaction               // name defaults to ClassName#methodName, type defaults to "request"
public void doStuff() {
    // original body, unchanged
}

@CaptureSpan                      // name defaults to ClassName#methodName, type defaults to "app"
private void helper() {
    // original body, unchanged — this is bytecode weaving of the method body,
    // not a proxy, so it works on private methods too (verified end-to-end)
}

Set explicit attributes only when the description implies specifics: type = "background" on @CaptureTransaction for non-request-like work; type/subtype/action on @CaptureSpan only when the method visibly does I/O of a specific kind ("db", "external", "cache").

Annotations can't express a span that only starts under a runtime condition, a name built from an argument value, or a span wrapping part of a method rather than the whole thing. If the description asks for one of those, say so and use manual mode for that method instead of forcing it.

Manual mode: co.elastic.apm.api calls

Use when the argument says "manually" / "without annotations" / "using the API", or when annotations can't express what's asked.

Root method → transaction:

import co.elastic.apm.api.ElasticApm;
import co.elastic.apm.api.Transaction;

public void doStuff() {
    Transaction transaction = ElasticApm.startTransaction();
    try {
        transaction.setName("ClassName#doStuff");
        transaction.setType(Transaction.TYPE_REQUEST);
        // original body
    } catch (RuntimeException e) {
        transaction.captureException(e);
        throw e;
    } finally {
        transaction.end();
    }
}

Catch the narrowest type the method could actually throw (RuntimeException if nothing checked is declared, or its existing checked type) — don't widen the method's contract.

Child method → span, preferring a direct reference. If the child is called directly from code that already holds the Transaction/Span object (the common case — a private helper called from the method just wrapped above), thread that reference through as a parameter and start the child span on it. No activation, nothing thread-local to get wrong:

import co.elastic.apm.api.Span;
import co.elastic.apm.api.Transaction;

private long helper(long arg, Transaction parent) throws InterruptedException {
    Span span = parent.startSpan("app", "ClassName", "helper");
    try {
        span.setName("ClassName#helper");
        // original body
        return result;
    } catch (InterruptedException e) {
        span.captureException(e);
        throw e;
    } finally {
        span.end();
    }
}

Only when there's no direct reference to pass (reached through a callback, an unrelated class, or another thread), fall back to thread-local lookup — which requires the parent to have been activate()d by its caller, or currentSpan()/currentTransaction() returns a no-op:

try (var scope = transaction.activate()) {
    // deeper in the call stack, with no reference to `transaction`:
    Span span = ElasticApm.currentSpan().startSpan("app", "ClassName", "methodName");
    ...
}

Prefer the direct-reference form whenever you can reach it — it's simpler and correct without reasoning about activation scope.

Conventions (both instrument sub-modes)

• type for in-process method spans defaults to "app". Use "db", "external", "cache", etc. only for calls that actually leave the process, and only when you can tell that from the code — ask rather than guess.
• subtype = the class name, action = the method name, when set explicitly.
• Always end() a manually-started span/transaction in finally; one started but never ended leaks and never reports.
• captureException before rethrowing, never instead of rethrowing — this is tracing, not error handling.

What this skill never does, in either mode

Add a logging framework or metrics library; add any dependency beyond the two co.elastic.apm artifacts; instrument a method the invocation didn't name; overwrite an existing observability/ folder without asking; or start the Docker stack itself (docker compose up) unless explicitly asked to.