agentleFS
ollygardenollygarden/opentelemetry-agent-skills/skills/otel-profiles/SKILL.md

otel-profiles

Skill125 starsChanged 4 months agoApache-2.0View on GitHub

OpenTelemetry profiles signal and the eBPF profiler (otelcol-ebpf-profiler, `profiling` receiver). Use when setting up continuous profiling with OpenTelemetry, deploying the eBPF profiler on Linux hosts, Docker, or Kubernetes (DaemonSet, kind, k3d), building a profiles pipeline, sending OTLP profiles to Pyroscope, Grafana Cloud Profiles, or another backend, getting flame graphs from OTel, enabling off-CPU profiling, or debugging profiler errors about tracefs, PID namespaces, missing pods, or the service.profilesSupport feature gate. Not for SDK tracing or metrics setup.

The file

We split this skill into the rules it gives an agent

3 rules. About 0 are common, 1 is distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈1

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Read references/receiver.md when setting keys beyond samples_per_second, enabling off-CPU or OBI correlation, or pinning a release not in …

    line 25
  2. Distinctive:

    Pick the image and pin: Use the latest release unless a needed key or component is missing from it (see the matrix).

    line 29
  3. Very specific:

    Export and verify: Add debug (verbosity: basic) first: each reporter_interval it logs "resource profiles": N, "sample records": M.

    line 35

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

OpenTelemetry Profiles

The main OpenTelemetry producer is the eBPF profiler, shipped as the profiling receiver in the otelcol-ebpf-profiler Collector distribution. It profiles every process on a Linux host with no application changes.

Maturity: the signal entered public Alpha in March 2026, the OTLP profiles proto (v1development) is Development, the profiling receiver is Development, and the Collector gate service.profilesSupport is alpha. Expect breaking changes; pin the profiler, Collector, and backend versions together.

Collector-side component facts (OTLP receiver/exporter profile support, the fixed /v1development/profiles HTTP path, k8s_attributes, memory_limiter, filter profile_conditions) live in the otel-collector skill's component pages; OTTL profile/profilesample contexts live in otel-ottl.

otelcol-ebpf-profiler 0.164.0 bundles only: receivers profiling, nop; processors memory_limiter, k8s_attributes, resource_detection, resource, transform (plus batch, which does not support profiles); exporters otlp_grpc, otlp_http, debug, file, nop; extensions health_check, pprof, opamp, offcpu. For anything else (filter, the basicauth extension, vendor exporters), relay to a regular Collector or build a distribution (otel-collector-builder skill).

Version matrix (Verified 2026-11-05)

ArtifactVersionWhat it means
otel/opentelemetry-collector-ebpf-profiler0.163.0 to 0.163.2, and 0.164.0 (newest release at verification, 2026-10-26)Pins ebpf-profiler v0.0.202640: pid_namespace_translation_mode (#1801); bundles the offcpu extension. Components unchanged from 0.163.0; 0.164.0 only bumps module versions.
Same image, 0.162.0last release before the changePins ebpf-profiler v0.0.202636: boolean pid_namespace_translation; does not bundle offcpu.

Evidence: Docker Hub tags, the v0.163.0, v0.163.1, v0.163.2, and v0.164.0 distribution manifests; the v0.164.0 changelog.

Read references/receiver.md when setting keys beyond samples_per_second, enabling off-CPU or OBI correlation, or pinning a release not in this table (keys follow the pinned ebpf-profiler, not the Collector version).

Workflow

  1. Pick the image and pin. Use the latest release unless a needed key or component is missing from it (see the matrix). Alternative: an OCB build (otel-collector-builder skill) based on the distribution manifest.
  2. Pick the environment.
    • Linux host or plain Docker: Run on a host below.
    • Kubernetes: references/kubernetes.md (DaemonSet, RBAC, Kubernetes metadata, memory sizing).
    • Kubernetes nodes that are themselves containers (kind, k3d, Docker-in-Docker, minikube with the docker driver): also references/nested-containers.md.
  3. Wire the pipeline. profiles pipeline: profiling receiver, memory_limiter first, enrichment processors, then otlp_grpc or otlp_http. Start the Collector with --feature-gates=+service.profilesSupport. Run validate --config=<file> --feature-gates=+service.profilesSupport with the target image; it catches wrong keys for the pinned version.
  4. Export and verify. Add debug (verbosity: basic) first: each reporter_interval it logs "resource profiles": N, "sample records": M; no lines or zero counts mean the receiver is not producing (see troubleshooting). verbosity: detailed shows resource attributes. Then add the backend: references/backends.md for endpoints, auth, and per-backend label behavior.
  5. Troubleshoot by exact error string: references/troubleshooting.md.

Run on a host

The profiler needs root or CAP_SYS_ADMIN/CAP_BPF/CAP_PERFMON, the host PID namespace, and a mounted tracefs. A container gets a fresh /sys without tracefs even when privileged, so bind-mount it:

docker run -d --name otel-ebpf-profiler --privileged --pid=host \
  -v /sys/kernel/tracing:/sys/kernel/tracing \
  -v "$PWD/config.yaml:/etc/otelcol/config.yaml:ro" \
  otel/opentelemetry-collector-ebpf-profiler:0.164.0 \
  --config=/etc/otelcol/config.yaml --feature-gates=+service.profilesSupport
receivers:
  profiling:
    samples_per_second: 20      # default
processors:
  memory_limiter:
    check_interval: 1s
    limit_mib: 800
    spike_limit_mib: 200
exporters:
  debug:
    verbosity: basic
  otlp_grpc:
    endpoint: profiles-backend:4317   # backend-specific, see references/backends.md
    tls:
      insecure: true
service:
  pipelines:
    profiles:
      receivers: [profiling]
      processors: [memory_limiter]
      exporters: [otlp_grpc, debug]

The image is built FROM scratch: no shell, so debug from logs, not exec. Expected startup lines: eBPF tracer loaded, Attached sched monitor, Everything is ready.

Known limitations

  • The receiver profiles every process; it has no process, container, or namespace selection, and upstream declined to add one (#358). To keep a subset, filter on enrichment attributes with filter profile_conditions; see "Keep only some workloads" in references/kubernetes.md. Sampling cost on the node stays the same; only exported volume drops.
  • Linux amd64/arm64, kernel 5.10+ (checked at config validation; no_kernel_version_check bypasses it). Backends may lag the proto; check which OTLP profiles version they accept.
  • Trace-to-profile correlation is partial: OTLP profiles can link samples to trace_id/span_id, but the profiler fills them only from a context producer (OBI via obi_process_ctx; the thread-context OTEP is accepted, but the receiver's thread_context support in v0.0.202640 is a stub). Go pprof labels are read as custom labels.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.