otel-profiles
Skill125 starsChanged 4 months agoApache-2.0View on GitHub
OpenTelemetry profiles signal and the eBPF profiler (otelcol-ebpf-profiler, `profiling` receiver). Use when setting up continuous profiling with OpenTelemetry, deploying the eBPF profiler on Linux hosts, Docker, or Kubernetes (DaemonSet, kind, k3d), building a profiles pipeline, sending OTLP profiles to Pyroscope, Grafana Cloud Profiles, or another backend, getting flame graphs from OTel, enabling off-CPU profiling, or debugging profiler errors about tracefs, PID namespaces, missing pods, or the service.profilesSupport feature gate. Not for SDK tracing or metrics setup.
In this repository19 other files it gives its agents.
- AGENTS.mdAGENTS.md
- otel-browserSkill
- otel-collector-builderSkill
- otel-collectorSkill
- otel-declarative-configSkill
- otel-dotnetSkill
- otel-goSkill
- otel-javaSkill
- otel-jsSkill
- otel-ottlSkill
- otel-pythonSkill
- otel-rubySkill
- otel-sdk-versionsSkill
- otel-semantic-conventionsSkill
- otel-span-events-to-logs-migrationSkill
- otel-telemetry-emissionsSkill
- otel-telemetrygenSkill
- otel-upgradeSkill
- otel-weaverSkill
The file
We split this skill into the rules it gives an agent
3 rules. About 0 are common, 1 is distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈1
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 25
Read references/receiver.md when setting keys beyond samples_per_second, enabling off-CPU or OBI correlation, or pinning a release not in …
- Distinctive: line 29
Pick the image and pin: Use the latest release unless a needed key or component is missing from it (see the matrix).
- Very specific: line 35
Export and verify: Add debug (verbosity: basic) first: each reporter_interval it logs "resource profiles": N, "sample records": M.
≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
OpenTelemetry Profiles
The main OpenTelemetry producer is the eBPF profiler, shipped as the profiling receiver in the otelcol-ebpf-profiler Collector distribution. It profiles every process on a Linux host with no application changes.
Maturity: the signal entered public Alpha in March 2026, the OTLP profiles proto (v1development) is Development, the profiling receiver is Development, and the Collector gate service.profilesSupport is alpha. Expect breaking changes; pin the profiler, Collector, and backend versions together.
Collector-side component facts (OTLP receiver/exporter profile support, the fixed /v1development/profiles HTTP path, k8s_attributes, memory_limiter, filter profile_conditions) live in the otel-collector skill's component pages; OTTL profile/profilesample contexts live in otel-ottl.
otelcol-ebpf-profiler 0.164.0 bundles only: receivers profiling, nop; processors memory_limiter, k8s_attributes, resource_detection, resource, transform (plus batch, which does not support profiles); exporters otlp_grpc, otlp_http, debug, file, nop; extensions health_check, pprof, opamp, offcpu. For anything else (filter, the basicauth extension, vendor exporters), relay to a regular Collector or build a distribution (otel-collector-builder skill).
Version matrix (Verified 2026-11-05)
| Artifact | Version | What it means |
otel/opentelemetry-collector-ebpf-profiler | 0.163.0 to 0.163.2, and 0.164.0 (newest release at verification, 2026-10-26) | Pins ebpf-profiler v0.0.202640: pid_namespace_translation_mode (#1801); bundles the offcpu extension. Components unchanged from 0.163.0; 0.164.0 only bumps module versions. |
Same image, 0.162.0 | last release before the change | Pins ebpf-profiler v0.0.202636: boolean pid_namespace_translation; does not bundle offcpu. |
Evidence: Docker Hub tags, the v0.163.0, v0.163.1, v0.163.2, and v0.164.0 distribution manifests; the v0.164.0 changelog.
Read references/receiver.md when setting keys beyond samples_per_second, enabling off-CPU or OBI correlation, or pinning a release not in this table (keys follow the pinned ebpf-profiler, not the Collector version).
Workflow
- Pick the image and pin. Use the latest release unless a needed key or component is missing from it (see the matrix). Alternative: an OCB build (
otel-collector-builderskill) based on the distribution manifest. - Pick the environment.
- Linux host or plain Docker: Run on a host below.
- Kubernetes: references/kubernetes.md (DaemonSet, RBAC, Kubernetes metadata, memory sizing).
- Kubernetes nodes that are themselves containers (kind, k3d, Docker-in-Docker, minikube with the docker driver): also references/nested-containers.md.
- Wire the pipeline.
profilespipeline:profilingreceiver,memory_limiterfirst, enrichment processors, thenotlp_grpcorotlp_http. Start the Collector with--feature-gates=+service.profilesSupport. Runvalidate --config=<file> --feature-gates=+service.profilesSupportwith the target image; it catches wrong keys for the pinned version. - Export and verify. Add
debug(verbosity: basic) first: eachreporter_intervalit logs"resource profiles": N, "sample records": M; no lines or zero counts mean the receiver is not producing (see troubleshooting).verbosity: detailedshows resource attributes. Then add the backend: references/backends.md for endpoints, auth, and per-backend label behavior. - Troubleshoot by exact error string: references/troubleshooting.md.
Run on a host
The profiler needs root or CAP_SYS_ADMIN/CAP_BPF/CAP_PERFMON, the host PID namespace, and a mounted tracefs. A container gets a fresh /sys without tracefs even when privileged, so bind-mount it:
docker run -d --name otel-ebpf-profiler --privileged --pid=host \
-v /sys/kernel/tracing:/sys/kernel/tracing \
-v "$PWD/config.yaml:/etc/otelcol/config.yaml:ro" \
otel/opentelemetry-collector-ebpf-profiler:0.164.0 \
--config=/etc/otelcol/config.yaml --feature-gates=+service.profilesSupportreceivers:
profiling:
samples_per_second: 20 # default
processors:
memory_limiter:
check_interval: 1s
limit_mib: 800
spike_limit_mib: 200
exporters:
debug:
verbosity: basic
otlp_grpc:
endpoint: profiles-backend:4317 # backend-specific, see references/backends.md
tls:
insecure: true
service:
pipelines:
profiles:
receivers: [profiling]
processors: [memory_limiter]
exporters: [otlp_grpc, debug]The image is built FROM scratch: no shell, so debug from logs, not exec. Expected startup lines: eBPF tracer loaded, Attached sched monitor, Everything is ready.
Known limitations
- The receiver profiles every process; it has no process, container, or namespace selection, and upstream declined to add one (#358). To keep a subset, filter on enrichment attributes with
filterprofile_conditions; see "Keep only some workloads" in references/kubernetes.md. Sampling cost on the node stays the same; only exported volume drops. - Linux
amd64/arm64, kernel 5.10+ (checked at config validation;no_kernel_version_checkbypasses it). Backends may lag the proto; check which OTLP profiles version they accept. - Trace-to-profile correlation is partial: OTLP profiles can link samples to
trace_id/span_id, but the profiler fills them only from a context producer (OBI viaobi_process_ctx; the thread-context OTEP is accepted, but the receiver'sthread_contextsupport inv0.0.202640is a stub). Go pprof labels are read as custom labels.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

