agentleFS
SawanaLabsSawanaLabs/agent-demos/.agents/skills/victoriatraces-query/SKILL.md

victoriatraces-query

Skill4 starsChanged 58 days agoMITView on GitHub

Query VictoriaTraces via curl using the Jaeger-compatible API. Use when discovering traced services and operations, searching traces by service/operation/duration/tags, retrieving traces by ID, or mapping service dependencies. Triggers on: trace queries, span search, trace ID lookup, service discovery, operation discovery, service dependencies, distributed tracing, Jaeger API.

1 other project uses a version of this, 1 of them edited.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

4 rules. About 0 are common, 3 are distinctive, and 1 is very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈3

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 1

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    All curl commands load auth from a curl config file:

    line 29
  2. Very specific:

    When VM_CURL_CONFIG is unset, curl reads /dev/null and sends no auth header.

    line 36
  3. Distinctive:

    Always discover services first, then operations, then search traces

    line 46
  4. Distinctive:

    No parameters. Returns all traced service names.

    line 57

≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

VictoriaTraces Query

Query VictoriaTraces Jaeger-compatible HTTP API directly via curl. Covers service/operation discovery, trace search, trace retrieval by ID, and dependency mapping.

Environment

# $VM_TRACES_URL - base URL including /select/jaeger prefix
#   Example: export VM_TRACES_URL="https://vtselect.example.com/select/jaeger"
# $VM_CURL_CONFIG - curl config file with auth header (set for remote, unset for local)
#   Remote: export VM_CURL_CONFIG="$HOME/.config/victoriametrics/curl.conf"
#   Local:  leave unset (defaults to /dev/null - no auth)

IMPORTANT: The Jaeger API lives under /select/jaeger/api/..., NOT root /api/.... The $VM_TRACES_URL env var already includes the /select/jaeger prefix, so all endpoints below use $VM_TRACES_URL/api/....

Auth Pattern

All curl commands load auth from a curl config file:

curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -s \
  "$VM_TRACES_URL/api/services" | jq .

When VM_CURL_CONFIG is unset, curl reads /dev/null and sends no auth header. When set, it must point to a mode-0600 curl config file containing header = "Authorization: Bearer <token>". Never print its contents.

Critical Rules

  • Trace search start/end use Unix MICROSECONDS (16 digits, e.g., 1709769600000000)
  • Dependencies endTs/lookback use Unix MILLISECONDS (13 digits, e.g., 1709769600000)
  • service is required for trace search — you must know the service name first
  • minDuration/maxDuration are string durations (e.g., "1s", "500ms")
  • Trace IDs not found return 404 in the errors array, not an HTTP 404
  • Trace search with time ranges outside retention returns plain text error, not JSON — handle gracefully
  • Always discover services first, then operations, then search traces

Core Endpoints

List Services

curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -s \
  "$VM_TRACES_URL/api/services" | jq '.data[]'

No parameters. Returns all traced service names. Always start here to discover what's available.

Service Operations

curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -s \
  "$VM_TRACES_URL/api/services/my-service/operations" | jq '.data[]'

service is a PATH parameter. Returns operation names for the given service.

Search Traces

# Basic search (last hour, limit 20) — timestamps in microseconds (16 digits)
curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -s \
  "$VM_TRACES_URL/api/traces?service=my-service&start=$(($(date +%s%6N) - 3600000000))&end=$(date +%s%6N)&limit=20" | jq .

# With operation and duration filter
curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -s \
  "$VM_TRACES_URL/api/traces?service=my-service&operation=GET+/api/health&start=$(($(date +%s%6N) - 3600000000))&end=$(date +%s%6N)&minDuration=100ms&limit=20" | jq .

# With tag filter (Jaeger JSON format)
curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -s \
  "$VM_TRACES_URL/api/traces?service=my-service&start=$(($(date +%s%6N) - 3600000000))&end=$(date +%s%6N)&tags=%7B%22http.status_code%22%3A%22500%22%7D&limit=20" | jq .

# With tag filter (VictoriaTraces extended format — key=value pairs, space-separated)
curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -s -G \
  --data-urlencode 'tags=http.status_code=500 resource_attr:service.namespace=production' \
  "$VM_TRACES_URL/api/traces?service=my-service&start=$(($(date +%s%6N) - 3600000000))&end=$(date +%s%6N)&limit=20" | jq .

Parameters: service (required), operation, start (Unix µs), end (Unix µs), limit, minDuration (e.g., 1s), maxDuration, tags (URL-encoded JSON object or VictoriaTraces key=value format)

Get Trace by ID

curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -s \
  "$VM_TRACES_URL/api/traces/abc123def456789" | jq .

Returns full trace with all spans. If trace not found, response contains "errors": [{"code": 404, "msg": "trace not found"}].

Service Dependencies

# Dependencies for the last hour
curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -s \
  "$VM_TRACES_URL/api/dependencies?endTs=$(date +%s%3N)&lookback=3600000" | jq '.data[]'

Parameters: endTs (Unix ms, required), lookback (duration in ms, required). Returns edges between services showing call relationships.

Timestamp Format

Trace search and dependencies use DIFFERENT timestamp units:

EndpointParameterUnitDigitsExample
/api/tracesstart, endMicroseconds161709769600000000
/api/dependenciesendTsMilliseconds131709769600000
/api/dependencieslookbackMilliseconds133600000 (1 hour)
# Current time in microseconds (for trace search)
date +%s%6N

# 1 hour ago in microseconds
echo $(($(date +%s%6N) - 3600000000))

# Specific time to microseconds (GNU date)
date -d "2026-03-07T09:00:00Z" +%s%6N

# Current time in milliseconds (for dependencies)
date +%s%3N

# Duration values for minDuration/maxDuration are strings: "1s", "500ms", "100us"
# Duration value for lookback is ms number: 3600000 = 1 hour

Response Parsing (jq)

# List service names
... | jq '.data[]'

# Extract trace IDs from search results
... | jq '.data[] | .traceID'

# Get span count per trace
... | jq '.data[] | {traceID: .traceID, spans: (.spans | length)}'

# Extract spans with duration > 1s (duration is in microseconds)
... | jq '.data[].spans[] | select(.duration > 1000000) | {operation: .operationName, duration_ms: (.duration / 1000)}'

# Get root span of each trace
... | jq '.data[] | .spans[] | select(.references == null or (.references | length) == 0) | {traceID: .traceID, operation: .operationName, service: .processID}'

# Extract service-to-service calls from dependencies
... | jq '.data[] | "\(.parent) -> \(.child) (\(.callCount) calls)"'

# Get all tags from a span
... | jq '.data[].spans[] | {operation: .operationName, tags: [.tags[] | {(.key): .value}] | add}'

# Find error spans
... | jq '.data[].spans[] | select(.tags[] | select(.key == "error" and .value == true)) | {traceID: .traceID, operation: .operationName}'

Common Patterns

# Full discovery workflow: services -> operations -> traces
curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -s "$VM_TRACES_URL/api/services" | jq '.data[]'
curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -s "$VM_TRACES_URL/api/services/my-service/operations" | jq '.data[]'
curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -s "$VM_TRACES_URL/api/traces?service=my-service&start=$(($(date +%s%6N) - 3600000000))&end=$(date +%s%6N)&limit=10" | jq '.data[] | .traceID'

# Find slow traces (> 5 seconds)
curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -s \
  "$VM_TRACES_URL/api/traces?service=my-service&start=$(($(date +%s%6N) - 3600000000))&end=$(date +%s%6N)&minDuration=5s&limit=20" | jq '.data[] | {traceID: .traceID, spans: (.spans | length)}'

# Look up a trace ID found in logs
curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -s \
  "$VM_TRACES_URL/api/traces/abc123def456789" | jq '.data[].spans[] | {operation: .operationName, duration_ms: (.duration / 1000), service: .processID}'

# Map service dependencies (last 24 hours) — dependencies use milliseconds
curl -q --config "${VM_CURL_CONFIG:-/dev/null}" -s \
  "$VM_TRACES_URL/api/dependencies?endTs=$(date +%s%3N)&lookback=86400000" | jq '.data[]'

Environment Switching

# Check current environment
echo "VM_TRACES_URL: $VM_TRACES_URL"
echo "VM_CURL_CONFIG: ${VM_CURL_CONFIG:-(unset - no auth)}"

Important Notes

  • The Jaeger API prefix /select/jaeger is included in $VM_TRACES_URL — do NOT add it again
  • tags parameter supports both Jaeger JSON format ({"key":"value"}) and VictoriaTraces extended format (key=value resource_attr:key=value scope_attr:key=value)
  • Span durations in responses are in MICROSECONDS (not milliseconds)
  • Process info is referenced by processID in spans, mapped in the trace's processes object
  • GET is sufficient for all endpoints — POST also works for trace search
  • For full endpoint details, parameters, and response formats, see references/api-reference.md

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.