agentleFS
PixiBixiPixiBixi/claude-skills/skills/grafana-dashboards/SKILL.md

grafana-dashboards

Skill0 starsChanged 12 days agoApache-2.0View on GitHub

Use for any Grafana dashboard or PromQL work, BEFORE gcx:create-dashboard: a pasted /d/<uid> URL to edit, fix or audit, "dash", "panel", creating, editing or cleaning a dashboard, a dashboard that OOMs Thanos, empty panels, slow variables, a wrong-looking number, an audit of dashboards using a metric. At Equativ, load equativ-observability with it. Not for stackdriver panels (grafana-cloud-monitoring), alert rules (equativ-add-alert) or explaining what a graph shows in prod (equativ-investigation).

  • Installs packages

14 other projects use a version of this.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

6 rules. 0 are common, 1 is distinctive, and 5 are very specific to this project.

  • 0

    Common

    Most of the 14 edited versions we read keep these.

  • 1

    Distinctive

    This author's own choices. Worth a look.

  • 5

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Write the panels with references/panels.md (tables, fleets, empty-state, template variables) and references/promql.md (global Thanos, …

    line 17
  2. Very specific:

    Every Grafana call in this skill's tools goes through gcx.

    line 33
  3. Very specific:

    Call gcx and kubectl directly, never through a shell function, a variable or an env prefix (zsh breaks the first two, and permission rules …

    line 38
  4. Distinctive:

    Before writing any expr, confirm the metric exists on the target datasource:

    line 52
  5. Very specific:

    Before pushing, grep your generated JSON for leftovers: check every title, description, legendFormat, displayName and mapping text.

    line 63
  6. Very specific:

    Before handing over: lint for negative gridPos.y, missing plugin panels and datasource uid remap, and check no panel shows "No data".

    line 89

“Common” is measured: kept by at least 6 in 10 of the 14 edited versions of this file we could read. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Authoring Grafana dashboards and PromQL

Load order: this skill first, then gcx:create-dashboard when the work is a visual redesign or new panels (its snapshot loop), or manage-dashboards for plain get/push. This skill's rules win on conflict.

Three tools ship in scripts/: lint_dashboard.py (schema baseline before saving), grafana_metric_usage.py (is a metric still read anywhere) and grafana_datasource_usage.py (is a datasource still read anywhere). The working directory is never this folder, so call them through SKILL=${CLAUDE_SKILL_DIR} as in the examples below. They route through gcx api, so gcx's own OAuth refresh applies; the target is picked as in § Which instance, the same chain as the sibling skill. When the output is evidence for a Jira ticket, an MR or a postmortem rather than a dashboard: on Grafana Cloud, gcx dashboards snapshot renders the existing panel to PNG; use the charting-grafana-metrics skill for a comparison no panel shows, or on a self-hosted Grafana without the Image Renderer plugin (where the snapshot errors out).

Workflow: create or edit a dashboard

  1. § gcx setup and § Which instance, once per session.
  2. § Query pre-flight on every metric you plan to read, on the target datasource.
  3. Platform folder: read references/folder-layout.md. Apply § Dashboard-level defaults and § Language.
  4. Write the panels with references/panels.md (tables, fleets, empty-state, template variables) and references/promql.md (global Thanos, aggregation and absence traps).
  5. § Before you save: run the linter, then push with § Every save carries a version note (a new dashboard: § Creating a dashboard with gcx).

A panel that is empty, slow, OOMs the query layer or shows a plausible but wrong number: start from the symptom table in references/troubleshooting.md.

References

TaskRead
Building panels, tables, fleets, healthy states, template variablesreferences/panels.md
PromQL on a global Thanos, aggregation traps, tails, absence, caching, sidecar jobreferences/promql.md
Is a metric or datasource still read, renaming a template variablereferences/audits-and-renames.md
A symptom on an existing panelreferences/troubleshooting.md
Component folder layoutreferences/folder-layout.md
Reading or patching an existing dashboard's JSON (v1 vs v2 paths, GCM panels, empty JSONPath)references/reading-dashboard-json.md

gcx setup

Every Grafana call in this skill's tools goes through gcx. Check it once per session:

command -v gcx >/dev/null || { echo "install: brew install gcx"; exit 1; }
gcx config check || { echo "not ready: gcx login"; exit 1; }

Call gcx and kubectl directly, never through a shell function, a variable or an env prefix (zsh breaks the first two, and permission rules and the auto mode classifier cannot match any of them).

Ad hoc PromQL goes through thq (scripts/thq, symlinked in PATH), never a curl wrapper or raw gcx metrics query -o json: it prints one line per series (min/avg/max + sparkline) and sets dedup=true, partial_response=false on Thanos. thq r '<expr>' --since 6h, thq q, thq names|labels|values, thq stores; -c <gcx context> (default current-context), --ds <uid|name|alias>. Aliases, per-context default datasources and the --direct URL come from ~/.config/thq/config.json (THQ_CONFIG), never from the script. A Tempo search spans 48h at most (exceeds 48h0m0s): walk a week day by day with --from/--to.

Which instance

RequestPass
A URL (https://<host>/d/<uid>)--url https://<host>: the tool uses the gcx context whose server has that host
An instance named without a URLgcx config list-contexts, then --gcx-context <name>
Neithernothing: gcx's current context

$GRAFANA_TOKEN (then $GRAFANA_SERVICE_ACCOUNT_TOKEN, $GTOK) is only sent to the host of $GRAFANA_URL, or used when gcx is missing; --token forces a static token. A host no gcx context serves exits with the gcx login command to run. The dashboard cache is kept per host.

Query pre-flight (before writing any expr)

Before writing any expr, confirm the metric exists on the target datasource:

count(<metric>) or on() vector(-1)    # -1 = absent on this datasource

-1 means "no sample right now", not "no such metric". On a sparse family (GCP quota metrics are the usual case) the bare count() returns -1 on a metric that exists and has data, so wrap the check in the same window the panel will use before concluding the name is wrong: count(last_over_time(<metric>[6h])) or on() vector(-1). To settle name versus absence, go to the descriptors API instead of retrying spellings, see the grafana-cloud-monitoring skill.

A panel that is empty on a global Thanos but fine on a per-cluster Prometheus means the metric is filtered out of the global store. Find the culprit query from the frontend's own slow log: kubectl logs -l app.kubernetes.io/component=query-frontend | grep -i slow -> param_query= + grafana_dashboard_uid/panel_id.

Language: dashboards are ALWAYS in English

Every user-facing string is English: dashboard title/description, row names, panel titles, panel descriptions, legendFormat, value-mapping text, variable labels and descriptions, table column displayName. Even when the conversation is in another language. Dashboards are shared artifacts read by international teams. Same for alert rule names, summaries and annotations.

Before pushing, grep your generated JSON for leftovers: check every title, description, legendFormat, displayName and mapping text.

Dashboard-level defaults

Set these without asking, they are schema baseline, not per-dashboard choices:

  • graphTooltip: 1 (shared crosshair), ALWAYS. Never 0 (no sharing), never 2 (shared tooltip, unreadable past ~10 panels). These dashboards are read by correlating panels against each other: hovering CPU must move the cursor line on GC, threadpool and network at the same instant. When auditing an existing dashboard, check $.graphTooltip and report it compliant rather than assuming it needs changing.
  • timezone: "utc", editable: true, and a dashboard-level description (1-2 sentences: what it is for, who reads it). It is what shows up in Grafana search.
  • Panel type timeseries/stat/table only. The graph plugin is deprecated; rewrite it when you touch a panel that still uses it.
  • Every panel needs a unique id. JSON authored without them saves with none at all, which silently breaks panel permalinks (?viewPanel=) and the panel_id label in the query-frontend slow log, i.e. it disables the debugging procedure in § Query pre-flight on exactly the dashboards you will need it for.
  • One name for the datasource variable, across the whole folder. Three names for the same thing (ds, datasource, Source) makes dashboard URLs non-transposable and breaks copy-paste between panels. When standardising, align on the form more than half the dashboards already carry, even an imported DS_* name, rather than on what a doc says: every rename breaks the ?var-<name>= in existing bookmarks and tickets. A second datasource variable takes the convention as prefix (dsTooling), never a new spelling.

Folder layout (common dashboard set)

A component folder holds the same five kinds of dashboard: <Comp> / 0 Start here, <Comp> / 1 SLA, <Comp> / Ops / <x>, <Comp> / Sizing / <x>, <Comp> / Deep dive / <x>, with readable uids, the component tag and the same variables everywhere. Read references/folder-layout.md before creating a dashboard in a platform folder, or before starting a new one: it gives what each kind contains and the variable block.

Before you save (run the linter)

The rules above are the ones that get skipped, and it is always on the quick dashboard built in twenty minutes, not on the big investigation one. scripts/lint_dashboard.py decides them from the JSON, reads v1 dashboards and dashboard.grafana.app/v2 manifests (gcx dashboards get -o json) alike, and descends into collapsed rows, which a flat $.panels[*] read misses:

SKILL=${CLAUDE_SKILL_DIR}
python3 $SKILL/scripts/lint_dashboard.py mydash.json          # or --folder "K8S" to sweep one folder

Errors exit 1, so it gates a commit; a --uid or folder dashboard it could not fetch exits 2, never a clean 0. The folder-layout rules are warnings only, so legacy folders still pass. Run --help for the flags. Two caveats: its language check is a heuristic that surfaces candidates, so read what it reports rather than trusting the count, and on a folder it aligns the datasource variable on the name carried by more than half of the dashboards that have one, because every rename breaks the ?var-<name>= in existing bookmarks. Below that bar the naming check is skipped: a mere plurality is not a convention, pass --expect-ds-var to pick one.

Read references/reading-dashboard-json.md before reading or patching an existing dashboard's JSON or auditing which dashboards use a metric: an empty JSONPath is not proof of absence, GCM panels hide their queries from an expr sweep, and gcx dashboards get -o json returns a v2 manifest where every v1 path is empty.

After a migration or import

Before handing over: lint for negative gridPos.y, missing plugin panels and datasource uid remap, and check no panel shows "No data".

Every save carries a version note

Each write to a dashboard lands in Settings > Versions, and the note is the only record of why it changed. Write it in English, ticket key first (PE-2210: replace the stop-writes boolean with the headroom to each threshold). Notes cannot be added afterwards: a version is immutable.

  • MCP grafana: the message argument of the update call.
  • gcx dashboards update: set metadata.annotations["grafana.app/message"] in the manifest, the command has no flag for it. Without it the version saves with an empty note, and under the gcx token's account rather than yours.
jq --arg m "PE-XXXX: <what changed>" 'del(.status) | .metadata.annotations["grafana.app/message"] = $m' dash.json > push.json
gcx dashboards update <uid> -f push.json

A 409 Conflict means the manifest's resourceVersion predates another save: re-get right before each push and patch that copy, never rebuild from an older fetch.

Creating a dashboard with gcx

Wrap the v1 model in a manifest and pass the API version: the server defaults to v2 and rejects a v1 body with does not match the expected API version. Use the same --api-version on the later get and update.

jq --arg u "<uid>" '{apiVersion: "dashboard.grafana.app/v1beta1", kind: "Dashboard",
  metadata: {name: $u, annotations: {"grafana.app/folder": "<folder uid>", "grafana.app/grant-permissions": "default"}},
  spec: (del(.id) | .uid = $u)}' dash.json > create.json
gcx dashboards create --api-version dashboard.grafana.app/v1beta1 -f create.json
  • grafana.app/grant-permissions: default is mandatory: without it the dashboard gets no permissions and answers 403, even to an Editor service account.
  • create ignores grafana.app/message: version 1 always reads "Initial save". The ticket note starts with the first update.

Google Cloud Monitoring datasources

A panel on the cloud-monitoring datasource (plugin type stackdriver), in PromQL or MQL mode, is covered by the grafana-cloud-monitoring skill: metric name conversion, the migrateQuery rewrite that empties a JSON-authored panel, monitored_resource, MQL tables, and the pre-flight query validation via gcx datasources cloudmonitoring query.

Two things worth knowing from here: a GCM datasource cannot serve label_values(), so migrating panels onto it kills the dashboard's type: query variables, and a bare expr sweep reads only PromQL-mode targets, so a GCM dashboard in MQL mode looks queryless unless promQLQuery.expr and timeSeriesQuery.query are also read (see references/reading-dashboard-json.md).

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.