Climate update (monthly umbrella update)

All OWID climate data is updated once a month as a single batch: every updateable climate-namespace dataset is bumped to one common version date, in one PR, with one Slack announcement. This skill owns which datasets, in what order, and the single-PR / single-announcement discipline. It delegates the per-dataset mechanics (snapshot → meadow → garden → grapher, diffs, metadata checks, chart upgrade) to /update-dataset — run each chain through that flow on the same branch.

The matching reminder lives in owid-issues/.github/workflows/update-climate.yml (one monthly issue on the 10th). Wildfires is the one dataset with a reminder of its own on top of that: update-climate-wildfires.yml fires weekly from May to the end of September.

Modes

• Full monthly update (default): bump every updateable dataset below to a common new version date.
• Wildfires-only subset: bump just weekly_wildfires (and optionally yearly_burned_area) to a new version, for the weekly in-season refresh. Follow /update-wildfires-data, which covers the source's weekly grid, its provisional last data point, and the snapshot's silent per-country failure mode. Come back here only if you are bumping the whole batch.

Updateable datasets (climate.yml "UPDATEABLE" section)

Grouped by source. Bump all of these to the same new version date.

Plus two derived/aggregate datasets with no snapshot of their own:

• long_run_ghg_concentration (combines NOAA ghg_concentration with the frozen EPA series)
• climate_change_impacts (the aggregate; pulls most of the rows above)

The yearly sources usually show no change in a given month — that is expected, not a bug.

Frozen — NEVER bump these (climate.yml "NOT UPDATEABLE" section)

Skip entirely. They receive no new data and several feed climate_change_impacts:

• EPA 2024-04-17: ghg_concentration, ocean_heat_content, ice_sheet_mass_balance, mass_balance_us_glaciers
• global_sea_level (2024-01-28, NOAA Climate.gov)
• ipcc_scenarios
• The 12 migrated legacy chains (one-off papers / historical sources, dates 2017-2022)

If a frozen source's producer ever republishes, that is a separate, deliberate version bump — not part of the monthly run.

Dependency order

1. Bump and run the leaf source chains (snapshot → meadow → garden) for every updateable dataset above. climate_change_impacts's frozen deps (EPA, global_sea_level) stay on their old versions — reference them unchanged.
2. Bump and run long_run_ghg_concentration, then the climate_change_impacts aggregate.
3. Run all grapher steps and the climate_change explorer (the explorer stays latest).

The grapher families to rebuild: the 7 surface_* graphers, total_precipitation_annual, the 4 wildfire graphers, sst/sst_by_month, nasa_ozone_hole, the 3 sea_ice_* graphers, climate_change_impacts_annual/_monthly, and yearly_burned_area.

Procedure

1. Create the branch + draft PR with etl pr "📊 Update climate data" data. One branch and one PR for the whole batch.
2. Run each updateable chain through the /update-dataset flow on that branch, all targeting today's date as <new_version> so they land on a common version. Follow the dependency order above, so the aggregate picks up the new versions once rather than repeatedly. Keep the DAG's nesting and comment headers: version-substitute the existing UPDATEABLE block instead of keeping the flat block etl update appends.

3. Do not remove or archive the old steps yet. The previous versions (their step files, their snapshot folders and their dag/climate.yml entries) stay active until the review is done. Two things depend on them being present:

   • The reviewer compares consecutive versions with the compare-previous-version VS Code extension, which diffs each step file against the same-named file in the nearest lower version folder. Delete the old folders and there is nothing to compare against.
   • The chart remap (step 4) finds each dataset's predecessor through the version tracker, which reads only the active DAG. Remove the old DAG entries first and etl indicator-upgrade auto reports "No dataset migrations detected".

   If the old entries were already removed, restore them: git checkout <base> -- <old folders> dag/archive/climate.yml, and append the old UPDATEABLE block to dag/climate.yml under a comment that says it is kept until the review is done.

4. Chart remap on staging via the Indicator Upgrader. The version bump mints new variable IDs for every grapher dataset, and charts do not follow on their own; until the remap, Chart Diff shows nothing. With both versions in the DAG, automatic detection works:

   STAGING=1 .venv/bin/etl indicator-upgrade auto --dry-run
   STAGING=1 .venv/bin/etl indicator-upgrade auto

   It detects 18 of the 20 pairs. The two climate_change_impacts_* grapher steps are declared only under the explorer's viz:// entry and the version tracker does not list them; remap those two by hand (match -old <id> -new <id> --perfect-match-only, then upgrade; dataset ids from datasets on staging, whose catalogPath has no grapher/ prefix). Staging can hold two old wildfire versions (the batch's and the last weekly one): pair the one that carries charts. Afterwards, check that no old dataset still carries a chart and that the per-dataset chart counts on staging equal production's. Watch the once-off cases: any dataset changing namespace needs its remap reviewed explicitly (see below).

5. Hand off for review. In the PR body and in the chat, list exactly which files changed in content relative to the previous version, so the reviewer does not have to open all ~110 files. Compute it with cmp between the old and new version folders; in a normal month only the snapshot folder differs (the .dvc files and any edited snapshot script), and every meadow, garden and grapher file is byte-identical to its predecessor. Then Anomalist + Chart Diff on staging (enable "Show all charts").
6. Only when the user says the review is done, archive the old versions, as the last commits before merge: remove the old block from dag/climate.yml and the old step and snapshot folders (git rm -r) → commit → etl archive-dag (it reads committed history) → commit dag/archive/climate.yml. It should add exactly the old climate steps (48 for a full batch). Remind the user of this step at the end of every hand-off; it is easy to forget and the PR must not merge with both versions active.
7. One announcement: run /draft-data-update-slack-post for the combined batch, post to #data-updates-comms, and draft the single /latest post. Do not produce per-dataset announcements.

One-off migrations (only on the first run after the refactor)

These are structural moves that happen once, then the dataset behaves like any other updateable one:

• ipcc_scenarios: namespace emissions → climate, and EPA 2024-04-17: namespace epa → climate (retires the epa namespace). ipcc also moves its standalone explorer. EPA has no charts of its own (it only feeds climate_change_impacts), so its move is chart-free; ipcc's needs an explorer/chart remap.

After these land, update this skill's inventory if any short_names or namespaces changed.