changelog-detective
Skill36 starsChanged 17 days agoMITView on GitHub
Detect what changed in the product by comparing before and after a deploy or date — find new behaviors, new errors, changed user flows, and unexpected side effects that weren't in the release notes. Use after a deploy or when the user suspects something changed but doesn't know what.
In this repository45 other files it gives its agents.
- a11y-analyzerSkill
- annotation-opsSkill
- anomaly-detectorSkill
- api-monitorSkill
- benchmark-analyzerSkill
- campaign-trackerSkill
- cohort-compassSkill
- comparisonsSkill
- customer-360Skill
- dashboard-builderSkill
- deploy-radarSkill
- error-forensicsSkill
- experiment-analyzerSkill
- feature-adoptionSkill
- form-analyzerSkill
- frustration-hunterSkill
- funnel-doctorSkill
- general-analysisSkill
- heatmap-generatorSkill
- incident-responderSkill
- jira-bug-reporterSkill
- journey-builderSkill
- lifecycle-analyzerSkill
- metric-auditorSkill
- mobile-analyzerSkill
- nps-proxySkill
- onboarding-tutorSkill
- page-performanceSkill
- persona-builderSkill
- predictive-alertsSkill
- privacy-auditorSkill
- query-translatorSkill
- quick-statsSkill
- report-deckSkill
- retention-analyzerSkill
- revenue-impactSkill
- scroll-depth-analyzerSkill
- segment-wizardSkill
- session-playlistSkill
- session-reviewSkill
- session-searchSkill
- slack-reporterSkill
- user-journeySkill
- webhook-healthSkill
- weekly-digestSkill
The file
We split this skill into the rules it gives an agent
3 rules. About 0 are common, 2 are distinctive, and 1 is very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈2
Distinctive · estimate
This author's own choices. Worth a look.
1
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 22
Use free fullstory:discover_groups and fullstory:get_opportunity_stats for live signals.
- Distinctive: line 32
Ask: "I'll compare 48 hours before and after the deploy (excluding the deploy hour).
- Distinctive: line 134
If you find undocumented changes that look like bugs, flag them immediately.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Changelog Detective
Find what actually changed after a deploy — not what the release notes say, but what the session data shows. Catch undocumented changes, side effects, and surprises.
When to Use
- "What actually changed after the deploy?"
- "Something feels different since Tuesday — what changed?"
- "Check for undocumented changes in the last release"
- "Did the deploy introduce any new user behaviors?"
- "What new errors appeared this week that weren't here last week?"
Mental Model
Release notes tell you what was intended to change. Session data tells you what actually changed. These often differ. A "minor CSS fix" might accidentally break form validation. A "performance improvement" might introduce a new race condition. This skill hunts for the gaps between intent and reality.
Use free fullstory:discover_groups and fullstory:get_opportunity_stats for live signals. Discovery accepts one scope at a time (page_id, domain, app_name, segment_id, or funnel_id); use a segment for device/user filters and the live schema for time ranges. Use computed metrics for exact custom windows.
Workflow
Step 1: Define the comparison window
Two time periods:
- Before: 24-72 hours before the deploy/change date
- After: 24-72 hours after (but not including the deploy window itself — things are chaotic during rollout)
Ask: "I'll compare 48 hours before and after the deploy (excluding the deploy hour). That OK?"
Step 2: Before/After diff — Errors
What errors existed before vs after?
fullstory:build_metric(query="errors by type", output_type="top_n")
fullstory:compute_metric(metric_id, time_range=before)
fullstory:compute_metric(metric_id, time_range=after)Look for:
- New errors: Error types that appear in After but not Before → these are deploy regressions
- Resolved errors: Error types in Before but not After → these are fixed (celebrate these)
- Changed errors: Same error type, different frequency → possible impact change
Step 3: Before/After diff — Frustrations
fullstory:discover_groups(compare_to_previous=true)
→ identify changed frustration signals
→ build metrics for those signals and compute for the exact before/after windowsLook for:
- New frustration signals: Rage clicks or dead clicks on elements that weren't problematic before
- Resolved frustrations: Frustrations that disappeared (fix worked!)
- Changed patterns: Same frustration, different page or device distribution
Step 4: Before/After diff — User behavior
Did user flows change?
fullstory:build_metric(query="page views by page", output_type="top_n")
→ compare before/after: any pages suddenly getting more/less traffic?
fullstory:build_metric(query="conversion rate", output_type="single_number")
→ compare before/after: did conversion change?Look for:
- Traffic shifts: Pages with significantly different traffic share
- Flow changes: New common page sequences (users navigating differently)
- Time changes: Sessions getting longer or shorter
- Conversion changes: Key funnels performing differently
Step 5: Before/After diff — Device/Browser
Did the change affect specific platforms?
fullstory:build_metric(query="sessions by device type", output_type="top_n")
→ compare before/after: did the device mix change?
fullstory:build_metric(query="errors on Safari", output_type="single_number")
→ compare before/after: any browser-specific regressions?Step 6: Assemble the changelog
## Actual Changelog — v2.3.1 Deploy (Aug 4 15:00 UTC)
### What the release notes said:
- "Fixed promo code validation"
- "Updated checkout button styling"
- "Performance improvements"
### What actually changed (from session data):
✅ **Promo code fixed**: Rage clicks on "Apply Promo" dropped from 412/week to 24/week (94% reduction)
✅ **Button styling**: No new dead clicks on checkout elements — visual change was clean
🟡 **Performance**: Page load time unchanged (was 2.1s, now 2.1s). "Improvements" may not be user-visible.
### Undocumented changes:
🔴 **New error introduced**: `TypeError: Cannot read property 'validate' of undefined` on /checkout — 34 users in 48 hours. Did not exist before deploy. Stack trace points to the new promo validation code.
🔴 **Mobile regression**: Checkout form fields shifted 20px down on iOS Safari. Not visible on desktop. Affects ~15% of mobile checkout users.
🟡 **New user behavior**: 8% of users now click "Apply Promo" before entering a code (dead click). The new button styling may make it look like it does something before the field is filled.
### Net assessment:
The deploy fixed the #1 frustration (promo code rage clicks) but introduced a new error and a mobile layout regression. Worth fixing both, but the fix was net positive — 412 fewer frustrated users per week vs 34 users seeing a new error.Changelog Diff Checklist
| Dimension | Before | After | Change |
| Top 5 errors | [list] | [list] | New? Resolved? Changed frequency? |
| Top frustrations | [list] | [list] | New signals? Resolved? Shifted? |
| Conversion rate | X% | Y% | Δ = Y-X |
| Bounce rate | X% | Y% | Δ |
| Session duration | X min | Y min | Δ |
| Device mix | D/M/T% | D/M/T% | Shift? |
| Browser errors | Per browser | Per browser | New browser-specific issues? |
| Page traffic share | Top pages | Top pages | Any pages gaining/losing >20%? |
Guidelines
- The deploy window (first hour after deploy) is noisy — caches warming, users getting new bundles, partial rollouts. Exclude it from before/after comparison. Start "after" at +1 hour.
- New errors are the strongest signal. An error that didn't exist before the deploy is almost certainly deploy-related. Prioritize these.
- Behavioral changes can be positive. If users are suddenly using a feature more, that might be intentional (better UX) rather than a bug. Investigate before flagging.
- Some changes are coincidental, not causal. A traffic spike might be a marketing campaign, not the deploy. Cross-reference with
campaign-trackerbefore attributing to the deploy. - If you find undocumented changes that look like bugs, flag them immediately. The team might not know. "Heads up: the deploy introduced a TypeError on /checkout that wasn't in the release notes. 34 users affected in 48 hours."
- For major deploys (50+ files changed), run this skill. For minor deploys (hotfix, config change),
deploy-radaris usually sufficient.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

