agentleFS
patonkikhpatonkikh/APES/skills/product/analytics-instrumentation-planner/skill.md

analytics-instrumentation-planner

Skill3 starsChanged 4 months agoMITView on GitHub

Design product analytics instrumentation: event taxonomy, properties schema, funnel definitions, and dashboard specs for AI-powered products. Use when defining metrics, tracking plans, growth experiments, or product analytics setup.

1 other project uses a version of this.

Adopted by

Most recently taken up by

The file

We split this skill into the rules it gives an agent

5 rules. About 0 are common, 3 are distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈3

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    If north-star undefined, run north-star-metric-advisor first or propose candidates.

    line 33
  2. Distinctive:

    Document drop-off questions each funnel step should answer.

    line 49
  3. Distinctive:

    No PII in event names or property keys

    line 64
  4. Very specific:

    Use Segment/Amplitude/Mixpanel taxonomy import where supported.

    line 146
  5. Distinctive:

    Run tracking plan review before each major release.

    line 149

≈ “Common” is an estimate: at least 1 in 5 of the 350,495 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
skill.md

Analytics Instrumentation Planner

Purpose

Design product analytics instrumentation: what to track, how events are named, which properties to capture, and how metrics connect to product goals.

Input: Product flows, north-star metric (optional), key hypotheses, platform constraints (web/mobile/API)
Output: Analytics Tracking Plan with event catalog, property schemas, funnels, and dashboard wireframes
Examples: See examples.md for worked input/output.


Workflow

Step 1: Align metrics to goals

LayerExamplesOwner
North starWeekly active teams, successful AI tasksProduct
Input metricsActivation rate, time-to-valueGrowth
Feature metricsPrompt success rate, RAG answer helpfulnessFeature teams
GuardrailError rate, cost per user, churnPlatform

If north-star undefined, run north-star-metric-advisor first or propose candidates.

Step 2: Map user journeys to funnels

For each critical journey:

Entry → Key actions → Success state → Retention signal

Example (AI assistant):

Sign up → First prompt → Successful response → Return within 7d

Document drop-off questions each funnel step should answer.

Step 3: Design event taxonomy

Naming convention: object_action in snake_case (e.g., prompt_submitted, rag_query_completed).

EventTriggerRequired properties
user_signed_upAccount createdsource, plan
prompt_submittedUser sends promptmodel, token_count_bucket
ai_response_ratedThumbs up/downrating, latency_ms

Rules:

  • One event per user-visible action (not per API call unless debugging)
  • No PII in event names or property keys
  • Hash or bucket sensitive values

Step 4: Define property schema

Global context on every event:

PropertyTypeExample
user_idstringinternal ID only
session_idstringUUID
app_versionstring2.4.1
environmentenumprod/staging

Per-event properties documented with type, allowed values, and nullability.

Step 5: AI-specific instrumentation

SignalEvent / propertyWhy
Model routingmodel_id, route_reasonCost/quality analysis
RAG retrievalchunks_retrieved, score_maxDebug bad answers
Tool callstool_name, successAgent reliability
Latencyttft_ms, total_msSLA monitoring
Feedbackrating, categoryQuality loop

Align with ai-evaluation-builder online metrics where possible.

Step 6: Dashboard and experiment specs

DashboardChartsAudience
ActivationFunnel, TTV distributionProduct
AI qualityRating trend, override rateAI team
Unit economicsCost per active userLeadership

For A/B tests: primary metric, guardrails, minimum detectable effect, runtime estimate.

Step 7: Validate

Run Validation checklist.


Decision Rules

ConditionAction
No consent for tracking (GDPR)Stop; define consent-gated events only
High-cardinality properties (raw prompts)Never log; use hashed buckets or opt-in debug
Pre-launch productTrack minimum viable set (≤15 events)
B2B multi-tenantAdd workspace_id to all events
AI cost visibility neededRequire model_id + token_count on inference events

Validation

  • North star and input metrics linked to events
  • ≥1 funnel with steps and success definition
  • Event catalog ≥10 events with naming convention doc
  • Global context properties defined
  • No PII in event/property design
  • AI-specific signals included for AI products
  • Dashboard wireframe with ≥3 charts
  • Implementation notes (SDK, server-side vs client)

Anti-patterns

  • Track everything — unusable warehouse, privacy risk.
  • Vague events — button_clicked without object context.
  • Metric without owner — dashboard nobody reads.
  • Client-only revenue events — trust server-side for billing.
  • Prompt logging by default — compliance incident waiting to happen.

Best Practices

  • Version the tracking plan in repo (analytics/tracking-plan.md).
  • Use Segment/Amplitude/Mixpanel taxonomy import where supported.
  • Pair events with acceptance-criteria-generator for measurable stories.
  • Review event volume and cost quarterly.
  • Run tracking plan review before each major release.

Output Structure

# Analytics Tracking Plan: [Product]

## Goals
| Metric | Definition | Target |
|--------|------------|--------|

## Funnels
### [Journey name]
[Steps + questions]

## Event Catalog
| Event | Trigger | Properties |
|-------|---------|------------|

## Global Properties
[Schema table]

## AI Events
[If applicable]

## Dashboards
[Wireframe descriptions]

## Privacy & Consent
[Rules]

## Implementation Checklist
[ ] SDK init [ ] Server events [ ] QA validation

Next Skills

OutcomeRecommended Skill
Define north starproduct/north-star-metric-advisor
Design experimentsproduct/okr-builder
AI quality metricsai/ai-evaluation-builder
Observability for engineersarchitecture/observability-planner
PRD success metrics sectionproduct/prd-generator

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.