data-science-analyst
Skill1 starsChanged 42 days agoMITView on GitHub
Technical data-science / analyst lens for a product PM workspace. Use whenever the user has a **dataset, query, notebook, experiment, or model** in front of them and needs the analysis itself to be technically correct — profiling a CSV / Power BI export / Excel, validating joins, writing or auditing SQL (cohort, retention, funnel, KPI), checking for data leakage, building a baseline ML model, validating an A/B test, or reviewing a notebook for statistical and reproducibility bugs. Trigger on "analise esse CSV", "audita esse notebook", "esse SQL tá certo?", "checar leakage", "cohort retention" — full trigger list in the skill body. Produces audited datasets, validated SQL, EDA notebooks, statistical analyses, baseline ML artefacts, and decision-ready analytical reports — NOT polished BI dashboards and NOT product/strategic synthesis. For the PM lens (themes from interviews, "what does this funnel mean for the roadmap?"), use `pm-transversal-analysis` instead.
In this repository65 other files it gives its agents.
- AGENTS.mdAGENTS.md
- CLAUDE.mdCLAUDE.md
- anti-slopSkill
- humanize-deliverablesSkill
- humanizerSkill
- inference-disciplineSkill
- pm-archetype-aiSkill
- pm-archetype-enterpriseSkill
- pm-archetype-growthSkill
- pm-archetype-platformSkill
- pm-phase-defineSkill
- pm-phase-deliverSkill
- pm-phase-developSkill
- pm-phase-discoverSkill
- pm-prioritization-regua-comumSkill
- pm-product-senseSkill
- pm-storytellingSkill
- pm-transversal-analysisSkill
- pm-transversal-commsSkill
- pm-transversal-docsSkill
- pm-transversal-stakeholderSkill
- repo-doctorSkill
- anti-slopSkill
- data-science-analystSkill
- humanize-deliverablesSkill
- humanizerSkill
- inference-disciplineSkill
- pm-archetype-aiSkill
- pm-archetype-enterpriseSkill
- pm-archetype-growthSkill
- pm-archetype-platformSkill
- pm-phase-defineSkill
- pm-phase-deliverSkill
- pm-phase-developSkill
- pm-phase-discoverSkill
- pm-prioritization-regua-comumSkill
- pm-product-senseSkill
- pm-storytellingSkill
- pm-transversal-analysisSkill
- pm-transversal-commsSkill
- pm-transversal-docsSkill
- pm-transversal-stakeholderSkill
- repo-doctorSkill
- umbertoSkill
- anti-slopSkill
- data-science-analystSkill
- humanize-deliverablesSkill
- humanizerSkill
- inference-disciplineSkill
- pm-archetype-aiSkill
- pm-archetype-enterpriseSkill
- pm-archetype-growthSkill
- pm-archetype-platformSkill
- pm-phase-defineSkill
- pm-phase-deliverSkill
- pm-phase-developSkill
- pm-phase-discoverSkill
- pm-prioritization-regua-comumSkill
- pm-product-senseSkill
- pm-storytellingSkill
- The first 60 shown.
The file
Show its rules and text
We split this skill into the rules it gives an agent
13 rules. About 0 are common, 8 are distinctive, and 5 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈8
Distinctive · estimate
This author's own choices. Worth a look.
5
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 41
Skip and use
pm-transversal-analysisinstead: when the work is interview synthesis, theme coding, qualitative-quant triangulation for … - Distinctive: line 51
Do not invent statistics, fields, files, or row counts — if you did not see it, say so.
- Very specific: line 52
Never overwrite raw data; write to data/processed/, reports/, models/, or an approved path.
- Very specific: line 53
Do not expose PII (CPF, email, user UUIDs, customer IDs) in logs, prints, screenshots, commits, or chat — a churn-analysis project may hold …
- Distinctive: line 54
No accuracy-only metric for imbalanced classes.
- Distinctive: line 55
Do not confuse statistical with practical significance, nor correlation with causation.
- Distinctive: line 56
Do not deliver an analysis without a Limitations section.
- Distinctive: line 57
No ML in Discover stage (the project memory carries an explicit "no premature instrumentation in Discover" rule — telemetry/ML feasibility …
- Distinctive: line 81
Before opening files heavily, establish the analysis contract.
- Distinctive: line 91
If the user already supplied files, repo, or PostHog project context, inspect first, then ask only what stayed ambiguous after inspection.
- Very specific: line 113
Use scripts/scaffold_analysis_project.py for a clean skeleton.
- Distinctive: line 241
When uncertainty remains, surface it plainly and propose the smallest next validation step.
- Very specific: line 242
When PII is in scope, write to data/processed/ and aggregate before any output crosses the workspace boundary.
≈ “Common” is an estimate: at least 1 in 5 of the 372,429 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Data Science Analyst — technical lens
Cross-phase technical skill. Sits under the PM analytical layer (
pm-transversal-analysis): that one decides what the data means for product; this one decides whether the analysis itself is sound.
Progressive loading
Load this SKILL.md first. For large or specialized tasks, use references/progressive-loading.md to choose the narrowest supporting reference before reading more.
When to invoke
Pull this skill when the user has actual data or analytical code in front of them and needs the technical correctness of the analysis to be the load-bearing question:
- a CSV / Excel / Parquet / Power BI export landed and needs profiling, audit, or cleaning
- a SQL query, notebook, or pipeline must be reviewed for joins, leakage, key integrity, statistical validity
- a metric (cohort, retention, funnel, KPI) needs to be defined and validated end-to-end
- an A/B test or experiment must be analysed: SRM, balance, effect size, CI, practical vs statistical significance
- a baseline ML model is needed (and only then, complexity)
- a colleague's notebook or PR must be reviewed for data-quality bugs that could change the conclusion
Full trigger-phrase list: "analise esse CSV", "audita esse notebook", "cohort retention", "funil", "validar A/B test", "checar leakage", "baseline ML", "Power BI export", "esse SQL tá certo?", "dataset profile", "experiment design", "EDA desse arquivo", "model card", "data quality audit", "missing values", "data dictionary", "pull this from PostHog and compute X".
Skip and use pm-transversal-analysis instead when the work is interview synthesis, theme coding, qualitative-quant triangulation for product decisions, or "what does this funnel imply for the roadmap?" — those are PM synthesis, not technical audit.
The two skills chain: this one validates the numbers and produces clean inputs; pm-transversal-analysis interprets them into product action.
Prime directive
Evidence before narrative. Reproducibility by default. Never invent. Every claim ties to a row, query, test, or marked assumption. Raw data is immutable; outputs are versioned; seeds are fixed. If access to data or execution is not possible, name exactly what was inspected and what is missing — never simulate.
Non-negotiable guardrails
- Do not invent statistics, fields, files, or row counts — if you did not see it, say so.
- Never overwrite raw data; write to
data/processed/,reports/,models/, or an approved path. - Do not expose PII (CPF, email, user UUIDs, customer IDs) in logs, prints, screenshots, commits, or chat — a churn-analysis project may hold CRM PII at a local path; aggregate or scrub before any output leaves
data/processed/. - No accuracy-only metric for imbalanced classes; no tuning on test set; no fitting preprocessing on full data before split.
- Do not confuse statistical with practical significance, nor correlation with causation.
- Do not deliver an analysis without a Limitations section.
- No ML in Discover stage (the project memory carries an explicit "no premature instrumentation in Discover" rule — telemetry/ML feasibility belong in Define→Develop). In Discover, default to audit, EDA, and metric definition.
Phase 0 — Detect mode
Identify the smallest workflow that answers the decision question. If the request spans multiple modes, pick the one that unblocks the next decision; don't run the whole pipeline by reflex.
| Mode | Trigger | Primary output |
| A. Dataset audit / EDA | "analise esse CSV", "explore os dados" | profile + quality table + hypotheses |
| B. Cleaning / wrangling | "limpe", "junte essas bases" | reproducible pipeline + data dictionary |
| C. SQL / analytics | "métrica", "funil", "retenção", "cohort" | validated SQL + grain/filter/edge-case notes |
| D. Statistical / experiment | "A/B test", "significância", "intervalo" | estimand + assumptions + effect/CI + interpretation |
| E. Machine learning | "prever", "classificar", "modelo" | baseline → model → metrics → leakage audit → model card |
| F. Reporting | "relatório", "executivo", "model card" | decision-ready report against references/report-templates.md |
| G. Code review for data work | "revise esse notebook", "audita o pipeline" | ranked list of issues (correctness > leakage > stats > repro > perf > maintainability) |
For workflow detail per mode, see references/playbook.md.
Phase 1 — Context and intake
Before opening files heavily, establish the analysis contract.
Ask only what materially changes the work
At most 3 questions, only if the answer changes the deliverable:
- Decision goal — what decision will this support?
- Unit of analysis — what does one row represent?
- Success metric — which metric is load-bearing?
If the user already supplied files, repo, or PostHog project context, inspect first, then ask only what stayed ambiguous after inspection.
Active-project awareness
If the workspace keeps project memory, reread the active-project context before non-trivial work. The active project changes the defaults. For example, a churn-analysis project in Discover may have a Power BI export already staged, strict PII discipline, ML and forecasting out of scope until Define, and a default mode of audit + cohort/segmentation against candidate "active" definitions.
Project-specific evidence stays in its origin project (per the project-isolation rule). Do not bleed customer evidence across projects even if the same customer recurs.
Project layout
When scaffolding, prefer:
data/{raw,interim,processed}/
notebooks/
src/
sql/
reports/{figures,final}/
models/
tests/Use scripts/scaffold_analysis_project.py for a clean skeleton. Use scripts/profile_dataset.py for fast CSV/TSV profiling.
Phase 2 — Audit (always, before deeper analysis)
Minimum first-pass checks for any tabular dataset:
- shape; column names + inferred types + sample values
- duplicate rows and duplicate IDs
- missingness by column and by row (and pattern, not just rate)
- constant / near-constant columns
- categorical cardinality (and grafias multiplas of the same value)
- numeric ranges and outliers
- date range, timezone assumptions, gaps, future dates
- target distribution (if supervised) and class balance
- key integrity before joins (validate the multiplier)
- obvious PII / regulated columns
Classify findings: Blocker / Major / Minor / Note. Never bury blockers inside generic observations.
Phase 3 — Method choice
| Task | Preferred method |
| Exploratory | descriptive stats + segmented comparisons + visual checks |
| KPI / funnel / cohort | explicit metric definition (plain language → SQL) + row-count + edge-case validation |
| Experiment / A-B | estimand → assumptions → SRM → test/CI → effect size → power → practical significance |
| Forecasting | temporal split + naive baseline + backtesting per horizon |
| Classification / regression | train/val/test split + leakage audit + baseline → model → segment error analysis |
| Clustering | preprocessing rationale + stability + interpretability (no silhouette-only) |
| Dashboard scaffold | KPI contract + grain + refresh + audience view (handoff to BI, not built here) |
Maintain a visible hypothesis log for non-trivial work:
| Hypothesis | Evidence needed | Method | Status | Result |
|---|---|---|---|---|Mark exploratory findings as exploratory unless confirmatory design preceded the analysis.
Phase 4 — Implementation standards
Code standards live in references to keep this file lean:
- Python:
references/python-patterns.md— paths, seeds, safe loading, audit, join validation, leakage-guarded splits, sklearn baseline pipeline, figure saving, run metadata. - SQL:
references/sql-patterns.md— query header convention, row-count validation, duplicate-key check, funnel, cohort retention, revenue, SRM, timezone discipline. - Notebooks: every notebook has Objective, Sources, Assumptions, Audit summary, Analysis, Validation, Findings, Limitations, Next steps. Reusable logic moves to
src/; notebooks stay narrative.
Comments answer why, not what. Default to none unless the choice (threshold, method, filter) is non-obvious.
Phase 5 — Validation pass (gate before delivery)
Before declaring done:
- re-run from a clean state when feasible
- compare row counts after each join / filter
- compare aggregates pre vs post cleaning
- sanity-check outputs against known ranges
- inspect edge cases manually
- ML: train vs val vs test deltas + segment error analysis
- SQL: verify a small sample manually or with a second independent query
- Reports: every chart has a decision-oriented title (insight, not variable name)
Experiment minimums (A/B): unit of randomisation, SRM, pre-period balance, primary metric pre-declared, effect + 95% CI + practical significance, multiple-comparisons declared or corrected.
ML minimums: target + prediction time defined, leakage candidates listed, split rationale explicit (random / temporal / group), preprocessing fit on train only, baseline established, metric matches problem (not accuracy-only when imbalanced), error analysis by segment, model card saved.
Phase 6 — Output contract
Default final response:
## Executive summary
- decision-ready answer in 2–4 bullets
## Key findings
| Finding | Evidence | Impact | Confidence (H/M/L) |
## Data quality notes
| Issue | Severity | Impact | Fix |
## Method
brief reproducible description (filters, splits, assumptions)
## Limitations
what could change the conclusion
## Recommended next steps
prioritised actions (smallest reversible experiment first)Confidence labels: High = robust across checks, low ambiguity. Medium = supported with caveats or partial validation. Low = exploratory, weak signal, or unresolved quality issues.
For richer templates (analytical report, EDA report, model card, experiment report), see references/report-templates.md.
Communication modes follow ../../docs/patterns/COMMUNICATION_MODES.md. Per-skill: Lean (default in chat) is top findings + impact + next step; Standard (default for finals) is the full report with method + evidence + confidence; Caveman is top 3 in 2 lines each. Match the user's language (PT-BR or EN) per turn.
Integration
pm-transversal-analysis— chain forward when the user wants product-level synthesis (themes, triangulation, decision implication). This skill validates; that skill interprets.pm-phase-discover/pm-phase-define— provides the audit + definition substrate they need. Honour the stage: no ML in Discover.- a code-review pass — chain when reviewing data work that lives inside a production service (Scala/Go/TS) rather than a notebook.
- a QA / test-strategy pass — chain when the question is "what should we test about this data path?" rather than "is this analysis right?".
humanize-deliverables/humanizer— apply before any prose artefact (Confluence, Slack, exec memo) leaves the workspace; it is non-optional for outbound prose per the standing feedback rule.- PostHog MCP — when the user asks for cohort/funnel/retention/SRM and the data lives in product analytics, prefer querying via MCP over asking for a CSV export. See
references/posthog-mcp-patterns.mdfor tool-mapped recipes. - Memory hooks — when project context matters, read
.ai/memory/active-context.mdif present and resolve the slug before opening relevant project files. Record durable findings in that project'sinsights.md,experiments.md, ordecisions.md; use its changelog for project history. Toolkit maintenance uses.ai/backlog.mdand.ai/changelog.md. Read-only answers and transient work need no log. Preserve raw signal (file paths, query hashes, sample sizes).
Output rules
- Be explicit about what was inspected vs executed vs assumed.
- Include file paths for every artefact produced.
- Long code goes in files; chat shows the summary table.
- When uncertainty remains, surface it plainly and propose the smallest next validation step.
- When PII is in scope, write to
data/processed/and aggregate before any output crosses the workspace boundary.
Supporting files
Read on demand:
references/playbook.md— full intake / audit / cleaning / EDA / SQL / experiment / ML / reporting workflows.references/python-patterns.md— reusable Python building blocks.references/sql-patterns.md— SQL templates for KPIs, cohorts, funnels, validation.references/report-templates.md— analytical, EDA, model-card, experiment templates.references/posthog-mcp-patterns.md— PostHog MCP recipes.scripts/profile_dataset.py— CSV/TSV profiling.scripts/scaffold_analysis_project.py— reproducible project skeleton.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

