agentleFS
noahwins-ngnoahwins-ng/equity-data-agent/.claude/skills/dagster-expert/SKILL.md

dagster-expert

Skill1 starsChanged 4 months agoMITView on GitHub

Brief definitions only (see reference files for detailed examples): When integrating with ANY external tool or service, read the Integration libraries index. This contains information about which integration libraries exist, and references on how to create new custom integrations for tools that do not have a published library. The dg CLI is the recommended way to programmatically interact with Dagster (adding definitions, launching runs, exploring project structure, etc.). It is installed as part of the dagster-dg-cli package. If a relevant…

The file

We split this skill into the rules it gives an agent

3 rules. About 0 are common, 3 are distinctive, and 0 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈3

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 0

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    When integrating with ANY external tool or service, read the Integration libraries index.

    line 18
  2. Distinctive:

    ONLY explore the existing project structure if it is strictly necessary to accomplish the user's goal.

    line 24
  3. Distinctive:

    NEVER answer from memory or guess at CLI commands, APIs, or syntax.

    line 39

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

Core Dagster Concepts

Brief definitions only (see reference files for detailed examples):

  • Asset: Persistent object (table, file, model) produced by your pipeline
  • Component: Reusable building block that generates definitions (assets, schedules, sensors, jobs, etc.) relevant to a particular domain.

Integration Workflow

When integrating with ANY external tool or service, read the Integration libraries index. This contains information about which integration libraries exist, and references on how to create new custom integrations for tools that do not have a published library.

dg CLI

The dg CLI is the recommended way to programmatically interact with Dagster (adding definitions, launching runs, exploring project structure, etc.). It is installed as part of the dagster-dg-cli package. If a relevant CLI command for a given task exists, always attempt to use it.

ONLY explore the existing project structure if it is strictly necessary to accomplish the user's goal. In many cases, existing CLI tools will have sufficient understanding of the project structure, meaning listing and reading existing files is wasteful and unnecessary.

Almost all dg commands that return information have a --json flag that can be used to get the information in a machine-readable format. This should be preferred over the default table output unless you are directly showing the information to the user.

UV Compatibility

Projects typically use uv for dependency management, and it is recommended to use it for dg commands if possible:

uv run dg list defs
uv run dg launch --assets my_asset

CRITICAL: Always Read Reference Files Before Answering

NEVER answer from memory or guess at CLI commands, APIs, or syntax. ALWAYS read the relevant reference file(s) from the Reference Index below before responding.

For every question, identify which reference file(s) are relevant using the index descriptions, read them, then answer based on what you read.

Reference Index

<!-- BEGIN GENERATED INDEX -->
  • Asset Selection Syntax — filtering assets by tag, group, kind, upstream, or downstream; AssetSelection in Python, UI search bar, or CLI
  • Environment Variables — configuring environment variables across different environments
  • Asset Patterns — defining assets, dependencies, metadata, partitions, or multi-asset definitions
  • Choosing an Automation Approach — deciding between schedules, sensors, and declarative automation
  • Schedules — time-based automation with cron expressions
  • Declarative Automation — asset-centric condition-based automation using AutomationCondition
  • Asset Sensors — triggering on asset materialization events
  • Basic Sensors — event-driven automation with file watching or custom polling
  • Run Status Sensors — reacting to run success, failure, or other status changes
  • dg check — validating project configuration or definitions
  • create-dagster — creating a new Dagster project from scratch
  • dg dev — starting a local Dagster development instance
  • dg launch — materializing assets or executing jobs locally
  • dg list components — seeing available component types for scaffolding
  • dg list defs — listing or filtering registered definitions
  • Dagster Plus API — dg api, programmatically querying or managing Dagster Plus resources (assets, runs, deployments, code locations, schedules, sensors, secrets, issues, etc.)
  • dg list — exploring project structure (component tree, environment variables, workspace projects)
  • Dagster Plus CLI — dg plus, Dagster Plus authentication, configuration, and deployment; logging in, setting config, creating API tokens, deploying code, pulling env vars, managing dbt manifests
  • dg scaffold component — creating a custom reusable component type
  • dg scaffold defs — adding new definitions (assets, schedules, sensors, components) to a project
  • dg utilities — dg utils, inspecting component types, viewing integrations, refreshing state-backed component cache
  • Creating Components — building a new custom component from scratch
  • Designing Component Integrations — designing a component that wraps an external service or tool; custom integrations
  • Resolved Framework — defining custom YAML schema types using Resolver, Model, or Resolvable
  • Subclassing Components — extending an existing component via subclassing; customize dagster integration component
  • Template Variables — using Jinja2 template variables in component YAML (env, dg, context, or custom scopes)
  • Creating State-Backed Components — building a component that fetches and caches external state
  • Using State-Backed Components — managing state-backed components in production, CI/CD, or refreshing state
  • Deployment Configuration Files — build.yaml, container_context.yaml, dagster_cloud.yaml; Dagster Plus deployment configuration; configuring Docker registry, container context, agent queue; Hybrid deployment files
  • Integration libraries index for 40+ tools and technologies (dbt, Fivetran, Snowflake, AWS, etc.). — integration, external tool, dagster-*; dbt, fivetran, airbyte, snowflake, bigquery, sling, aws, gcp
  • Migration Guides — sensor migration to declarative automation, sensor migration to automation condition
<!-- END GENERATED INDEX -->

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.