agentleFS
Sign inStart free
KhaledAufDBKhaledAufDB/lakeforge/CLAUDE.md

lakeforge

CLAUDE.md0 starsChanged 2 months agoMITView on GitHub

Lakeforge onboards new data sources into a Databricks lakehouse using a bronze / silver / gold medallion architecture with conformed dimensions. The target workspace is named once, in lakeforge.yml. 1. Conformed dimensions are sacred. Before creating any dimension, read shared/conformed/dimensions.yml. If the entity exists in gold, reuse it through an FK. 2. Plan before build. Never generate notebooks without specs/<source>-integration-plan.md. The plan is the contract and records the rejected alternatives. 3. Bronze is raw, silver is clean, gold is modelled.…

The file

We split this CLAUDE.md into the rules it gives an agent

3 rules. About 0 are common, 1 is distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈1

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    Conformed dimensions are sacred: Before creating any dimension, read shared/conformed/dimensions.yml.

    line 10
  2. Very specific:

    Plan before build: Never generate notebooks without specs/<source>-integration-plan.md.

    line 12
  3. Distinctive:

    Bronze is raw, silver is clean, gold is modelled: No business logic in bronze.

    line 14

≈ “Common” is an estimate: at least 1 in 5 of the 26,980 CLAUDE.md files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
CLAUDE.md

Lakeforge — Agent Instructions

Purpose

Lakeforge onboards new data sources into a Databricks lakehouse using a bronze / silver / gold medallion architecture with conformed dimensions. The target workspace is named once, in lakeforge.yml.

Critical rules

  1. Conformed dimensions are sacred. Before creating any dimension, read shared/conformed/dimensions.yml. If the entity exists in gold, reuse it through an FK.
  2. Plan before build. Never generate notebooks without specs/<source>-integration-plan.md. The plan is the contract and records the rejected alternatives.
  3. Bronze is raw, silver is clean, gold is modelled. No business logic in bronze; no source-specific naming in gold.
  4. Every MERGE is change-detected and null-safe. WHEN MATCHED AND (… IS DISTINCT FROM …), <=> for nullable keys, CDF input deduplicated first. Never UPDATE SET * blindly, never a try: merge / except: overwrite.
  5. Incremental by default. Bronze loads by stored watermark with a lookback; full reload is a documented exception.
  6. No credentials in the repo. Notebooks call dbutils.secrets.get(scope, key); local scripts use the Databricks SDK default auth chain. Config files hold secret key names only.
  7. Ask on sticky decisions. A grain, a conform-vs-distinct call, a source-system id, a table name, PII handling — if more than one answer is defensible, ask; never default silently.
  8. Test everything in gold. Row count, unique grain, referential integrity, FK resolution rate, and a 0/0/0 second run.

Expert files — read, don't recall

FileRead it when
.claude/commands/experts/principles.mdalways (loaded by /prime)
.claude/commands/experts/data-modelling.mdplanning: what earns a gold object, conformance, naming
.claude/commands/experts/databricks.mdbuilding or debugging: MERGE, watermarks, types, clustering, governance, ops

These are the single home of each fact. Edit the fact there; link to it elsewhere; never restate it.

Workflow

/prime → /forge-plan <source> → (human review) → /forge-build <source> → /forge-review <source> → deploy → /forge-test <source>

File locations

WhatWhere
Workspace targetlakeforge.yml
Source integrationssources/<name>/
Integration plansspecs/<name>-integration-plan.md
Conformed dimension registryshared/conformed/dimensions.yml
Loaders / transforms / quality / deployshared/
Worked example sourcesources/example/
Unit teststests/

Naming

ObjectPattern
bronze table<bronze_schema>.<source>__<entity> (source names verbatim)
silver table<silver_schema>.<source>__<entity> (snake_case)
gold dimension<gold_schema>.<entity> (singular)
gold fact<gold_schema>.fact_<business_process> (plural head noun)
surrogate<table>_id = xxhash64(business key…, source_system_id)
business key carrysrc_<source column>
UTC timestamp*_at

Git workflow

Branch per source: feat/<source-name>. Build in sources/<source-name>/, open a PR, merge triggers the workspace Git-folder sync. Never force-push main.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

Check your setup, free