ddia-data-modeling
Skill1 starsChanged 6 months agoMITView on GitHub
Data modeling, storage engines, and encoding choices distilled from "Designing Data-Intensive Applications" (Kleppmann, 2nd ed) chapters 3-5. Covers relational, document, graph, and event-sourced models; LSM, B-tree, in-memory, and columnar storage; specialized indexes; encoding formats; and modes of dataflow.
In this repository21 other files it gives its agents.
- ai-context-engineeringSkill
- ai-context-engine-productionSkill
- ai-context-engineSkill
- ai-dataset-engineeringSkill
- ai-evaluationSkill
- ai-finetuningSkill
- ai-foundation-modelsSkill
- ai-inference-optimizationSkill
- ai-multi-agent-mcpSkill
- ai-production-architectureSkill
- ai-prompt-engineeringSkill
- ai-rag-and-agentsSkill
- ai-rag-defenseSkill
- ddia-architectureSkill
- ddia-batch-stream-processingSkill
- ddia-data-ethicsSkill
- ddia-replication-shardingSkill
- ddia-transactions-consistencySkill
- job-description-analyzerSkill
- resume-content-writerSkill
- resume-fit-scorerSkill
The file
Files
DDIA Data Modeling
A reference skill for designing data models, picking storage engines, and choosing encoding/dataflow strategies. Knowledge is split across 8 categories with progressive disclosure: load only the references relevant to the decision in front of you.
Quick Start
- Open
guidelines.md— find your task, symptom, or topic. - Load only the listed reference files (typically 1-3).
- Apply the rules and patterns; consult
examples.mdfiles for concrete code/schema. - For multi-step decisions, follow the relevant workflow under
workflows/.
Contents
References
| Category | Files | Purpose |
relational-document-models | knowledge, rules, examples | Relational vs document trade-offs, joins, schema-on-write vs schema-on-read |
graph-models | knowledge, rules, examples | Property graphs, triple stores, Cypher/SPARQL/Datalog, traversal patterns |
event-sourcing-cqrs | knowledge, rules, examples | Append-only event logs, CQRS read views, DataFrames for analytics |
oltp-storage | knowledge, rules, examples | LSM-trees, B-trees, in-memory engines, write/read trade-offs |
olap-storage | knowledge, rules, examples | Columnar layout, vectorized execution, separation of storage and compute |
specialized-indexes | knowledge, rules, examples | Multidimensional, geo (R-tree), full-text, vector indexes |
encoding-formats | knowledge, rules, examples | JSON, Protobuf, Avro, Thrift; schema evolution and compatibility |
dataflow-modes | knowledge, rules, examples | Databases, REST/RPC services, messaging, durable execution |
Workflows
workflows/choosing-database-type.md— relational vs document vs graph vs columnarworkflows/choosing-storage-engine.md— LSM vs B-tree vs in-memory vs columnarworkflows/choosing-encoding-format.md— JSON vs Avro vs Protobuf vs Thriftworkflows/designing-event-sourced-system.md— event log + CQRS read views
Workflows
| Task | Workflow |
| Choose between relational/document/graph/columnar database | workflows/choosing-database-type.md |
| Choose storage engine (LSM/B-tree/in-memory/columnar) | workflows/choosing-storage-engine.md |
| Pick wire encoding format & schema evolution strategy | workflows/choosing-encoding-format.md |
| Design an event-sourced system with CQRS | workflows/designing-event-sourced-system.md |
Guidelines
See guidelines.md for:
- Task-based file selection (database choice, storage choice, analytics, encoding, services)
- Symptom/question lookup (joins, connected data, audit, write throughput, etc.)
- Topic-by-topic file index (8 categories, 24 files)
- Decision trees for the most common selection paths
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.