agentleFS
Venkateshvenki404224Venkateshvenki404224/shipcraft-skills/skills/dsa-conscious-coding/SKILL.md

dsa-conscious-coding

Skill2 starsChanged 2 days agoMITView on GitHub

Use when designing, implementing, reviewing, or optimizing any app feature so data structures, algorithms, Big O, and real production tradeoffs are considered before code is written.

The file

Show its rules and text

We split this skill into the rules it gives an agent

26 rules. About 0 are common, 24 are distinctive, and 2 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈24

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 2

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Distinctive:

    Use this skill whenever you build or review application code.

    line 17
  2. Distinctive:

    Use a dictionary when you need exact lookup.

    line 25
  3. Distinctive:

    Use a queue when work must be processed in order.

    line 26
  4. Distinctive:

    Use a heap when urgent items must be processed first.

    line 27
  5. Distinctive:

    Use a trie or indexed search when prefix search matters.

    line 28
  6. Distinctive:

    Use graph thinking when records are connected.

    line 29
  7. Distinctive:

    Avoid fetching everything and looping when the database can filter/index.

    line 30
  8. Distinctive:

    Use this skill when:

    line 34
  9. Distinctive:

    Do not use this to overengineer tiny one-off scripts.

    line 43
  10. Distinctive:

    Before writing code, answer these five questions:

    line 47
  11. Distinctive:

    If you cannot answer these, pause and inspect the real requirement first.

    line 55
  12. Distinctive:

    If a report/API frequently filters by a field, consider indexing that field in the DocType or migration.

    line 115
  13. Distinctive:

    Do not blindly index every field.

    line 126
  14. Distinctive:

    If a request sends emails, generates PDFs, syncs APIs, or processes many records, use a background job.

    line 130
  15. Distinctive:

    Do not block the user request for long operations.

    line 140
  16. Distinctive:

    If repeated many times, convert to set:

    line 184
  17. Distinctive:

    Only optimize where data size/access pattern needs it.

    line 222
  18. Distinctive:

    Test with 10 records and 500+ records.

    line 230
  19. Distinctive:

    Before marking a feature done, verify:

    line 235
  20. Distinctive:

    Use SQL filtering instead of Python filtering.

    line 258
  21. Distinctive:

    Use DB indexes instead of building manual BSTs for persistent records.

    line 259
  22. Distinctive:

    Use Redis/queue workers instead of custom in-memory queues for production jobs.

    line 260
  23. Distinctive:

    Use full-text/search engines for serious search.

    line 261
  24. Distinctive:

    Building clever DSA too early: If the dataset is tiny and will stay tiny, simple code is better.

    line 275
  25. Very specific:

    Forgetting cycles in graphs: Always track visited in graph traversal unless the graph is guaranteed acyclic.

    line 281
  26. Very specific:

    Using list.pop(0) as a queue: Use collections.deque for FIFO queues.

    line 283

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

DSA-Conscious Coding

Overview

Use this skill whenever you build or review application code. The goal is not to force complex DSA everywhere. The goal is to choose simple, correct data structures and algorithms that keep the app fast, readable, and maintainable as data grows.

DSA in real projects means asking:

If this data becomes 10x, 100x, or 1000x bigger, will this code still survive?

Good app code usually comes from simple choices made early:

  • Use a dictionary when you need exact lookup.
  • Use a queue when work must be processed in order.
  • Use a heap when urgent items must be processed first.
  • Use a trie or indexed search when prefix search matters.
  • Use graph thinking when records are connected.
  • Avoid fetching everything and looping when the database can filter/index.

When to Use

Use this skill when:

  • Writing a new app feature, API, report, job, or dashboard.
  • Reviewing code that loops over records, filters data, searches, sorts, or builds summaries.
  • Choosing between list, dict, set, queue, heap, tree, graph, or database query.
  • Handling Frappe/ERPNext records, background jobs, reports, workflows, document links, or dashboards.
  • A feature works on small data but may fail on large data.
  • You need to explain the performance or scalability of code.

Do not use this to overengineer tiny one-off scripts. Prefer the simplest structure that satisfies the real data size and access pattern.

Core Rule

Before writing code, answer these five questions:

  1. What data am I storing?
  2. How will I access it most often?
  3. Will I search by exact key, prefix, range, priority, order, or connection?
  4. What is the expected data size now and later?
  5. Can the database do this better than in-memory Python?

If you cannot answer these, pause and inspect the real requirement first.

Quick Decision Table

NeedPreferWhy
Exact lookup by ID/email/nameHashMap / dict / DB indexed lookupAverage O(1) in memory; fast indexed DB lookup
Remove duplicates or membership checksetAverage O(1) membership
Preserve ordered recent itemslistSimple, direct index access
Process first-created item firstQueue / dequeFIFO, O(1) append/pop-left
Process latest action firstStack / list.append/popLIFO, O(1) push/pop
Process most urgent item firstHeap / Priority QueueO(log n) insert/pop priority item
Search sorted dataBinary SearchO(log n) if data is sorted
Prefix/autocomplete searchTrie or search indexSearch by typed prefix, not full scan
Parent-child hierarchyTreeCategories, workflows, accounts, org charts
Connected records/routes/dependenciesGraphBFS/DFS/path/dependency reasoning
Frequent range queries/reportsDB indexes, prefix sums, materialized summariesAvoid repeated full scans
Complex filtering over many recordsDatabase query with filters/indexesDB should reduce data before Python sees it

Frappe/ERPNext Rules

1. Do not fetch all records and loop unless data is truly small

Bad:

orders = frappe.get_all("Sales Order", fields=["name", "customer", "status"])

for order in orders:
    if order.customer == customer:
        return order

Better:

orders = frappe.get_all(
    "Sales Order",
    filters={"customer": customer},
    fields=["name", "customer", "status"],
)

Why:

  • Bad code moves filtering into Python.
  • Better code lets the database filter and use indexes.

2. Use frappe.db.get_value for exact document lookup

Good:

status = frappe.db.get_value("Sales Order", sales_order_name, "status")

This is like exact-key lookup. Do not load unnecessary fields or scan lists.

3. Add indexes when filters become common

If a report/API frequently filters by a field, consider indexing that field in the DocType or migration.

Common indexed fields:

  • name
  • owner
  • creation
  • modified
  • status fields used in dashboards
  • foreign keys / Link fields used in reports

Do not blindly index every field. Indexes speed reads but add write/storage cost.

4. Use background queues for slow work

If a request sends emails, generates PDFs, syncs APIs, or processes many records, use a background job.

frappe.enqueue(
    "my_app.tasks.process_orders",
    queue="long",
    timeout=1500,
)

Do not block the user request for long operations.

Big O Check Before Finalizing Code

For any loop, ask:

What is n?
How big can n become?
Is this loop inside another loop?
Can this be converted to dict/set/DB filter?

Common Patterns

Bad nested lookup

for order in orders:
    for customer in customers:
        if order.customer == customer.name:
            order.customer_name = customer.customer_name

This is usually O(n * m).

Better:

customer_map = {c.name: c.customer_name for c in customers}

for order in orders:
    order.customer_name = customer_map.get(order.customer)

This becomes roughly O(n + m).

Bad repeated list membership

if user_email in email_list:
    print("exists")

If repeated many times, convert to set:

email_set = set(email_list)

if user_email in email_set:
    print("exists")

Bad sorting inside loop

for item in items:
    scores.sort()

Sorting repeatedly is expensive. Sort once outside the loop unless the data truly changes every time.

Practical Coding Workflow

  1. Understand access pattern
    • Exact lookup?
    • Prefix search?
    • Sorted search?
    • Priority processing?
    • Relationship traversal?
  2. Choose the simplest fitting structure
    • list for ordered small collections.
    • dict for key-value lookup.
    • set for membership/deduplication.
    • deque for FIFO queues.
    • heapq for priority queues.
    • DB filter/index for persistent large data.
  3. Write clear baseline code first
    • Make it correct.
    • Add tests.
    • Only optimize where data size/access pattern needs it.
  4. Add Explain Mode when teaching or debugging
    • Return steps checked.
    • Return Big O label.
    • Return trace messages.
  5. Verify with realistic data
    • Test with 10 records and 500+ records.
    • For production-like features, test with enough data to expose slow loops.

App Feature Checklist

Before marking a feature done, verify:

  • No unnecessary get_all without filters for large DocTypes.
  • No nested loops over large lists unless justified.
  • Exact lookups use dict/set or DB indexed lookup.
  • Prefix search has a proper strategy: Trie/search index/DB search.
  • Priority work uses heap/priority queue thinking.
  • Background work uses queue/worker pattern.
  • Relationship traversal tracks visited to avoid cycles.
  • Sorting is not repeated unnecessarily.
  • Tests cover small, empty, duplicate, and large-ish data cases.
  • Code comments explain the DSA choice where it is not obvious.

DSA Is Not Always Manual DSA

In real apps, the best DSA choice is often:

Let the database do it.

Examples:

  • Use SQL filtering instead of Python filtering.
  • Use DB indexes instead of building manual BSTs for persistent records.
  • Use Redis/queue workers instead of custom in-memory queues for production jobs.
  • Use full-text/search engines for serious search.

Manual DSA is useful for:

  • In-memory transformations.
  • Small local indexes.
  • Algorithms inside one request/job.
  • Learning, testing, and explaining performance.
  • Cases where database/query tools are not enough.

Common Pitfalls

  1. Using lists for everything. Lists are simple, but repeated search by value is O(n). Use dict/set when lookup matters.
  2. Building clever DSA too early. If the dataset is tiny and will stay tiny, simple code is better.
  3. Ignoring database indexes. In Frappe apps, DB filtering and indexing are often more important than Python data structures.
  4. Nested loops hidden inside helpers. A helper function inside a loop may create accidental O(n²) behavior.
  5. Forgetting cycles in graphs. Always track visited in graph traversal unless the graph is guaranteed acyclic.
  6. Using list.pop(0) as a queue. Use collections.deque for FIFO queues.
  7. Sorting repeatedly. Sorting inside loops can quietly become a performance killer.
  8. Optimizing without tests. Performance improvements are useless if behavior changes. Add tests first.

Verification Checklist

  • Data access pattern is identified.
  • Chosen structure/algorithm matches the access pattern.
  • Big O is understood and documented when relevant.
  • Frappe/database work is pushed into filters/indexes where possible.
  • Code avoids unnecessary full scans and nested loops.
  • Edge cases are tested: empty data, missing keys, duplicates, large-ish data.
  • If performance matters, compare baseline vs optimized behavior.
  • The final implementation remains readable and maintainable.

Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.