agentleFS
yuchenmiao04yuchenmiao04/My_skills/skills/matlab/matlab-use-duckdb/SKILL.md

matlab-use-duckdb

Skill8 starsChanged 5 months agoMITView on GitHub

Generates MATLAB code for DuckDB database operations using Database Toolbox. Use when connecting to DuckDB (in-memory or file-based), querying CSV/Parquet/JSON files with SQL, creating development databases, preprocessing out-of-memory data, using duckdb(), installing DuckDB extensions, or using DuckDB as an analytical engine in MATLAB.

The file

We split this skill into the rules it gives an agent

9 rules. About 0 are common, 1 is distinctive, and 8 are very specific to this project.

  • ≈0

    Common · estimate

    Many files like this have a rule of this kind.

  • ≈1

    Distinctive · estimate

    This author's own choices. Worth a look.

  • 8

    Very specific

    Tied to their project's names, files or tools. Probably skip.

  • Common
  • Distinctive
  • Very specific
  1. Very specific:

    ALWAYS: use duckdb() to connect — not database(), not JDBC, not ODBC.

    line 47
  2. Very specific:

    ALWAYS: verify with isopen(conn) and close with close(conn).

    line 48
  3. Very specific:

    All standard functions work: sqlread, fetch, execute, sqlwrite, sqlfind, sqlinnerjoin, sqlouterjoin, commit, rollback.

    line 51
  4. Very specific:

    Use ExcludeDuplicates via databaseImportOptions when reading from database tables (with sqlread).

    line 53
  5. Very specific:

    ALWAYS: use fetch (not sqlread) for file queries — they require SQL syntax like SELECT * FROM read_csv('file.csv').

    line 56
  6. Very specific:

    ALWAYS: use single quotes for file paths inside SQL: read_csv('data.csv').

    line 57
  7. Distinctive:

    Before finalizing DuckDB code, verify:

    line 172
  8. Very specific:

    Solution: Use fetch(conn, "SELECT * FROM read_csv('file.csv')") — sqlread expects table names only.

    line 187
  9. Very specific:

    Solution: Add WHERE, GROUP BY, LIMIT, or aggregation in SQL to reduce result size before it enters MATLAB.

    line 193

≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.

Keep what fits and drop the rest. Your version, ready for your agents.
Files
SKILL.md

MATLAB Database Toolbox Interface to DuckDB

Use when working with DuckDB databases from MATLAB using Database Toolbox. DuckDB is an embedded analytical database engine that ships with Database Toolbox starting in R2026a. It enables SQL-based analytics on files, out-of-memory data preprocessing, and portable development databases — all without external database server configuration.

When to Use This Skill

  • Connecting to a DuckDB database (in-memory or file-based)
  • Creating a new DuckDB database file for development workflows
  • Querying CSV, Parquet, or JSON files directly with SQL
  • Preprocessing large data that doesn't fit in memory before importing into MATLAB
  • Using DuckDB as an analytical engine for filtering, aggregation, joins, or sorting
  • Installing and using DuckDB extensions
  • User mentions keywords: DuckDB, duckdb, analytical engine, embedded database, parquet, CSV analytics, in-memory database, portable database, development database, out-of-memory preprocessing

When NOT to Use

  • Connecting to MySQL, PostgreSQL, SQLite, or other external databases — use their native interfaces or JDBC/ODBC
  • Data fits in memory and only needs standard MATLAB operations — use readtable/readmatrix directly
  • Object-relational mapping — use ORM (ormread/ormwrite with Mappable classes)
  • MongoDB, Cassandra, or Neo4j — use their dedicated Database Toolbox interfaces

What Is DuckDB and Why Does Database Toolbox Ship It?

DuckDB is an embedded, serverless analytical database engine. Unlike MySQL or PostgreSQL, it requires no server, no configuration, and runs in-process within MATLAB.

Why it ships with Database Toolbox (R2026a+):

  • Zero-config database — conn = duckdb() gives you a full SQL engine instantly.
  • Analytical engine for files — Query CSV, Parquet, and JSON files directly with SQL without loading them into memory.
  • Out-of-memory preprocessing — Filter, aggregate, join, and sort datasets larger than memory, then bring only results into MATLAB.
  • Portable development databases — .duckdb or .db files work on any machine with Database Toolbox. No database setup needed.
  • AI agent advantage — An agent's SQL knowledge directly translates to powerful analytical queries.

DuckDB does NOT replace MATLAB's file I/O (readtable, etc.). It is a performant alternative when data exceeds memory or SQL operations are more natural than MATLAB table operations.

Critical Rules

Connection

  • ALWAYS use duckdb() to connect — not database(), not JDBC, not ODBC.
  • ALWAYS verify with isopen(conn) and close with close(conn).

API Surface

  • All standard functions work: sqlread, fetch, execute, sqlwrite, sqlfind, sqlinnerjoin, sqlouterjoin, commit, rollback.
  • DuckDB does NOT support databasePreparedStatement. Use execute or sqlwrite instead.
  • Use ExcludeDuplicates via databaseImportOptions when reading from database tables (with sqlread). For direct file queries (read_csv/read_parquet via fetch), use SELECT DISTINCT in SQL.

File Queries

  • ALWAYS use fetch (not sqlread) for file queries — they require SQL syntax like SELECT * FROM read_csv('file.csv').
  • ALWAYS use single quotes for file paths inside SQL: read_csv('data.csv').

Decision Framework

Which connection mode should I use?

GoalConnectionWhy
Analytical queries on filesduckdb()No persistence needed; query files directly
Temporary workspaceduckdb()Fast, discarded on close
Portable development databaseduckdb("mydata.duckdb")Creates a .duckdb or .db file; works on any machine
Open existing databaseduckdb("existing.db")Read/write access to pre-existing .db or .duckdb file
Read-only shared databaseduckdb("shared.duckdb", ReadOnly=true)Prevents accidental writes

When should I use DuckDB vs. MATLAB file I/O?

ScenarioRecommendation
Small data, simple operationsreadtable / readmatrix
Data exceeds memory, needs filtering/aggregationDuckDB (preprocess in SQL, analyze in MATLAB)
Query across multiple CSV/Parquet filesDuckDB with glob patterns
Portable development databaseDuckDB file-based connection
MATLAB-specific analysis (signal processing, ML)Preprocess in DuckDB, analyze in MATLAB

Common Patterns

Pattern 1: Analytical Engine on Files

conn = duckdb();
result = fetch(conn, "SELECT region, SUM(revenue) as total " + ...
    "FROM read_parquet('sales.parquet') " + ...
    "GROUP BY region ORDER BY total DESC");
close(conn);

Pattern 2: Out-of-Memory Preprocessing

conn = duckdb();
summary = fetch(conn, "SELECT date, AVG(value) as avg_val " + ...
    "FROM read_csv('huge_dataset.csv') " + ...
    "WHERE status = 'valid' " + ...
    "GROUP BY date ORDER BY date");
close(conn);
% summary fits in memory — continue with MATLAB analysis

Pattern 3: Development Database

conn = duckdb("dev.duckdb");
sqlwrite(conn, "experiments", experimentData);
rf = rowfilter("score");
results = sqlread(conn, "experiments", RowFilter=rf.score > 0.8);
close(conn);

Pattern 4: Multi-File Query with Glob

conn = duckdb();
data = fetch(conn, "SELECT * FROM read_parquet('data/year=2024/*.parquet') " + ...
    "WHERE category = 'A'");
close(conn);

Pattern 5: Extensions

conn = duckdb();
execute(conn, "INSTALL httpfs");
execute(conn, "LOAD httpfs");
data = fetch(conn, "SELECT * FROM read_parquet('https://example.com/data.parquet') LIMIT 1000");
close(conn);

For detailed examples, see:

  • File analytics and out-of-memory preprocessing: reference/cards/file-analytics.md
  • Development database workflows: reference/cards/development-database.md
  • DuckDB extensions: reference/cards/extensions.md

Common Mistakes

% WRONG — using database() or JDBC to connect to DuckDB
conn = database("", "", "", "org.duckdb.DuckDBDriver", "jdbc:duckdb:");
% CORRECT
conn = duckdb();

% WRONG — using sqlread for file queries (expects a table name)
data = sqlread(conn, "read_csv('data.csv')");
% CORRECT — use fetch with SQL
data = fetch(conn, "SELECT * FROM read_csv('data.csv')");

% WRONG — double quotes for file paths in SQL
data = fetch(conn, "SELECT * FROM read_csv(""data.csv"")");
% CORRECT — single quotes
data = fetch(conn, "SELECT * FROM read_csv('data.csv')");

% WRONG — loading huge file into MATLAB then filtering
data = readtable("huge.parquet"); filtered = data(data.val > 100, :);
% CORRECT — let DuckDB filter on disk
conn = duckdb();
filtered = fetch(conn, "SELECT * FROM read_parquet('huge.parquet') WHERE val > 100");
close(conn);

% WRONG — using databasePreparedStatement (not supported)
pstmt = databasePreparedStatement(conn, "INSERT INTO t VALUES(?, ?)");
% CORRECT — use sqlwrite
sqlwrite(conn, "t", data);

Checklist

Before finalizing DuckDB code, verify:

  • Connected with duckdb() or duckdb("file.duckdb") / duckdb("file.db") — not database() or JDBC
  • isopen(conn) checked after connection
  • File queries use fetch with SQL (not sqlread)
  • File paths in SQL use single quotes
  • Out-of-memory data preprocessed in DuckDB before importing to MATLAB
  • No databasePreparedStatement usage (not supported)
  • close(conn) called when done

Troubleshooting

Issue: duckdb function not found

  • Solution: Requires R2026a+ with Database Toolbox. Check with ver('database').

Issue: sqlread errors with file query

  • Solution: Use fetch(conn, "SELECT * FROM read_csv('file.csv')") — sqlread expects table names only.

Issue: Permission denied on ReadOnly connection

  • Solution: Reconnect without ReadOnly: conn = duckdb("file.duckdb").

Issue: Out of memory when querying large file

  • Solution: Add WHERE, GROUP BY, LIMIT, or aggregation in SQL to reduce result size before it enters MATLAB.

Issue: File path not found in read_csv/read_parquet

  • Solution: Paths are relative to pwd. Use absolute paths or verify with dir('file.csv').

Issue: Extension install fails

Issue: Type mismatch on sqlwrite

  • Solution: DuckDB supports rich types (ARRAY, LIST, STRUCT, MAP). Use sqlfind(conn, "tableName") to check column types.

Copyright 2026 The MathWorks, Inc.


Download now (ZIP)

Does it work?

Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.

No reviews yet

People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.

  • It worked
  • Partly
  • It didn't
What did you use it for? e.g. Reviewing pull requests in a Rails app

Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.

Your trail

Files you open will collect here, so you can find them again.

Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.

Now do this for your own.

Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.