OpenSpec Instructions

These instructions are for AI assistants working in this project.

Always open @/openspec/AGENTS.md when the request:

• Mentions planning or proposals (words like proposal, spec, change, plan)
• Introduces new capabilities, breaking changes, architecture shifts, or big performance/security work
• Sounds ambiguous and you need the authoritative spec before coding

Use @/openspec/AGENTS.md to learn:

• How to create and apply change proposals
• Spec format and conventions
• Project structure and guidelines

Keep this managed block so 'openspec update' can refresh the instructions.

CLAUDE.md

This file provides guidance to Claude Code (claude.ai/code) when working with code in this repository.

Important: This project uses English as its primary language. All documentation, code comments, and commit messages should be in English. This includes LOG.md, tasks/, issues and pull request descriptions — no exceptions.

Project Overview

CKAN MCP Server - MCP (Model Context Protocol) server for interacting with CKAN-based open data portals (dati.gov.it, data.gov, demo.ckan.org, etc.).

The server exposes MCP tools for:

• Advanced dataset search with Solr syntax
• DataStore queries for tabular data analysis
• Organization and group exploration
• Complete metadata access

Build and Development

Main Commands

# Build project (uses esbuild - fast and lightweight)
npm run build

# Run test suite (191 tests - unit + integration)
npm test

# Watch mode for tests during development
npm run test:watch

# Test coverage report
npm run test:coverage

# Start server in stdio mode (default for local integration)
npm start

# Start server in HTTP mode (for remote access)
# HTTP requires a domain allowlist (or CKAN_HTTP_ALLOW_ALL=true to opt out)
CKAN_ALLOWED_DOMAINS="www.dati.gov.it" TRANSPORT=http PORT=3000 npm start

# Watch mode for development
npm run watch

# Build + run
npm run dev

# Cloudflare Workers deployment (v0.4.0+)
npm run build:worker      # Build for Workers
npm run dev:worker        # Test locally with Wrangler
npm run deploy            # Deploy to Cloudflare Workers

Local MCP Client Build Test

Before deploying, you can test the current dev build by pointing your MCP client at the Node entrypoint in dist/:

1. Build: npm run build
2. Example absolute path: /home/aborruso/git/idee/ckan-mcp-server/dist/index.js (adjust to your local checkout)

GitHub CLI Notes

When creating issues with multi-line bodies, avoid literal \n in --body. Use a here-doc or -F - / --body-file - to preserve newlines:

cat <<'EOF' | gh issue create --title "Title" --body-file - --repo ondata/ckan-mcp-server
Line 1
Line 2
EOF

When writing GitHub release notes, avoid literal \n in --notes. Always use a here-doc and --notes-file - so line breaks render correctly:

cat <<'EOF' | gh release create v0.X.Y --title "v0.X.Y - Title" --notes-file -
## What's New

### Changes
- Item 1
- Item 2

### No Breaking Changes
- All existing functionality preserved

**Full Changelog**: https://github.com/ondata/ckan-mcp-server/compare/v0.X-1.Y...v0.X.Y
EOF

Build System

The project uses esbuild for compilation and vitest for testing:

• Build: Ultra-fast builds (milliseconds instead of minutes)
• Tests: 191 tests (unit + integration) with 100% success rate
• Coverage: ~39% overall (utils: 98%, tools: 15-20%) - available via vitest with v8 coverage engine

The build:tsc script is available as a fallback but can cause memory issues in some environments (particularly WSL). Always use npm run build which uses esbuild.

The esbuild build bundles all internal modules but keeps external dependencies (@modelcontextprotocol/sdk, axios, express, zod) as external, so they must be present in node_modules.

Testing

The project has a comprehensive test suite using Vitest:

tests/
├── unit/
│   ├── formatting.test.ts    # Utility functions (19 tests)
│   ├── http.test.ts           # HTTP client (11 tests)
│   └── uri.test.ts            # URI parsing (11 tests)
├── integration/
│   ├── package.test.ts        # Package tools (31 tests)
│   ├── organization.test.ts   # Organization tools (6 tests)
│   ├── datastore.test.ts      # DataStore tools (19 tests)
│   ├── resources.test.ts      # MCP Resources (11 tests)
│   └── status.test.ts         # Status tools (2 tests)
└── fixtures/
    ├── responses/             # Success response mocks
    └── errors/                # Error scenario mocks

Test Coverage: 191 tests total (117 unit + 74 integration)

When making changes:

1. Run tests before committing: npm test
2. Ensure all tests pass
3. Add tests for new features or bug fixes
4. Follow existing test patterns in tests/ directory

Test target rule: Never use demo.ckan.org for tests. Always use https://www.dati.gov.it/opendata.

Before committing and pushing: For any locally testable change, always:

1. Build locally: npm run build
2. Run automated tests: npm test
3. Run real HTTP server tests to verify end-to-end behavior:
   • Start: CKAN_ALLOWED_DOMAINS="www.dati.gov.it,dati.comune.messina.it" TRANSPORT=http PORT=3001 node dist/index.js & disown
   • Call each affected tool via curl against a real CKAN portal
   • Stop: kill $(lsof -ti:3001)

Portal selection for HTTP tests:

• General search / organization tools: https://www.dati.gov.it/opendata
• DataStore tools: https://dati.comune.messina.it or https://open.canada.ca/data — dati.gov.it does NOT have DataStore
• To find a valid datastore resource_id: curl -s "https://dati.comune.messina.it/api/3/action/package_search?q=*:*&rows=20" | jq -r '[.result.results[].resources[] | select(.datastore_active==true) | .id][0]'

Architecture

Code Structure

The server is implemented with a modular structure to improve maintainability and testability:

src/
├── index.ts              # Entry point Node.js (42 lines)
├── worker.ts             # Entry point Cloudflare Workers (95 lines) [v0.4.0+]
├── server.ts             # MCP server setup (12 lines)
├── types.ts              # Types & schemas (16 lines)
├── utils/
│   ├── http.ts           # CKAN API client (51 lines)
│   └── formatting.ts     # Output formatting (37 lines)
├── tools/
│   ├── package.ts        # Package tools (350 lines)
│   ├── organization.ts   # Organization tools (341 lines)
│   ├── datastore.ts      # DataStore tools (146 lines)
│   └── status.ts         # Status tools (66 lines)
├── resources/            # MCP Resource Templates
│   ├── index.ts          # Resource registration (19 lines)
│   ├── uri.ts            # URI parsing utilities (50 lines)
│   ├── dataset.ts        # Dataset resource (56 lines)
│   ├── resource.ts       # Resource resource (56 lines)
│   └── organization.ts   # Organization resource (58 lines)
└── transport/
    ├── stdio.ts          # Stdio transport (12 lines)
    └── http.ts           # HTTP transport (27 lines)

Total: ~1445 lines (including Workers deployment)

Note: worker.ts (v0.4.0+) is an alternative entry point for Cloudflare Workers deployment. Tool handlers (tools/*) are shared between Node.js and Workers runtimes.

The server (src/index.ts):

1. Entry Point (index.ts)
   • Imports and registers all tools
   • Chooses transport (stdio/http) from environment variable
   • Handles startup and error handling
2. Registered Tools (in separate modules)
   • tools/package.ts: ckan_package_search, ckan_package_show
   • tools/organization.ts: ckan_organization_list, ckan_organization_show, ckan_organization_search
   • tools/datastore.ts: ckan_datastore_search
   • tools/status.ts: ckan_status_show
3. MCP Resource Templates (resources/)
   • ckan://{server}/dataset/{id} - Dataset metadata
   • ckan://{server}/resource/{id} - Resource metadata
   • ckan://{server}/organization/{name} - Organization metadata
4. Utility Functions (utils/)
   • http.ts: makeCkanRequest<T>() - HTTP client for CKAN API v3
   • formatting.ts: truncateText(), formatDate(), formatBytes()
   • scheming.ts: ckan_package_show reads the portal's scheming_dataset_schema_show for the dataset's own type and adds the schema's other root fields as fields (label, value as stored). Any failure means "no schema" and is remembered (an hour for 404/400, five minutes otherwise). Every string goes through cleanText and, in Markdown, sanitizeInline/wrapUntrusted
5. Type Definitions (types.ts)
   • ResponseFormat enum (MARKDOWN, JSON)
   • ResponseFormatSchema Zod validator
   • CHARACTER_LIMIT constant
6. Transport Layer (transport/)
   • stdio.ts: Standard input/output (Claude Desktop)
   • http.ts: HTTP server (remote access)
7. Validation Schema
   • Uses Zod to validate all tool inputs
   • Each tool has a strict schema that rejects extra parameters
8. Output Formatting
   • All tools support two formats: markdown (default) and json
   • Markdown format optimized for human readability
   • JSON format returns compact objects with only essential fields (~70% token reduction vs raw CKAN API)
   • JSON truncation is safe: shrinks known arrays, then degrades to a small {_truncated, _error} object — never a string cut mid-value (always valid JSON)
   • Errors respect response_format: JSON callers get {error, _error: true}, not prose (see formatError)
   • structuredContent carries the same capped payload as the text (jsonToolResult), so the character limit is not bypassable via the structured channel
   • See docs/JSON-OUTPUT.md for complete field schemas

Transport Modes

The server supports three transport modes:

• stdio (default): for integration with Claude Desktop and other local MCP clients
• http: exposes POST /mcp endpoint on configurable port (default 3000)
• Cloudflare Workers (v0.4.0+): global edge deployment via src/worker.ts

Cloudflare Workers Deployment

Added in v0.4.0. The server can be deployed to Cloudflare Workers for global HTTP access.

Key files:

• src/worker.ts - Workers entry point using Web Standards transport
• wrangler.toml - Cloudflare configuration

Deployment workflow:

1. npm run dev:worker - Test locally (http://localhost:8787)
2. npm run deploy - Deploy to Cloudflare
3. Access at: https://ckan-mcp-server.<account>.workers.dev

Architecture:

• Uses WebStandardStreamableHTTPServerTransport from MCP SDK
• Compatible with Workers runtime (no Node.js APIs)
• Stateless mode (no session management)
• All tools and resource templates work identically to the Node.js version (/health reports the current counts: 20 tools, 7 resources, 6 prompts as of v0.4.120)

See docs/DEPLOYMENT.md for complete deployment guide.

CKAN API Integration

The server uses CKAN API v3 available on any CKAN portal. All requests:

• Use axios with 30 second timeout
• Send User-Agent CKAN-MCP-Server/1.0
• Handle HTTP errors, timeouts, and server not found
• Normalize server URL (removing trailing slash)
• Validate that response.success === true

Solr Queries

CKAN uses Apache Solr for search. The ckan_package_search tool supports:

• q (query): complete Solr syntax (field:value, AND/OR/NOT, wildcard, range)
• fq (filter query): additional filters without affecting score
• facet_field: aggregations for statistical analysis
• sort: result ordering
• start/rows: pagination

Common query examples are documented in EXAMPLES.md.

TypeScript Configuration

The project uses ES2022 as target and module system.

Note: tsconfig.json is present mainly for editor support (IDE, LSP). The actual compilation uses esbuild which ignores most TypeScript options to maximize speed.

TypeScript configuration (for IDE):

• Output in dist/ directory
• Strict mode enabled
• Strict type checking with noUnusedLocals, noUnusedParameters, noImplicitReturns
• Skip lib check to reduce overhead
• Declaration and source map disabled

Dependencies

Runtime:

• @modelcontextprotocol/sdk - Official MCP SDK
• axios - HTTP client for CKAN API
• zod - Schema validation
• express - HTTP server (only for http mode)

Dev:

• esbuild - Build tool (bundler and compiler)
• typescript - Only for type checking and editor support
• @types/node, @types/express - Type definitions

Supported CKAN Portals

The server can connect to any CKAN instance. Some main portals:

• 🇮🇹 https://dati.gov.it (Italy)
• 🇨🇦 https://open.canada.ca/data (Canada)
• 🇬🇧 https://data.gov.uk (United Kingdom)
• 🌍 https://demo.ckan.org (Official CKAN Demo)

Each portal may have different configurations for:

• DataStore availability
• Custom dataset fields
• Available organizations and tags
• Supported resource formats

The project uses Vitest for automated testing:

# Run all tests
npm test

# Run tests in watch mode
npm run test:watch

# Run tests with coverage report
npm run test:coverage

Current test coverage: ~39% overall (utility modules: 98%, tool handlers: 15-20%).

Test suite includes:

• Unit tests for utility functions (formatting, HTTP, URI parsing, URL generation, search)
• Integration tests for MCP tools with mocked CKAN API responses
• Mock fixtures for CKAN API success and error scenarios

Coverage is strong for utility modules but needs improvement for tool handlers. See tests/README.md for detailed testing guidelines and fixture structure.

Manual Testing

For manual testing, use HTTP transport with curl:

# Terminal 1 — start server
npm run build
CKAN_ALLOWED_DOMAINS="www.dati.gov.it" TRANSPORT=http PORT=3001 node dist/index.js

# Terminal 2 — call a tool
curl -s -X POST http://localhost:3001/mcp \
  -H "Content-Type: application/json" \
  -H "Accept: application/json, text/event-stream" \
  -d '{
    "jsonrpc":"2.0",
    "method":"tools/call",
    "params":{
      "name":"ckan_package_search",
      "arguments":{
        "server_url":"https://www.dati.gov.it/opendata",
        "q":"ambiente",
        "page":1,
        "page_size":3
      }
    },
    "id":1
  }'

Key points:

• Always include both Content-Type: application/json and Accept: application/json, text/event-stream headers
• Use node dist/index.js directly (not npm start) to avoid npm buffering
• Use port 3001 to avoid conflicts with other services on 3000
• Check output for correct field values, pagination hints, and Showing: N results (from X) line

To test with Claude Desktop, add MCP configuration to config file.

Development Notes

Version History

v0.4.0 (2026-01-10): Cloudflare Workers deployment

• Added src/worker.ts for Workers runtime
• Global edge deployment support
• Web Standards transport integration
• See docs/DEPLOYMENT.md for deployment guide

v0.3.0 (2026-01-08): MCP Resource Templates

• Added ckan:// URI scheme support
• Direct data access for datasets, resources, organizations

v0.2.0 (2026-01-08): Comprehensive test suite

• 191 tests (unit + integration)
• ~39% code coverage (utils well-tested, tools improving)

v0.1.0 (2026-01-08): Modular refactoring

• Restructured from monolithic file to 11 modules
• Improved maintainability and testability

Known Limitations

• Output limit: 50,000 characters hardcoded in types.ts (could be configurable)
• Date formatting: Uses fixed ISO YYYY-MM-DD in utils/formatting.ts (could be configurable)
• Read-only: All tools are read-only (no data modification on CKAN)
• Caching: Read-through cache in makeCkanRequest. Action-based TTL (metadata 300s, datastore 60s, status 3600s). Backend: Cloudflare Cache API on Workers, in-memory LRU on Node. Disable with CKAN_CACHE_ENABLED=false. Env vars: CKAN_CACHE_TTL_DEFAULT, CKAN_CACHE_MAX_ENTRIES, CKAN_CACHE_MAX_ENTRY_BYTES.
• No authentication: Uses only public CKAN endpoints
• No WebSocket: MCP over HTTP uses JSON responses (not SSE streaming in Workers)
• SSRF protection (v0.4.108+): requests are validated against private/internal IP ranges, including hostnames that resolve to internal addresses (DNS-based SSRF) — validateServerUrl() (string/literal guard) + isBlockedIp() + a connection-pinning lookup agent on the Node/axios path (getSafeAgents()) and, since v0.4.118, an equivalent undici dispatcher on the fetch/SPARQL path (getSafeDispatcher(), used by safeFetch) so both paths connect to exactly the address they validated — no DNS-rebinding window. assertHostnameResolvesSafe() remains as defence in depth where no dispatcher is available (Workers).
• Domain allowlist: CKAN_ALLOWED_DOMAINS=domain1.com,domain2.org (comma-separated, default-deny when set). Mandatory for the HTTP transport: TRANSPORT=http refuses to start without it, unless CKAN_HTTP_ALLOW_ALL=true is set (logs a warning). stdio is unaffected (no allowlist required). Enforced in validateServerUrl() + assertHttpAllowlistConfigured().
• Audit logging: Every makeCkanRequest call writes a JSON line to stderr (Node modes only; Workers use console.log). Fields: ts, server, action, cache_hit, plus relevant query params (q, fq, sql truncated to 200 chars, id, rows, limit).

Adding New Tools

1. Create new file in src/tools/
2. Export registerXxxTools(server: McpServer) function
3. Add to registerAll() in src/server.ts
4. Add tests in tests/integration/
5. Build and test: npm run build && npm test

README Authoring Rule

The project has two README files:

• README.md — full README, shown on GitHub
• .readme-npm.md — short README published to npm (intro + essentials + link to full README)

prepack/postpack hooks in package.json swap them automatically during npm publish. Never run npm publish manually without building first (npm run build).

Always use absolute GitHub URLs in both README files — never relative paths. npm cannot resolve relative paths from the tarball.

• Images/GIFs: https://raw.githubusercontent.com/ondata/ckan-mcp-server/main/...
• Links to files: https://github.com/ondata/ckan-mcp-server/blob/main/...

Release Workflow

When releasing a new version:

1. Update version: Edit the version field in package.json, package-lock.json, manifest.json, server.json (in server.json there are two fields: top-level version and packages[0].version — both must match), src/server.ts (MCP server version) and src/worker.ts (/health response). The two source files are easy to forget: v0.4.119 shipped with them still at 0.4.118. Check with grep -rn "<old version>" package.json manifest.json server.json src/server.ts src/worker.ts — it must return nothing
2. Run the release gate: npm run smoke — known-answer search queries against live portals, asserting which dataset comes back rather than how many. It must be green before tagging. It exists because v0.4.122 shipped with counts verified and ranking broken: "22 results" and "the right dataset first" are different claims
3. Update LOG.md: Add entry with date and changes
4. Commit changes on a branch: git checkout -b <type>/<description> then git add . && git commit -m "...". Code never goes straight to main; documentation-only changes may.
5. Open a PR and merge it: git push -u origin <branch>, gh pr create, wait for green checks, gh pr merge --squash --delete-branch, then git checkout main && git pull. main has a non_fast_forward rule: a commit pushed there by mistake needs a revert, not a force-push.
6. Create tag: git tag -a v0.x.0 -m "..." && git push origin v0.x.0 — ⚠️ this triggers the npm publish, see step 10
7. Build DXT: npm run pack:dxt → produces ckan-mcp-server.dxt
8. Build skill: npm run pack:skill → produces tmp/ckan-mcp.skill
9. Attach to release: gh release upload v0.x.0 ckan-mcp-server.dxt tmp/ckan-mcp.skill
10. npm publish happens automatically: pushing the tag in step 6 starts .github/workflows/release.yml, which verifies the tag matches package.json, builds, tests, and runs npm publish --provenance. Do not run npm publish by hand — the two paths collide and the loser gets EPUBLISHCONFLICT. Watch the run: gh run watch $(gh run list --workflow=release.yml --limit 1 --json databaseId -q '.[0].databaseId')
11. Publish to the MCP Registry — only after the step 10 run has finished green, since the registry validates that the npm version exists: mcp-publisher login github && mcp-publisher publish. If login fails with incorrect_device_code, the local binary is stale: compare mcp-publisher --version against the latest release and update it — the device-auth flow has changed between versions, and the error does not say so
12. Deploy to Cloudflare (if code changed): npm run deploy

See docs/DEPLOYMENT.md for detailed Cloudflare deployment instructions.

Why steps 1 and 11 matter: server.json feeds the official MCP Registry entry, which is what clients installing via the registry resolve. It is not updated by npm publish. Skipping it silently pins public installs to an old version: between v0.4.83 (2026-03-12) and v0.4.114 (2026-08-03) the registry advertised a build predating the v0.4.108 SSRF remediation, while npm was current. Verify after publishing:

curl -s "https://registry.modelcontextprotocol.io/v0/servers?search=io.github.aborruso/ckan-mcp-server" | \
  jq -r '.servers[] | select(.server.name|test("aborruso")) | "\(.server.version) | pkg \(.server.packages[0].version) | latest \(._meta["io.modelcontextprotocol.registry/official"].isLatest)"'

npm Provenance

Releases are published from CI with npm provenance (Sigstore), so every tarball carries a signed attestation binding it to the commit and workflow run that built it. Adopters see a "Provenance" badge on the npm page and can verify the package really came from this repository — the supply-chain equivalent of the registry alignment above.

Authentication uses trusted publishing (OIDC), not a token: no NPM_TOKEN secret exists and none should be created. Each publish authenticates with a short-lived, workflow-specific credential that cannot be exfiltrated or reused, which also sidesteps npm's ongoing restriction of 2FA-bypassing tokens for direct publishing.

Requirements:

• Trusted publisher configured on npmjs.com (one-off, manual): package page → Settings → Trusted Publisher → GitHub Actions → organization ondata, repository ckan-mcp-server, workflow release.yml, no environment. The workflow is inert until this is done.
• id-token: write permission in release.yml (set)
• a repository field in package.json matching this repo (set)
• npm ≥ 11.5.1 in CI — the workflow upgrades npm explicitly, because Node 22 ships npm 10.x and older versions fall back to token auth silently, publishing without provenance

⚠️ The workflow filename is part of the trusted-publisher identity. Renaming release.yml breaks publishing with a non-obvious error; rename it on npmjs.com first.

Verify after a release:

npm view @aborruso/ckan-mcp-server@<version> dist.attestations

CSV Data Exploration

For exploring CSV resources from datasets, use duckdb CLI (already installed) with direct HTTP URL:

duckdb -jsonlines -c "DESCRIBE SELECT * FROM read_csv('http://url/file.csv')"
duckdb -jsonlines -c "SUMMARIZE SELECT * FROM read_csv('http://url/file.csv')"
duckdb -jsonlines -c "SELECT * FROM read_csv('http://url/file.csv') USING SAMPLE 10"

Use direct resource URLs (http/https), not GitHub view/blob URLs. The -jsonlines parameter outputs in JSONL format, easier for AI to parse.

For random sampling, use USING SAMPLE N syntax (where N is the number of rows):

duckdb -jsonlines -c "SELECT * FROM read_csv('http://url/file.csv') USING SAMPLE 10"