arcgis-parquet
Skill1 starsChanged 6 months agoMITView on GitHub
Load Parquet and GeoParquet files on a map with ParquetLayer in the ArcGIS Maps SDK for JavaScript 5.1 (beta): data with ParquetFilesData (several files) or ParquetPortalItemData, geometryEncoding for GeoParquet, WKB or x/y columns, and the spatialReference and geometryType a file without GeoParquet metadata needs (without them nothing draws and nothing errors). Covers the silent 8000-row query cap, the host requirements (HEAD, Range, CORS) and what the layer cannot do (edit, 3D). Use this skill whenever the user loads a .parquet or GeoParquet file.
In this repository73 other files it gives its agents.
- skills/arcgis-3d-layers/AGENTS.mdAGENTS.md
- skills/arcgis-advanced-layers/AGENTS.mdAGENTS.md
- skills/arcgis-ai-components/AGENTS.mdAGENTS.md
- skills/arcgis-arcade/AGENTS.mdAGENTS.md
- skills/arcgis-authentication/AGENTS.mdAGENTS.md
- skills/arcgis-charts/AGENTS.mdAGENTS.md
- skills/arcgis-cim-symbols/AGENTS.mdAGENTS.md
- skills/arcgis-coding-components/AGENTS.mdAGENTS.md
- skills/arcgis-coordinates-projection/AGENTS.mdAGENTS.md
- skills/arcgis-core-maps/AGENTS.mdAGENTS.md
- skills/arcgis-core-utilities/AGENTS.mdAGENTS.md
- skills/arcgis-custom-rendering/AGENTS.mdAGENTS.md
- skills/arcgis-editing/AGENTS.mdAGENTS.md
- skills/arcgis-embeddable-maps/AGENTS.mdAGENTS.md
- skills/arcgis-feature-effects/AGENTS.mdAGENTS.md
- skills/arcgis-geometry-operations/AGENTS.mdAGENTS.md
- skills/arcgis-imagery/AGENTS.mdAGENTS.md
- skills/arcgis-interaction/AGENTS.mdAGENTS.md
- skills/arcgis-knowledge-graphs/AGENTS.mdAGENTS.md
- skills/arcgis-layers/AGENTS.mdAGENTS.md
- skills/arcgis-map-tools/AGENTS.mdAGENTS.md
- skills/arcgis-parquet/AGENTS.mdAGENTS.md
- skills/arcgis-performance/AGENTS.mdAGENTS.md
- skills/arcgis-popup-templates/AGENTS.mdAGENTS.md
- skills/arcgis-portal-content/AGENTS.mdAGENTS.md
- skills/arcgis-rest-services/AGENTS.mdAGENTS.md
- skills/arcgis-scene-environment/AGENTS.mdAGENTS.md
- skills/arcgis-smart-mapping/AGENTS.mdAGENTS.md
- skills/arcgis-spatial-analysis/AGENTS.mdAGENTS.md
- skills/arcgis-starter-app/AGENTS.mdAGENTS.md
- skills/arcgis-tables-forms/AGENTS.mdAGENTS.md
- skills/arcgis-time-animation/AGENTS.mdAGENTS.md
- skills/arcgis-utility-networks/AGENTS.mdAGENTS.md
- skills/arcgis-visualization/AGENTS.mdAGENTS.md
- skills/arcgis-widgets-advanced/AGENTS.mdAGENTS.md
- skills/arcgis-widgets-ui/AGENTS.mdAGENTS.md
- skills/hello-world/AGENTS.mdAGENTS.md
- arcgis-3d-layersSkill
- arcgis-advanced-layersSkill
- arcgis-ai-componentsSkill
- arcgis-arcadeSkill
- arcgis-authenticationSkill
- arcgis-chartsSkill
- arcgis-cim-symbolsSkill
- arcgis-coding-componentsSkill
- arcgis-coordinates-projectionSkill
- arcgis-core-mapsSkill
- arcgis-core-utilitiesSkill
- arcgis-custom-renderingSkill
- arcgis-editingSkill
- arcgis-embeddable-mapsSkill
- arcgis-feature-effectsSkill
- arcgis-geometry-operationsSkill
- arcgis-imagerySkill
- arcgis-interactionSkill
- arcgis-knowledge-graphsSkill
- arcgis-layersSkill
- arcgis-map-toolsSkill
- arcgis-performanceSkill
- arcgis-popup-templatesSkill
- The first 60 shown.
The file
We split this skill into the rules it gives an agent
2 rules. About 0 are common, 0 are distinctive, and 2 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈0
Distinctive · estimate
This author's own choices. Worth a look.
2
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Very specific: line 81
No geometry at all → it loads as a table (isTable: true).
- Very specific: line 144
Only WGS84 and Web Mercator are supported.
≈ “Common” is an estimate: at least 1 in 5 of the 364,980 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Parquet Layers
When to use: Activate this skill whenever the user puts a
.parquetor GeoParquet file on a map, queries one, or asks why a Parquet layer loads but shows nothing.
Overview
ParquetLayer reads Apache Parquet files in the browser (with a WebAssembly reader) and draws their
rows as features in a 2D view. It is beta since 4.33; 4.34 added geometryEncoding, isTable,
queryFeatures and queryObjectIds, and 5.1 replaced urls with data and added portal items,
definitionExpression and createQuery().
- Problem it solves: showing hundreds of thousands of rows from a columnar file without a feature service.
- Who it serves: developers with columnar data exported from DuckDB, GeoPandas, BigQuery or ArcGIS.
- Pattern it teaches: pick the encoding from what the file carries, declare what it does not, and check the host.
- What it does NOT cover: querying in general (see
arcgis-layers) or renderers (seearcgis-visualization).
Quick Start
import ParquetLayer from "@arcgis/core/layers/ParquetLayer.js";
import ParquetFilesData from "@arcgis/core/layers/support/ParquetFilesData.js";
// GeoParquet (with "geo" metadata) or columns named lon/lat, lng/lat, x/y…: detected on load.
const layer = new ParquetLayer({
data: new ParquetFilesData({
urls: ["https://jsapi.maps.arcgis.com/sharing/rest/content/items/6ab6f959c1684eb5a54db91644a89747/data"],
}),
title: "Kansas water wells",
});
map.add(layer);
await layer.load();
console.log(layer.isTable, layer.geometryType, layer.geometryEncoding?.type); // false "point" "location"Measured with 5.1.27 on Esri's sample file (310 702 rows, 56 MB): it loads without sign-in, detects its
lng/lat columns and draws all 310 702 points in a MapView.
Decision Tree
- What does the file carry?
- GeoParquet
geometadata → nothing to declare: encoding,geometryType, SR andfullExtentare read. - Coordinate columns named
lon/lat,lng/lat,long/lat,longitude/latitude,x/y,xcenter/ycenter,point-x/point-y… (any case) → detected. - Coordinate columns with other names (
coord_x,este…) →GeometryEncodingLocationplusgeometryType: "point"plusspatialReference. - A WKB column without
geometadata →GeometryEncodingWkb({ field })plusgeometryTypeplusspatialReference. - No geometry at all → it loads as a table (
isTable: true); use it inmap.tablesor a feature table.
- GeoParquet
- Where is the file?
- URLs you control →
ParquetFilesData({ urls }); several files with the same schema add up. - An ArcGIS Online item of type Apache Parquet →
ParquetPortalItemData({ portalItem: { id } }). - A file the user picks →
URL.createObjectURL(file)as the URL (measured: it works).
- URLs you control →
- Which view?
MapView/<arcgis-map>only. In aSceneViewthe layer view is rejected.
Default: ParquetFilesData with GeoParquet in EPSG:4326, Snappy or GZIP.
Import Patterns
Direct ESM (recommended for npm/Vite/Webpack)
import ParquetLayer from "@arcgis/core/layers/ParquetLayer.js";
import ParquetFilesData from "@arcgis/core/layers/support/ParquetFilesData.js";
import ParquetPortalItemData from "@arcgis/core/layers/support/ParquetPortalItemData.js";
import GeometryEncodingWkb from "@arcgis/core/layers/support/GeometryEncodingWkb.js";
import GeometryEncodingLocation from "@arcgis/core/layers/support/GeometryEncodingLocation.js";
import { getParquetLayerInfo } from "@arcgis/core/layers/support/parquetUtils.js";The class the docs call ParquetGeometryEncodingWkb does not exist as a module: it is GeometryEncodingWkb.
Dynamic Imports (CDN)
const [ParquetLayer, ParquetFilesData] = await $arcgis.import([
"@arcgis/core/layers/ParquetLayer.js",
"@arcgis/core/layers/support/ParquetFilesData.js",
]);Map Components (recommended for new projects)
// The layer is Core API; add it to the component's map. 2D only: <arcgis-map>, not <arcgis-scene>.
import "@arcgis/map-components/components/arcgis-map";Key Concepts
Concept 1: data replaced urls in 5.1
data takes a ParquetFilesData (one or more URLs) or a ParquetPortalItemData. The old urls still works but
is @deprecated since 5.1 and logs a warning. Several files are read as one layer; the fields come from the
first file, so they must share a schema (see pitfalls). getParquetLayerInfo(data) returns the fields,
encoding, geometry type and SR it infers from the first file, ready to pass to new ParquetLayer(info).
Concept 2: geometryEncoding and the two properties that go with it
With GeoParquet metadata everything is detected. Without it, geometryEncoding alone is not enough:
| Declared | Measured result (5.1.27) |
| nothing (WKB or custom x/y) | loads as a table, isTable: true, no error |
| encoding only, polygons | geometryType stays "point", geometry null |
encoding + geometryType, no spatialReference | geometry null, the view draws nothing, no error |
encoding + geometryType + spatialReference | correct |
layer.spatialReference reads 4326 by default even when it was not set, so checking it proves nothing: set it.
Only WGS84 and Web Mercator are supported; a projected file (EPSG:9377, UTM) is rejected on load().
Concept 3: Queries run in the client, with a cap
queryFeatures, queryFeatureCount, queryObjectIds, queryExtent, orderByFields and outStatistics work
(measured). Two behaviors differ from a feature service: a query returns at most 8000 rows
(capabilities.query.maxRecordCount) with exceededTransferLimit: false, and attribute values in where are
case-sensitive ('constructed' gave 0 of 244 149; UPPER(STATUS) = 'CONSTRUCTED' works).
Concept 4: The host must answer HEAD and Range, with CORS
The reader asks HEAD for the size and then reads 64 KB pieces with Range (206). A host without CORS, one
that ignores Range (answers 200 with the whole file), one that refuses HEAD, and a 404 all reject load()
with the same message: Failed to parse file. An ArcGIS Online item's /data URL and Vite's dev server qualify
(measured); check any other host before blaming the file. Compression must be none, Snappy or GZIP (ZSTD is rejected).
Concept 5: What the layer does not do
- No editing: no
applyEdits,capabilities.operations.supportsEditingisfalse. - No 3D:
SceneView.whenLayerView()rejects with "layerview creation failed". - Beta:
ParquetLayer,ParquetFilesData,ParquetPortalItemDataandGeometryEncodingWkbare@beta; pin the SDK version and re-test on upgrades.
Common Pitfalls (Top 3)
Pitfall 1: Declaring the encoding but not the spatial reference
// ❌ Anti-pattern
const layer = new ParquetLayer({
data: new ParquetFilesData({ urls: ["https://example.com/stations.parquet"] }),
geometryEncoding: new GeometryEncodingLocation({ xField: "coord_x", yField: "coord_y" }),
geometryType: "point", // no spatialReference: "it defaults to WGS84"
});// ✅ Correct
const layer = new ParquetLayer({
data: new ParquetFilesData({ urls: ["https://example.com/stations.parquet"] }),
geometryEncoding: new GeometryEncodingLocation({ xField: "coord_x", yField: "coord_y" }), // x = longitude
geometryType: "point",
spatialReference: { wkid: 4326 },
});Impact: measured: load() resolves, the layer view is created, every geometry is null and nothing is
drawn; no error anywhere. The same happens with a WKB column. Setting spatialReference fixes both.
Pitfall 2: Trusting exceededTransferLimit
// ❌ Anti-pattern
const { features, exceededTransferLimit } = await layer.queryFeatures({ where: "1=1", outFields: ["DEPTH"] });
if (!exceededTransferLimit) exportAll(features); // 8000 of 310 702 rows// ✅ Correct
const total = await layer.queryFeatureCount();
const size = layer.capabilities?.query?.maxRecordCount ?? 8000;
for (let start = 0; start < total; start += size) {
const q = layer.createQuery();
q.start = start;
q.num = size;
const { features } = await layer.queryFeatures(q);
exportAll(features);
}Impact: measured on 310 702 rows: 8000 features and exceededTransferLimit: false, also with num: 20000.
Paging works (start: 310000 returned the last 702). For totals, prefer outStatistics over downloading rows.
Pitfall 3: Debugging the file when the host is the problem
// ❌ Anti-pattern
await layer.load(); // "Failed to parse file" → re-export the parquet, try another compression…// ✅ Correct
const url = "https://example.com/data.parquet";
const head = await fetch(url, { method: "HEAD" });
const piece = await fetch(url, { headers: { Range: "bytes=0-3" } });
console.log(head.ok, head.headers.get("content-length"), piece.status); // true, size, 206Impact: measured: 404, no CORS, no Range support and no HEAD all give Failed to parse file. A real format
problem says so (Compression codec zstd is unsupported…, Only WGS84 and Web Mercator are supported).
When to Read references/
| If you need... | Read |
| ParquetLayer, data and encoding signatures | references/api-detail.md |
| GeoParquet, WKB, x/y, portal item, local file, paging | references/examples.md |
| More pitfalls (12, measured) and what was not measured | references/pitfalls.md |
Related Skills
arcgis-layers— queries,createQueryand paging on any feature layerarcgis-visualization— renderers for the layerarcgis-tables-forms— showing a non-spatial Parquet (isTable) in a tablearcgis-core-maps— theMapViewthe layer needs
Reference Samples
Changelog
- 0.1.0 (2026-10-28): Initial release, SDK 5.1.27. Every claim executed in
tests/runtime/casos/arcgis-parquet.ts(32 checks) against Esri's public sample file and generated GeoParquet/WKB/x-y files: the missingspatialReferencethat draws nothing, the 8000-row cap withoutexceededTransferLimit, case-sensitive values, mixed schemas, µs timestamps and the host requirements.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

