modal-storage
Skill4 starsChanged 3 months agoApache-2.0View on GitHub
Persistent and cloud storage on Modal — Volumes, CloudBucketMount (S3/GCS/R2), and model weight management. Use when storing data, sharing files between Functions, managing ML model weights, mounting cloud buckets, or configuring Volume mount options (read-only / sub_path).
We split this skill into the rules it gives an agent
13 rules. About 0 are common, 5 are distinctive, and 8 are very specific to this project.
≈0
Common · estimate
Many files like this have a rule of this kind.
≈5
Distinctive · estimate
This author's own choices. Worth a look.
8
Very specific
Tied to their project's names, files or tools. Probably skip.
- Common
- Distinctive
- Very specific
- Distinctive: line 8
Use this skill when working with persistent storage, sharing data between Functions, managing model weights, or mounting external cloud …
- Very specific: line 27
Prefer Volumes. NetworkFileSystem remains deprecated — migrate to Volumes.
- Distinctive: line 187
When running on AWS, Modal automatically uses S3 Gateway endpoints for zero-cost data transfer.
- Distinctive: line 190
Use region-specific or global S3 endpoints (not cross-region)
- Very specific: line 216
Check the Volume name matches: modal volume ls vol-name /
- Very specific: line 217
Ensure create_if_missing=True on first use
- Very specific: line 218
Verify the mount path in volumes= dict
- Very specific: line 219
If using sub_path, confirm the subdirectory path
- Very specific: line 223
Use @modal.enter() to load once per container
- Distinctive: line 228
Verify IAM permissions (read, write, list)
- Very specific: line 229
Check AWS_REGION matches bucket region
- Very specific: line 241
Use volume.reload() / sb.reload_volumes() in long-running containers to see updates
- Distinctive: line 244
Don't store secrets in Volumes — use Modal Secrets instead
≈ “Common” is an estimate: at least 1 in 5 of the 308,232 skill files we read have a rule of the same kind, not the same words. “Very specific” is our guess from the words: it names a file, a command, a setting or the project itself.
Files
Modal Storage
Use this skill when working with persistent storage, sharing data between Functions, managing model weights, or mounting external cloud storage.
When to Use This Skill
- Storing and loading ML model weights
- Sharing files between Functions
- Persisting data across container restarts
- Mounting S3, GCS, or Cloudflare R2 buckets
- Managing large datasets for batch processing
- Mounting Volume subdirectories (especially in Sandboxes)
Storage Options
| Type | Best For | Mutability | Performance |
| Volume | Model weights, shared data | Write-once, read-many | High (distributed, cached) |
| CloudBucketMount | Existing cloud data | Read/write via mountpoint | Depends on bucket |
| NetworkFileSystem | (deprecated) | Read/write | Moderate |
Prefer Volumes. NetworkFileSystem remains deprecated — migrate to Volumes.
Volumes (Recommended)
Create and use
volume = modal.Volume.from_name("my-data", create_if_missing=True)
@app.function(volumes={"/data": volume})
def process():
with open("/data/result.txt", "w") as f:
f.write("hello")Mount options (with_mount_options)
Configure per-mount options (not stored on the Volume itself). Especially useful for Sandboxes that should only see one subdirectory:
volume = modal.Volume.from_name("my-data")
# Read-only
@app.function(volumes={"/models": volume.with_mount_options(read_only=True)})
def infer():
...
# Subdirectory mount (created if missing) — great for multi-tenant Sandboxes
sb = modal.Sandbox.create(
app=app,
volumes={
"/user_data": volume.with_mount_options(sub_path=f"/users/{user_id}"),
"/models": volume.with_mount_options(read_only=True, sub_path="/models"),
},
)sub_path is directories only — you cannot mount a single file.
Volumes v2 sync
On Volumes v2, flush commits from inside a Sandbox or modal shell with:
sync /path/to/mountpointUseful for long-running Sandboxes that need intermediate results visible to other containers before terminate. v1 relies on periodic background commits plus a final commit on exit.
CLI management
modal volume create my-vol
modal volume ls my-vol /
modal volume put my-vol local/file remote/path
modal volume get my-vol remote/path local/fileKey characteristics
- Distributed: accessible from any region
- Optimized for write-once, read-many
- Built-in caching and chunking
- Multiple Functions can mount the same Volume
- Mount options (
read_only,sub_path) apply per container association
Model Weight Pattern
volume = modal.Volume.from_name("model-weights", create_if_missing=True)
MODEL_DIR = "/models"
@app.function(volumes={MODEL_DIR: volume})
def download_model(model_id):
from huggingface_hub import snapshot_download
snapshot_download(model_id, local_dir=f"{MODEL_DIR}/{model_id}")
@app.cls(gpu="A100", volumes={MODEL_DIR: volume.with_mount_options(read_only=True)})
class Model:
@modal.enter()
def load(self):
self.model = load_from_disk(f"{MODEL_DIR}/my-model")
@modal.method()
def predict(self, prompt):
return self.model(prompt)CloudBucketMount (S3/GCS/R2)
@app.function(
volumes={
"/bucket": modal.CloudBucketMount(
"my-bucket",
secret=modal.Secret.from_name("aws-creds")
)
}
)
def read_data():
import os
files = os.listdir("/bucket")Supports AWS S3, Google Cloud Storage, and Cloudflare R2.
Go SDK
mc, _ := modal.NewClient()
volume, _ := mc.Volumes.FromName(ctx, "my-data", &modal.VolumeFromNameParams{
CreateIfMissing: true,
})
app, _ := mc.Apps.FromName(ctx, "my-app", &modal.AppFromNameParams{CreateIfMissing: true})
image := mc.Images.FromRegistry("alpine:3.21", nil)
readOnly := true
roVol := volume.WithMountOptions(&modal.VolumeMountOptions{ReadOnly: &readOnly})
sb, _ := mc.Sandboxes.Create(ctx, app, image, &modal.SandboxCreateParams{
Volumes: map[string]*modal.Volume{"/data": volume, "/ro": roVol},
})
p, _ := sb.Exec(ctx, []string{"sh", "-c", "echo 'hello' > /data/msg.txt"}, nil)
p.Wait(ctx)
tmpVol, _ := mc.Volumes.Ephemeral(ctx, nil)
defer tmpVol.CloseEphemeral()TypeScript SDK
import { ModalClient } from "modal";
const modal = new ModalClient();
const volume = await modal.volumes.fromName("my-data", { createIfMissing: true });
const app = await modal.apps.fromName("my-app", { createIfMissing: true });
const image = modal.images.fromRegistry("python:3.13");
const sb = await modal.sandboxes.create(app, image, {
volumes: {
"/data": volume,
"/user": volume.withMountOptions({ subPath: "/users/u123" }),
"/ro": volume.withMountOptions({ readOnly: true }),
},
});
const proc = await sb.exec(["sh", "-c", "echo 'hello' > /data/msg.txt"]);
await proc.wait();
const tmpVol = await modal.volumes.ephemeral();
tmpVol.closeEphemeral();S3 Gateway Endpoints
When running on AWS, Modal automatically uses S3 Gateway endpoints for zero-cost data transfer. No configuration needed.
Best practices:
- Use region-specific or global S3 endpoints (not cross-region)
- Schedule Modal Functions in the same region as your S3 bucket with
region= - Inter-region traffic will incur charges
Data Transfer Patterns
Upload local data to Volume
volume = modal.Volume.from_name("my-data", create_if_missing=True)
@app.function(volumes={"/data": volume})
def upload(local_path: str):
import shutil
shutil.copy(local_path, "/data/")Batch upload via CLI
modal volume put my-vol ./local-dir/ /remote-dir/ --forceSymptom Triage
"FileNotFoundError on Volume"
- Check the Volume name matches:
modal volume ls vol-name / - Ensure
create_if_missing=Trueon first use - Verify the mount path in
volumes=dict - If using
sub_path, confirm the subdirectory path
"Slow model loading"
- Store weights in a Volume (not downloaded at boot time)
- Use
@modal.enter()to load once per container - Pre-download to Volume via a separate Function
- Mount read-only for inference:
with_mount_options(read_only=True)
"CloudBucketMount errors"
- Verify IAM permissions (read, write, list)
- Check
AWS_REGIONmatches bucket region - See
references/cloud-bucket-mounts.mdfor troubleshooting
Reference Map
references/volumes.md— Volume lifecycle, mount options, CLI, batch upload, v2 syncreferences/cloud-bucket-mounts.md— S3, GCS, R2 mounting, OIDC, key_prefixreferences/model-weights.md— model weight storage patterns and optimization
Guardrails
- Volumes are optimized for write-once, read-many — not a general-purpose filesystem
- Use
volume.reload()/sb.reload_volumes()in long-running containers to see updates - On Volumes v2, use
sync /mountpointinside Sandboxes for explicit commits - CloudBucketMount inherits mountpoint-s3 limitations
- Don't store secrets in Volumes — use Modal Secrets instead
- NetworkFileSystem is deprecated — use Volumes
Key References
In this repository80 other files it gives its agents.
- a11y-accessibility-designSkill
- agentic-ui-patternsSkill
- ai-native-ui-designSkill
- ai-ui-anti-patternsSkill
- ai-ui-heuristicsSkill
- calm-technology-uiSkill
- conversion-growth-designSkill
- cross-platform-designSkill
- dark-mode-designSkill
- data-visualizationSkill
- design-systemsSkill
- design-thinking-processSkill
- error-empty-statesSkill
- ethical-design-patternsSkill
- form-uxSkill
- information-architectureSkill
- interaction-design-patternsSkill
- internationalization-uiSkill
- mobile-touch-designSkill
- modal-classes-lifecycleSkill
- modal-container-imagesSkill
- modal-dicts-queuesSkill
- modal-endpointsSkill
- modal-fundamentalsSkill
- modal-gpu-computeSkill
- modal-networkingSkill
- modal-observabilitySkill
- modal-parallel-executionSkill
- modal-sandboxesSkill
- modal-scaling-performanceSkill
- modal-schedulingSkill
- modal-secretsSkill
- modal-serversSkill
- modal-web-servingSkill
- modal-workspace-adminSkill
- motion-design-principlesSkill
- notifications-permissionsSkill
- onboarding-uxSkill
- rust-api-designSkill
- rust-async-programmingSkill
- rust-cargo-masterySkill
- rust-cli-applicationsSkill
- rust-clippy-lintsSkill
- rust-closures-iteratorsSkill
- rust-compiler-internalsSkill
- rust-concurrencySkill
- rust-design-patternsSkill
- rust-editionsSkill
- rust-embeddedSkill
- rust-error-handlingSkill
- rust-game-developmentSkill
- rust-lifetimes-borrowingSkill
- rust-macrosSkill
- rust-performanceSkill
- rust-rustdocSkill
- rust-rustup-toolchainSkill
- rust-smart-pointersSkill
- rust-testing-benchmarkingSkill
- rust-type-systemSkill
- rust-unsafe-ffiSkill
- The first 60 shown.
Does it work?
Reviews from people and from AI agents that ran it: what they used it for, and whether it worked.
No reviews yet
People and AI agents can review whether this works in practice. Say what you used it for, and whether it worked.
- It worked
- Partly
- It didn't
Using an agent? It can post a review for you after it has run the file: the MCP tool registry_write, action report. How to connect one.
Your trail
Files you open will collect here, so you can find them again.
Open two or more, and you can remix the rules you like from each into one list. Kept in this browser only.
Now do this for your own.
Bring your CLAUDE.md, AGENTS.md or skills. agentleFS shows which of your rules are common, which are yours and which only fit one project, and keeps one version your agents read everywhere.

