Data & Storage

Three ways to get data onto your cluster and persist results.

Mechanism	YAML field	What it does	Lifecycle
Cloud buckets	file_mounts	Sync data to a path on the cluster	Data lives in your bucket
Persistent volumes	volumes	Mount Mithril network storage	Survives instance termination
Ephemeral volumes	volumes	Fast local scratch space	Deleted with cluster

Your local code is handled separately by workdir — see Syncing your code.

Cloud buckets

Use file_mounts to make data available at a path on the cluster.

file_mounts:
  /data: s3://my-bucket/training-data
  /models: gs://my-bucket/pretrained

Local files

You can also mount local files and directories. They are uploaded to a temporary cloud bucket behind the scenes and synced to the cluster:

file_mounts:
  /remote/path: /local/path/to/data
  /remote/config.yaml: ./config.yaml

Supported sources

Source	Example
AWS S3	s3://my-bucket/path
Google Cloud Storage (GCS)	gs://my-bucket/path
Cloudflare R2	r2://my-bucket
CoreWeave Object Storage	cw://my-bucket
OCI Object Storage	oci://my-bucket@region
Local directory	/absolute/path or ./relative/path
Local file	./config.yaml

For latest list of supported providers → Cloud Buckets (SkyPilot docs)

Storage modes

Cloud buckets support three access modes:

Mode	Reads	Writes	Best for
MOUNT (default)	Streamed from bucket	Replicated to bucket and visible to other VMs	Shared datasets, multi-node access
COPY	Pre-fetched to local disk	Local only, not synced back	Fast I/O on data that fits on disk
MOUNT_CACHED	Cached locally on access	Cached locally, uploaded in background before task completes	Checkpoints and large writes

file_mounts:
  /data:
    source: s3://my-bucket/dataset
    mode: COPY # MOUNT, COPY, or MOUNT_CACHED

→ Cloud Buckets (SkyPilot docs) — advanced storage options, bucket creation, CLI management, and YAML reference

Persistent volumes

Use volumes to mount Mithril network storage that survives instance termination, preemption, and restarts. Ideal for training checkpoints and datasets you reuse across runs.

Create a volume

ml sky volumes apply \
  --name my-data \
  --infra mithril/us-central5-a \
  --type mithril-file-share \
  --size 100GB

Use in task YAML

resources:
  infra: mithril/us-central5-a  # must match volume region

volumes:
  /data: my-data
  /checkpoints: my-checkpoints

Volume interfaces

Interface	--type	Use case
File (NFS)	mithril-file-share	Shared access across multiple instances
Block	mithril-block	Single instance, high throughput

Not all regions support both interfaces. Check the Mithril console for availability.

Manage volumes

ml sky volumes ls               # list volumes
ml sky volumes delete my-volume # delete a volume

Region matching

Volume and cluster must be in the same region:

resources:
  infra: mithril/us-central5-a # ← must match

volumes:
  /data: my-volume # ← created in us-central5-a

Ephemeral storage

Every Mithril instance comes with NVMe SSD ephemeral storage at no extra cost, automatically mounted at /mnt/local. No YAML configuration needed — it's available on every instance by default.

Event	Ephemeral storage
VM restart	Retained
Preemption	Wiped (re-mounted on reallocation)
Termination	Wiped
Host maintenance	Wiped

Use /mnt/local for scratch work, caches, and shuffle buffers. Don't store anything you need to keep — use persistent volumes or object storage (via cloud buckets) for that.

→ Ephemeral Storage — instance storage specs and detailed behavior

Syncing your code

workdir syncs a local directory to ~/sky_workdir/ on the cluster:

workdir: .

Your run commands execute from ~/sky_workdir/, so relative paths work as expected. The workdir is re-synced on every ml launch and ml exec.

Choosing the right mechanism

Scenario	Use
Training data	file_mounts with bucket URL
Checkpoints you need across runs	volumes (persistent)
Scratch space for shuffling/caching	/mnt/local Node-local NVMe (ephemeral)
Your code and configs	workdir
Workload output (final weights, LoRA adapters, logs, eval artifacts)	Object storage

Complete example

name: training-run

resources:
  infra: mithril/us-central5-a
  accelerators: B200:8

workdir: .

file_mounts:
  /datasets: s3://my-bucket/imagenet

volumes:
  /checkpoints: my-checkpoints-volume

run: |
  python train.py \
    --data /datasets \
    --scratch /scratch \
    --output /checkpoints