DataCrunch: mounting shared storage

Mount persistent shared filesystems on DataCrunch instances using Network File System (NFS) protocol.

Overview

DataCrunch shared filesystems (SFS) use NFS protocol for high-performance network-attached storage. Each filesystem is accessible via a datacenter-specific NFS endpoint and identified by a unique pseudopath.

Protocol: NFS Connection: Datacenter-region NFS endpoint Performance: Parallel connections via nconnect option

Prerequisites

Before starting, ensure you have:

Created a volume with provider: datacrunch via the Spheron dashboard (Create Volume page)
Attached the shared filesystem to your DataCrunch instance
Retrieved the following from the Spheron AI dashboard:
- Pseudopath: starts with / (e.g., /grgwrgwrg-20f77f6a)
- Datacenter location slug (e.g., fin-01)
SSH access to your DataCrunch instance

Mounting process

Connect to your instance

SSH into your DataCrunch GPU instance using the connection details from the instance's Details drawer:

ssh ubuntu@<your-instance-ip>

See SSH Connection Setup for detailed connection instructions.

Install NFS client tools

Ensure the NFS client is installed on your instance:

sudo apt-get update
sudo apt-get install -y nfs-common

Create mount directory

Create a directory where you'll mount the shared filesystem. Replace <SFS_NAME> with your preferred directory name:

sudo mkdir -p /mnt/<SFS_NAME>

Common choices include:

/mnt/storage: standard mount location
/mnt/datasets: for ML/AI datasets
/mnt/models: for model checkpoints

Mount the filesystem

Mount the shared filesystem. Replace the placeholders with your values:

<DC>: datacenter location slug (e.g., fin-01)
<PSEUDOPATH>: filesystem pseudopath (e.g., /grgwrgwrg-20f77f6a)
<SFS_NAME>: your mount directory name

sudo mount -t nfs -o nconnect=16 nfs.<DC>.datacrunch.io:<PSEUDOPATH> /mnt/<SFS_NAME>

Example:

sudo mount -t nfs -o nconnect=16 nfs.fin-01.datacrunch.io:/grgwrgwrg-20f77f6a /mnt/storage

Persist the mount in `/etc/fstab` (recommended)

Add the filesystem to /etc/fstab so it remounts automatically on reboot:

grep -qxF 'nfs.<DC>.datacrunch.io:<PSEUDOPATH> /mnt/<SFS_NAME> nfs defaults,nconnect=16 0 0' /etc/fstab || echo 'nfs.<DC>.datacrunch.io:<PSEUDOPATH> /mnt/<SFS_NAME> nfs defaults,nconnect=16 0 0' | sudo tee -a /etc/fstab

Example:

grep -qxF 'nfs.fin-01.datacrunch.io:/grgwrgwrg-20f77f6a /mnt/storage nfs defaults,nconnect=16 0 0' /etc/fstab || echo 'nfs.fin-01.datacrunch.io:/grgwrgwrg-20f77f6a /mnt/storage nfs defaults,nconnect=16 0 0' | sudo tee -a /etc/fstab

Verify mount

Confirm the filesystem mounted successfully:

df -h

Look for your shared filesystem in the output:

Filesystem                                          Size  Used Avail Use% Mounted on
...
nfs.fin-01.datacrunch.io:/grgwrgwrg-20f77f6a       100G  1.0G   99G   1% /mnt/storage

Set permissions (optional)

Make the mounted storage writable by your user:

sudo chown ubuntu:ubuntu /mnt/<SFS_NAME>

Replace ubuntu:ubuntu with your username if different.

Find the pseudopath

After attaching the volume to an instance, open the Volumes page in the Spheron AI dashboard. Select your volume from the sidebar. The Pseudopath is displayed in the volume details; it starts with / and looks like /grgwrgwrg-20f77f6a.

The datacenter location slug is also visible there in the NFS mount command (e.g., nfs.fin-01.datacrunch.io).

Mount to every node in a cluster

To mount the shared filesystem across all nodes in a cluster at once, use pdsh. Replace the placeholders as above:

pdsh -a "sudo mkdir -vp /mnt/<SFS_NAME> && grep -qxF 'nfs.<DC>.datacrunch.io:<PSEUDOPATH> /mnt/<SFS_NAME> nfs defaults,nconnect=16 0 0' /etc/fstab || echo 'nfs.<DC>.datacrunch.io:<PSEUDOPATH> /mnt/<SFS_NAME> nfs defaults,nconnect=16 0 0' | sudo tee -a /etc/fstab && sudo mount /mnt/<SFS_NAME>"

Use your mounted volume

Access the storage

Once mounted, use the shared filesystem like any local directory:

# Navigate to the storage
cd /mnt/storage
 
# Create files and directories
mkdir my-project
echo "Hello, storage!" > my-project/readme.txt
 
# List contents
ls -lh /mnt/storage/

Check storage usage

# Check space on the mounted filesystem
df -h /mnt/storage
 
# Check detailed disk usage
du -sh /mnt/storage/*

Work with large datasets

Download datasets and checkpoints to the mounted volume. Data persists across instance restarts and is accessible to other instances in the same datacenter region.

# Example: Download dataset to storage
cd /mnt/storage
wget https://example.com/large-dataset.tar.gz
tar -xzf large-dataset.tar.gz

Unmount shared storage

Unmount when swapping volumes, detaching the filesystem, or terminating the instance.

Unmount the filesystem

sudo umount /mnt/<SFS_NAME>

If you get a "target is busy" error, check for active processes:

# Check what's using the storage
lsof /mnt/<SFS_NAME>
 
# Or use fuser
fuser -m /mnt/<SFS_NAME>

Remove from file system table

Remove the entry from /etc/fstab to prevent auto-mount on next boot:

# Edit fstab manually
sudo nano /etc/fstab
 
# Or remove automatically with sed
sudo sed -i '/datacrunch\.io.*nfs/d' /etc/fstab

Troubleshooting

Mount fails with "Connection refused"

Cause: Wrong datacenter slug, pseudopath, or filesystem not shared with this instance.

Solution:

Verify the volume is attached to your instance in the Spheron AI dashboard
Double-check the datacenter slug and pseudopath
Confirm the instance and filesystem are in the same datacenter location

Mount fails with "No such file or directory"

Cause: Mount point directory doesn't exist.

Solution:

sudo mkdir -p /mnt/<SFS_NAME>
sudo mount -t nfs -o nconnect=16 nfs.<DC>.datacrunch.io:<PSEUDOPATH> /mnt/<SFS_NAME>

Filesystem not mounting on boot

Cause: Network not ready when fstab mounts are processed.

Solution: Add the _netdev option to the fstab entry:

nfs.<DC>.datacrunch.io:<PSEUDOPATH> /mnt/<SFS_NAME> nfs defaults,nconnect=16,_netdev 0 0

The _netdev option tells the system to wait for the network before mounting.

Performance issues

Cause: Suboptimal nconnect value or high network latency.

Solutions:

Increase nconnect for more parallel connections (try 32)
Ensure the instance and filesystem are in the same datacenter

nfs.<DC>.datacrunch.io:<PSEUDOPATH> /mnt/<SFS_NAME> nfs defaults,nconnect=32 0 0

Check mount status

# List all NFS mounts
mount | grep nfs
 
# Check NFS statistics
nfsstat
 
# Verify fstab syntax
sudo mount -fav

Best practices

Organization:

Use descriptive mount points: /mnt/data, /mnt/models, /mnt/datasets
Create subdirectories for different projects
Document what data is stored where

Performance:

Use nconnect=16 or higher for better throughput
Keep instances and shared filesystems in the same datacenter location

Data safety:

Maintain backups of critical data
Filesystems persist independently of instances, but data can still be lost due to accidental deletion
Test backup and restore procedures

Security:

Restrict access using filesystem permissions
Only mount filesystems shared explicitly with your instance
Audit who has access to shared storage

What's next

Volume Mounting Overview: Multi-provider volume mounting guide
Volume API Reference: Create and manage volumes programmatically
SSH Connection Setup: Secure instance access

For questions, use chat support in the Spheron AI dashboard or contact Spheron support for infrastructure issues.