ROCm and AMD Instinct

Spheron AM offers AMD Instinct MI300X GPUs, the first non-NVIDIA silicon in the catalog. They run ROCm, AMD's compute stack, rather than CUDA. This page covers what changes for your workflow and what does not.

If you are deploying an NVIDIA GPU, see CUDA and NVIDIA drivers instead.

What ROCm is

ROCm (Radeon Open Compute) is AMD's equivalent of the CUDA stack: the kernel driver, runtime, compiler, and math libraries that let a framework run compute work on an AMD GPU. It ships pre-installed on every Spheron AM machine, so there is nothing to install before you start.

The parts map closely onto their NVIDIA counterparts:

MI300X hardware

The 192 GB of memory per GPU is the practical headline: a model that needs several 80 GB NVIDIA GPUs to hold its weights can fit on fewer MI300X cards, which changes how you shard it.

Verify the GPU after connecting

SSH into the instance, then query the GPU:

# Equivalent of nvidia-smi
rocm-smi

Check what the installed stack reports:

# ROCm version
cat /opt/rocm/.info/version
 
# HIP compiler and runtime configuration
hipconfig --version

nvidia-smi does not exist on these machines. A command not found for it is expected, not a fault.

PyTorch on ROCm

PyTorch supports ROCm through HIP. Install the ROCm build rather than the default CUDA wheels, matching the ROCm version reported above:

# Check the installed ROCm version first, then match the wheel index to it
pip install torch torchvision torchaudio \
  --index-url https://download.pytorch.org/whl/rocm6.2

import torch
 
print(torch.cuda.is_available())    # True
print(torch.version.hip)            # e.g. '6.2.41133-dd7f95766'
print(torch.version.cuda)           # None on a ROCm build
print(torch.cuda.get_device_name(0))

Porting CUDA code

Most PyTorch and TensorFlow code runs on ROCm without changes, because the frameworks absorb the difference. Attention is needed where code reaches below the framework:

• Framework-level code (PyTorch, TensorFlow, JAX): usually runs unchanged. Keep using .to("cuda").
• Custom CUDA kernels: port with hipify-perl or hipify-clang, which translate CUDA source to HIP.
• Prebuilt CUDA-only wheels: packages shipping compiled CUDA kernels (some quantization and attention libraries) need a ROCm build. Check for one before you deploy.
• Docker images: use a ROCm base image. A nvidia/cuda image does not run here.

# Translate a CUDA source tree to HIP
hipify-perl my_kernel.cu > my_kernel.hip.cpp
hipcc my_kernel.hip.cpp -o my_kernel

What differs on Spheron AM

Beyond the silicon, a few platform behaviors differ from other providers:

• One OS image. Machines boot Ubuntu with ROCm, plus Docker or Podman. There is no image picker, because the machine type determines what it boots.
• The boot disk comes with the machine type. Its size is shown on the offer rather than being something you set.
• No volumes. Persistent volumes are not available on Spheron AM. A deployment that asks for one is refused rather than created without it. Keep data you need to survive termination somewhere off the instance.
• Restart, but no stop. A Spheron AM instance bills at the full hourly rate until it is destroyed. Terminate an instance you have finished with. See Instance lifecycle.
• A minimum reservation, charged in full. Each machine type carries its own minimum runtime, which can be longer than the 20 minute default. Terminating early does not reduce the charge. The wizard shows the figure before you deploy.
• Startup scripts work normally. Cloud-init is supported, so the startup script flow is unchanged.

Troubleshooting

Issue: nvidia-smi: command not found

Symptoms: The command is missing after connecting to a Spheron AM instance.

Diagnosis: This is expected. The machine has an AMD GPU and no NVIDIA driver.

Resolution: Use rocm-smi instead.

Issue: PyTorch reports no GPU

Symptoms: torch.cuda.is_available() returns False.

Diagnosis: The installed PyTorch is a CUDA build, which cannot see an AMD GPU.

Resolution: Reinstall from the ROCm wheel index. Confirm the fix with torch.version.hip, which must be a version string rather than None.

pip uninstall -y torch torchvision torchaudio
pip install torch torchvision torchaudio \
  --index-url https://download.pytorch.org/whl/rocm6.2

Issue: A pip package fails to build or import

Symptoms: A library that compiles CUDA kernels fails to install, or imports and then errors on first use.

Diagnosis: The package ships CUDA-only compiled kernels with no ROCm build.

Resolution: Check the project for a ROCm or HIP build. Where none exists, either drop the dependency or run that part of the workload on an NVIDIA instance.

Issue: A Docker container sees no GPU

Symptoms: The GPU is invisible inside a container that works on NVIDIA hosts.

Diagnosis: The image is CUDA-based, or the container was started without the AMD device nodes.

Resolution: Use a ROCm base image and pass the devices through:

docker run --device=/dev/kfd --device=/dev/dri \
  --group-add video --ipc=host \
  rocm/pytorch:latest

What's next

• Instance lifecycle: Restart support and why stopping is unavailable here
• Regions and providers: Spheron AM inventory and region
• CUDA and NVIDIA drivers: The NVIDIA equivalent of this page
• Startup scripts: Automate setup on first boot
• SSH connection: Connect to your instance