Cost optimization

This page covers strategies for reducing compute costs on Spheron, including moving non-GPU work to CPU nodes, GPU tier selection, instance type trade-offs, reserved GPU savings, and spend monitoring.

Move non-GPU work to CPU nodes

The largest saving is not renting a GPU at all. Data preprocessing, ETL, build steps, schedulers, queue workers, and API front ends never touch CUDA, and running them on a GPU instance wastes the accelerator you are paying for.

CPU Node starts at $0.09/hr on Verda spot, against ~$0.25/hr for the cheapest GPU spot instance and several dollars per hour for an H100. Sizes start at 4 vCPU with 4 GB of memory. Check the Deploy CPU page for the sizes on offer right now.

1. Preprocess and shard the dataset on a CPU node, writing to a persistent volume.
2. Attach the same volume to a GPU instance and run training.
3. Terminate the GPU instance the moment training finishes; leave the CPU node running for postprocessing if needed.

Deploy CPU nodes from Deploy CPU in the dashboard sidebar, or through the API with computeType=cpu.

GPU tier selection matrix

Choose the right GPU tier based on how much VRAM your workload needs:

Check current prices in the dashboard; prices vary by provider and availability.

Instance type strategy

Spot (lowest cost)

Spot instances are 30-60% cheaper than Dedicated. The trade-off: the provider can reclaim them at any time.

• Experiments and hyperparameter search
• Batch training jobs with checkpoint saving enabled
• Any workload under 4 hours that can tolerate interruption

Handling interruption: Save checkpoints to a persistent volume every N steps. If the instance is reclaimed, resume from the latest checkpoint on a new instance without losing progress.

# Save checkpoint every 100 steps
if step % 100 == 0:
    torch.save(state, '/checkpoints/checkpoint_latest.pt')

Dedicated (guaranteed)

Dedicated instances cannot be reclaimed. Use them when interruption would be costly:

• Production inference servers
• Multi-day training runs
• Interactive workloads and demos

Cluster (largest scale)

Full physical servers with NVLink interconnects. Use for:

• Multi-GPU distributed training (PyTorch DDP, DeepSpeed)
• Workloads requiring maximum GPU-to-GPU bandwidth
• Jobs sensitive to virtualization overhead

Stop an instance instead of leaving it running

An instance you are coming back to does not have to keep paying for its GPU. Stopping releases the GPU, keeps the disk and everything on it, and bills only for the retained disk plus the public IP where the provider charges for one. Starting it again returns the same disk, the same contents, and the same public IP.

This is the cheapest option for work that pauses rather than ends:

• An environment you spent hours building and will use again tomorrow
• A dataset already downloaded to the boot disk that you do not want to fetch twice
• An overnight or weekend gap in an otherwise active project

Stopping is available on Spheron AI and Spheron ES, and on Spheron AI it depends on the machine type and region. Verda, Sesterce, Spheron MS, and Massed Compute keep billing at the full rate until an instance is destroyed. Check the note in the deploy summary before you commit.

Reserved GPUs for long-term work

For multi-week or multi-month projects, Reserved GPUs offer significant savings:

• Submit requests via dashboard > Reserved GPU
• Multiple providers compete to offer the lowest price
• Typical savings: 30-50% vs on-demand hourly rates for 3-12 month commitments
• Select "Any Location" to maximize provider competition

See Reserved GPUs for the request form.

Team discount program

Teams with active discounts automatically see reduced prices on the dashboard. The discounted price is applied at deployment without any additional steps.

• Discounts are either volume-based or admin-assigned; the higher of the two is applied automatically

To inquire about discount eligibility for high-volume usage, use the chat widget on app.spheron.ai or email info@spheron.ai.

Monitoring burn rate

Check remaining balance

View your current credit balance on the Billing page in the dashboard. The balance updates in real time as instances run.

Track per-instance spend

Open the instance details drawer from the Instances page to see the hourly rate and total cost accumulated for a running deployment.

Stop or terminate when done

Terminate instances from the dashboard as soon as your workload finishes to stop charges immediately. Navigate to Instances, select the instance, and click Terminate.

When you will return to the same environment, click Stop instead where the provider offers it. The GPU stops billing, the disk is preserved, and you start it again without rebuilding. The dialog quotes both rates before you commit.

Set up balance alerts in User Settings to receive a notification before credits run out.

Practical tips

Use persistent volumes for datasets and model weights. Avoid re-downloading multi-GB datasets on every deployment; mount a volume with data pre-loaded. This saves both time and egress costs.

Prefer Spot for short jobs. Any job under 4 hours that can be checkpointed is a good Spot candidate. Switch to Dedicated for multi-day runs requiring uninterrupted time.

Batch GPU use. Avoid leaving instances running idle. Terminate immediately when your job finishes, or stop the instance when you will return to the same environment, and re-deploy from a checkpoint when neither applies.

Use RTX 4090 for development. The RTX 4090 is the most cost-effective GPU for code iteration, small model experiments, and inference serving at low traffic. Move to A100/H100 only when VRAM or compute requirements demand it.

What's next

• Instance Types: Detailed Spot/Dedicated/Cluster comparison
• Regions & Providers: Provider capabilities and GPU tiers
• Reserved GPUs: Long-term GPU reservation form
• Instance Lifecycle: Stop and start an instance to pause GPU billing
• Billing: Credit management, auto top-up, and team discounts
• Volume mounting: Persistent storage for datasets and checkpoints