Vast.ai is a peer-to-peer GPU marketplace: individuals and datacenters rent out spare cards, and you bid for them. That makes it the cheapest way to get GPU-hours — often 30–50% below RunPod — at the cost of more variability. This guide shows how to pick a good host, launch an instance, and run a model without overpaying or leaking data.
When (and why) to use Vast.ai
Vast.ai is the right tool when:
- Price is the priority — RTX 3090/4090-class cards frequently list from ~$0.20–0.40/hr, and an RTX 3060 can dip to ~$0.06/hr.
- The job is interruptible — experiments, batch jobs, short fine-tunes where a lost instance just means restarting.
- You want to run a big model once to see its quality before committing to hardware — pair it with the GPU & VRAM checker to know what you'd need locally.
It's the wrong tool for sensitive data or production uptime — you may be running on someone's personal machine. For that, use RunPod Secure Cloud, Lambda, or keep it local.
Need more VRAM than you own? Spin up a cloud GPU big enough for any model in minutes — pay for the hours you use instead of buying a card.
Affiliate links — we may earn a commission if you sign up, at no extra cost to you.
Marketplace pricing — the cheapest per-hour rates for spot/interruptible GPUs.
Rent GPUs on Vast.ai →On-demand pods with a simple UI — good for a quick one-off inference or fine-tune.
Rent GPUs on RunPod →Step 1 — Create an account and add credit
- Sign up at Vast.ai and add credit (card or crypto).
- In your account settings, add an SSH public key — you'll use it to connect to instances.
Step 2 — Search the marketplace like a pro
The search page is a filterable list of offers. The settings that matter:
- GPU + VRAM — filter to the card you need (e.g. "RTX 4090", 24 GB). Use the VRAM Requirements Guide to size it.
- Reliability — Vast.ai shows a per-host reliability score. Prefer > 99% for anything that runs more than a few minutes.
- DLPerf / price — sort by "$/hr" for the cheapest, or by DLPerf-per-dollar for the best value.
- On-demand vs interruptible — interruptible is cheaper but can be outbid and paused. Use on-demand for anything you don't want killed mid-run.
- Internet speed — if you're downloading big model weights, a slow host wastes billed time.
Step 3 — Launch with a template
Choose a Docker image before you rent. Good starting points:
- Ollama or vLLM for LLM inference.
- PyTorch for fine-tuning.
- ComfyUI for image/video models.
Then click Rent. The instance boots in a minute or two.
Step 4 — Connect and run
# Vast.ai shows the exact SSH command per instance
ssh -p <port> root@<host> -L 8080:localhost:8080
# Install Ollama if it's not in your image, then run a model
curl -fsSL https://ollama.com/install.sh | sh
ollama run llama3.1:8bWatch the on-screen billing meter — Vast.ai bills per second while the instance is running.
Step 5 — Protect your budget and your data
- Destroy the instance when you're done. A running instance bills even while idle. Download outputs first.
- Never upload personal or proprietary data to a community host — assume the machine's owner could see the disk. Sensitive work belongs on Secure Cloud or local hardware.
- Prefer high-reliability hosts for long jobs; a cheap host that vanishes mid-fine-tune costs more than it saves.
- Checkpoint often during training so an interruption doesn't cost hours.
Vast.ai vs the alternatives
| You want… | Use |
|---|---|
| The absolute cheapest GPU-hours | Vast.ai (this guide) |
| A simpler UI + per-second serverless | RunPod |
| Guaranteed datacenter uptime | Lambda / Hyperstack — see the Cloud GPU Rental Guide |
| To stop renting and own the hardware | Build Guide + cost calculator |
FAQ
Is Vast.ai safe? For non-sensitive experimentation, yes. For confidential data, no — you may be on a stranger's machine. Keep private workloads on Secure Cloud or local.
Why is my instance so cheap? Interruptible/spot pricing and consumer hardware. The trade-off is it can be paused if someone outbids you.
How does Vast.ai compare to RunPod on price? Vast.ai is usually cheaper (marketplace competition); RunPod trades a little cost for a smoother UI and per-second serverless. Many people prototype on Vast.ai and move production to RunPod or local hardware.
Related Guides
To size a specific model, use the VRAM calculator: the memory it needs at every quantization and context length.