Whether you're training models, running inference, or experimenting with AI, GPU costs are likely your biggest expense. The good news: competition among providers has driven prices down significantly. The challenge: comparing pricing across 7+ providers with different commitment models, spot pricing, and instance types.
H100 Pricing: The Current King
NVIDIA's H100 SXM is the most sought-after GPU for AI workloads. Here's how providers compare for on-demand pricing:
| Provider | $/hr (on-demand) | Monthly (24/7) | Type |
|---|---|---|---|
| CoreWeave | $2.06 | $1,483 | Rental |
| Vast.ai | $2.20 | $1,584 | Marketplace |
| Lambda Labs | $2.49 | $1,793 | Rental |
| RunPod | $3.29 | $2,369 | Rental |
| Azure | $3.67 | $2,642 | Cloud |
| GCP | $3.76 | $2,707 | Cloud |
| AWS | $4.14 | $2,981 | Cloud |
Cloud vs GPU Marketplace vs On-Premise
Major Cloud Providers (AWS, GCP, Azure)
- Highest prices but strongest reliability and SLAs
- Best integration with existing cloud infrastructure
- Reserved instances (1-3 year) can cut costs 40-60%
- Spot/preemptible instances offer 60-70% savings but can be interrupted
GPU Rental Platforms (Lambda, CoreWeave, RunPod)
- 30-50% cheaper than major clouds for on-demand
- Purpose-built for ML workloads
- Less overhead — no need to manage full cloud infrastructure
- Availability can be limited for popular GPUs
GPU Marketplaces (Vast.ai)
- Cheapest option — peer-to-peer GPU rental
- Highly variable pricing and reliability
- Best for experimental work, not production
A100 80GB: The Workhorse
If you don't need the latest hardware, the A100 80GB offers excellent value:
| Provider | $/hr (on-demand) | Monthly (24/7) |
|---|---|---|
| Vast.ai | $1.10 | $792 |
| CoreWeave | $1.28 | $922 |
| Lambda Labs | $1.29 | $929 |
| RunPod | $1.64 | $1,181 |
| GCP | $2.21 | $1,591 |
| Azure | $2.48 | $1,786 |
| AWS | $3.06 | $2,203 |
On-Premise: When Does It Make Sense?
Purchasing GPUs outright makes sense when you have consistent, high utilization. An H100 costs roughly $30,000 to purchase. Amortized over 4 years with electricity, the effective monthly cost is around $700-800 — compared to $1,500-3,000/month in the cloud.
The break-even point for buying vs renting is typically 12-18 months of 24/7 usage against the cheapest cloud provider, or 6-8 months against major clouds.
Tips for Reducing GPU Costs
- Use spot instances for training — Training can be checkpointed and resumed. The 60-70% savings outweigh the interruption risk.
- Right-size your GPU — An RTX 4090 ($0.25-0.39/hr) handles many inference workloads that don't need 80GB VRAM.
- Commit when you can — 1-year reserved instances save 40% on AWS/GCP/Azure.
- Consider mixed strategies — Production inference on reserved instances, training on spot, experiments on Vast.ai.
Use our GPU Cost Calculator to compare prices across all providers for your specific GPU and usage pattern.