[ad_1]
If you’re flexible about your GPU model and want to choose the most efficient cloud GPU, check out our cloud GPU $/bandwidth chart.
If you know a specific cloud GPU (eg A100) you want, name the cloud providers that offer it.
Cloud GPU cost per bandwidth
Spot GPUs
cloud | GPU Type / Memory | # GPU | on the place | Bandwidth * | Bandwidth* / $*** |
---|---|---|---|---|---|
Jarvislabs | A100 / 40 GB | 1 | 0.69 | 179 | 259 |
GCP | A100 / 40 GB | 1 | 1.1 | 179 | 163 |
blue | A100 / 80 GB | 1 | 1.47 | 232 | 157 |
blue | A100 / 80 GB | 4 | 5.88 | 821 | 140 |
blue | A100 / 80 GB | 2 | 2.94 | 406 | 138 |
GCP | V100 / 16 GB | 1 | 0.84 | 42 | 50 |
GCP | V100 / 16 GB | 2 | 1.68 | 77 | 46 |
AWS | V100 / 16 GB | 1 | 0.92 | 42 | 46 |
GCP | V100 / 16 GB | 4 | 3.36 | 153 | 46 |
GCP | V100 / 16 GB | 8 | 6.72 | 289 | 43 |
AWS | V100 / 16 GB | 4 | 3.67 | 153 | 42 |
AWS | V100 / 16 GB | 8 | 7.34 | 289 | 39 |
blue | V100 / 16 GB | 1 | 1.35 | 42 | 31 |
blue | V100 / 16 GB | 2 | 2.7 | 77 | 29 |
blue | V100 / 16 GB | 4 | 5.4 | 153 | 28 |
* Training throughput is a good metric to measure GPU relative performance. It measures the number of tokens processed by the GPU per second for a language model (ie bert_base_squad).1 Please note that this bandwidth should be taken into account when comparing different GPUs. The same hardware will have significantly different throughput for your workload because there is significant bandwidth even between LLMs running on the same hardware.2
*** Does not include cost of storage, network performance, ingress/egress etc. This is just the GPU cost.3
In all this bandwidth per dollar:
- Not all possible configurations are listed, including deep learning-oriented configurations similar to AWS options.
- The western or central regions of the US were used whenever possible
- These are listed prices for each category, they can be improved
On-demand GPUs from major technical cloud providers
cloud | GPU Type / Memory | # GPU | $ on request | Bandwidth * | Bandwidth* / $*** |
---|---|---|---|---|---|
blue | A100 / 80 GB | 1 | 3.67 | 232 | 63 |
blue | A100 / 80 GB | 4 | 14.69 | 821 | 56 |
blue | A100 / 80 GB | 2 | 7.35 | 406 | 55 |
GCP | A100 / 40 GB | 1 | 3.67 | 179 | 49 |
blue | A100 / 80 GB | 8 | 37.18 | 1362 | 37 |
AWS | A100 / 80 GB | 8 | 40.97 | 1362 | 33 |
GCP | V100 / 16 GB | 1 | 2.95 | 42 | 14 |
AWS | V100 / 16 GB | 1 | 3.06 | 42 | 14 |
blue | V100 / 16 GB | 1 | 3.06 | 42 | 14 |
GCP | V100 / 16 GB | 2 | 5.91 | 77 | 13 |
GCP | V100 / 16 GB | 4 | 11.81 | 153 | 13 |
AWS | V100 / 16 GB | 4 | 12.24 | 153 | 13 |
blue | V100 / 16 GB | 2 | 6.12 | 77 | 13 |
blue | V100 / 16 GB | 4 | 12.24 | 153 | 13 |
GCP | V100 / 16 GB | 8 | 23.63 | 289 | 12 |
AWS | V100 / 16 GB | 8 | 24.48 | 289 | 12 |
On-demand GPU from other cloud providers
cloud | GPU Type / Memory | # GPU | $ on request | Bandwidth * | Bandwidth* / $ |
---|---|---|---|---|---|
Jarvislabs | A100 / 40 GB | 1 | 1.1 | 179 | 163 |
Lambda | A100 / 40 GB | 1 | 1.1 | 179 | 163 |
Lambda | H100 / 80 GB | 1 | 1.99 | 322 | 162 |
Datacrunch | A100 / 80 GB | 1 | 1.85 | 232 | 125 |
latitude | H100 / 8 GB | 8 | 22.42 | 2693 | 120 |
Lambda | A100 / 80 GB | 8 | 12 | 1362 | 114 |
Datacrunch | A100 / 80 GB | 4 | 7.4 | 821 | 111 |
Datacrunch | A100 / 80 GB | 2 | 3.7 | 406 | 110 |
latitude | H100 / 8 GB | 4 | 11.96 | 1321 | 110 |
Datacrunch | A100 / 80 GB | 8 | 14.8 | 1362 | 92 |
Oblivus Cloud | A100 / 80 GB | 1 | 2.55 | 232 | 91 |
RunPod | A100 / 80 GB | 8 | 15.12 | 1362 | 90 |
Oblivus Cloud | A100 / 80 GB | 2 | 5.1 | 406 | 80 |
Oblivus Cloud | A100 / 80 GB | 4 | 10.2 | 821 | 80 |
Oblivus Cloud | A100 / 40 GB | 1 | 2.39 | 179 | 75 |
paper space | A100 / 80 GB | 1 | 3.18 | 232 | 73 |
Oblivus Cloud | A100 / 80 GB | 8 | 20.4 | 1362 | 67 |
Lambda | V100 / 16 GB | 8 | 4.4 | 289 | 66 |
Oblivus Cloud | V100 / 16 GB | 1 | 0.65 | 42 | 65 |
paper space | A100 / 80 GB | 4 | 12.72 | 821 | 65 |
paper space | A100 / 80 GB | 2 | 6.36 | 406 | 64 |
Oblivus Cloud | V100 / 16 GB | 2 | 1.3 | 77 | 59 |
Oblivus Cloud | V100 / 16 GB | 4 | 2.6 | 153 | 59 |
paper space | A100 / 40 GB | 1 | 3.09 | 179 | 58 |
paper space | A100 / 80 GB | 8 | 25.44 | 1362 | 54 |
exoscale | V100 / 16 GB | 4 | 3.32 | 153 | 46 |
Datacrunch | V100 / 16 GB | 1 | 1 | 42 | 42 |
Datacrunch | V100 / 16 GB | 2 | 2 | 77 | 39 |
Datacrunch | V100 / 16 GB | 4 | 4 | 153 | 38 |
exoscale | V100 / 16 GB | 2 | 2.01 | 77 | 38 |
Datacrunch | V100 / 16 GB | 8 | 8 | 289 | 36 |
exoscale | V100 / 16 GB | 1 | 1.38 | 42 | 30 |
OVHcloud | V100 / 16 GB | 1 | 1.97 | 42 | 21 |
OVHcloud | V100 / 16 GB | 2 | 3.94 | 77 | 20 |
OVHcloud | V100 / 16 GB | 4 | 7.89 | 153 | 19 |
paper space | V100 / 16 GB | 1 | 2.3 | 42 | 18 |
Cloud GPU availability in different clouds
Enter the desired model in the search field to identify all cloud providers that offer it:
provider | GPU | Multi-GPU | On Demand $ / One GPU Hour*** |
---|---|---|---|
AWS | M60 8 GB | 1, 2, 4x | $1.14 |
AWS | T4 16 GB | 1, 2, 4, 8x | $1.20 |
AWS | A10G 24 GB | 1, 4, 8x | $1.62 |
AWS | V100 16 GB | 1, 4, 8x | $3.06 |
AWS | V100 32GB | 8x | $3.90** |
AWS | A100 40 GB | 8x | $4.10** |
AWS | A100 80 GB | 8x | $5.12 ** |
CoreWeave | Quadro RTX 4000 8 GB | 1, 2, 3, 4, 5, 6, 7x | $0.35 |
CoreWeave | RTX A4000 16 GB | 1, 2, 3, 4, 5, 6, 7x | $0.75 |
CoreWeave | Quadro RTX 5000 16 GB | 1, 2, 3, 4x | $0.95 |
CoreWeave | V100 16 GB | 1, 2, 3, 4, 5, 6, 7, 8x | $1.00 |
CoreWeave | RTX A5000 24 GB | 1, 2, 3, 4x | $1.01 |
CoreWeave | RTX A40 48 GB | 1, 2, 3, 4, 5, 6, 7, 8x | $1.68 |
CoreWeave | RTX A6000 48 GB | 1, 2, 3, 4, 5, 6, 7, 8x | $1.68 |
CoreWeave | A100 40 GB | 1, 2, 3, 4, 5, 6, 7, 8x | $2.46 |
CoreWeave | A100 80 GB | 1, 2, 3, 4, 5, 6, 7, 8x | $2.61 |
GCP | T4 16 GB | 1, 2, 4x | $0.75 |
GCP | K80 12 GB | 1, 2, 4, 8x | $0.85 |
GCP | P4 8 GB | 1, 2, 4x | $1.00 |
GCP | P100 for 16GB | 1, 2, 4x | $1.86 |
GCP | V100 16 GB | 1, 2, 4, 8x | $2.88 |
GCP | A100 40 GB | 1, 2, 4, 8, 16x | $3.67 |
Jarvis Laboratories | Quadro RTX 5000 16 GB | 1x | $0.49 |
Jarvis Laboratories | Quadro RTX 6000 24 GB | 1x | $0.99 |
Jarvis Laboratories | RTX A5000 24 GB | 1x | $1.29 |
Jarvis Laboratories | RTX A6000 48 GB | 1x | $1.79 |
Jarvis Laboratories | A100 40 GB | 1x | $2.39 |
Lambda Labs | Quadro RTX 6000 24 GB | 1, 2, 4x | $1.25 |
Lambda Labs | RTX A6000 48 GB | 1, 2, 4x | $1.45 |
Lambda Labs | V100 16 GB | 8x | $6.80 |
Linode | Quadro RTX 6000 24 GB | 1, 2, 4x | $1.50 |
blue | K80 12 GB | 1, 2, 4x | $0.90 |
blue | T4 16 GB | 1, 4x | $1.20 |
blue | P40 24 GB | 1, 2, 4x | $2.07 |
blue | P100 for 16GB | 1, 2, 4x | $2.07 |
blue | V100 32GB | 8x | $2.75 |
blue | V100 16 GB | 1, 2, 4x | $3.06 |
blue | A100 40 GB | 8x | $3.40** |
blue | A100 80 GB | 1, 2, 4x | $3.67 |
blue | A100 80 GB | 8x | $4,096** |
OVH | V100 16 GB | 1, 2, 4x | $1.79 |
OVH | V100 32 GB | 1, 2, 4x | $1.99 |
paper space | Quadro M4000 8 GB | 1x | $0.45 |
paper space | Quadro P4000 8 GB | 1, 2, 4x | $0.51 |
paper space | Quadro RTX 4000 8 GB | 1, 2, 4x | $0.56 |
paper space | RTX A4000 16 GB | 1, 2, 4x | $0.76 |
paper space | Quadro P5000 16 GB | 1, 2, 4x | $0.78 |
paper space | Quadro RTX 5000 16 GB | 1, 2, 4x | $0.82 |
paper space | Quadro P6000 24 GB | 1, 2, 4x | $1.10 |
paper space | RTX A5000 24 GB | 1, 2, 4x | $1.38 |
paper space | RTX A6000 48 GB | 1, 2, 4x | $1.89 |
paper space | V100 32GB | 1, 2, 4x | $2.30 |
paper space | V100 16 GB | 1x | $2.30 |
paper space | A100 40 GB | 1x | $3.09 |
paper space | A100 80 GB | 1, 2, 4, 8x | $3.19 |
** Calculated values. This was required when single GPU instances were not available.4 5
Disclaimer
Please note that cloud providers are constantly updating their offerings, so there may be updates.
External links
- Deep Learning GPU Benchmarks, Lambda Labs, Received July 15, 2023
- Open LLM-Perf Leaderboard Hugging Face Read on July 15, 2023
- the-full-stack/website/docs/cloud-gpus, GitHub, retrieved 15 July 2023
- The Definitive Guide to Cloud GPU Providers, Paperspace, Received July 15, 2023
- CloudOptimizer, CloudOptimizer, Retrieved July 15, 2023
Share LinkedIn
[ad_2]
Source link