Price, availability, $ / performance in 2023

If you’re flexible about your GPU model and want to choose the most efficient cloud GPU, check out our cloud GPU $/bandwidth chart.

If you know a specific cloud GPU (eg A100) you want, name the cloud providers that offer it.

Cloud GPU cost per bandwidth

Spot GPUs

* Training throughput is a good metric to measure GPU relative performance. It measures the number of tokens processed by the GPU per second for a language model (ie bert_base_squad).¹ Please note that this bandwidth should be taken into account when comparing different GPUs. The same hardware will have significantly different throughput for your workload because there is significant bandwidth even between LLMs running on the same hardware.²

*** Does not include cost of storage, network performance, ingress/egress etc. This is just the GPU cost.³

In all this bandwidth per dollar:

• Not all possible configurations are listed, including deep learning-oriented configurations similar to AWS options.
• The western or central regions of the US were used whenever possible
• These are listed prices for each category, they can be improved

On-demand GPUs from major technical cloud providers

On-demand GPU from other cloud providers

Cloud GPU availability in different clouds

Enter the desired model in the search field to identify all cloud providers that offer it:

** Calculated values. This was required when single GPU instances were not available.^{4 5}

Disclaimer

Please note that cloud providers are constantly updating their offerings, so there may be updates.

External links

1. Deep Learning GPU Benchmarks, Lambda Labs, Received July 15, 2023
2. Open LLM-Perf Leaderboard Hugging Face Read on July 15, 2023
3. the-full-stack/website/docs/cloud-gpus, GitHub, retrieved 15 July 2023
4. The Definitive Guide to Cloud GPU Providers, Paperspace, Received July 15, 2023
5. CloudOptimizer, CloudOptimizer, Retrieved July 15, 2023

Share LinkedIn

Cem has been the chief analyst at AIMultiple since 2017. AIMultiple reports to hundreds of thousands of businesses each month (according to similar websites) including 55% of the Fortune 500.

Jam’s work has been cited by leading global publications including Business Insider, Forbes, Washington Post, global firms such as Deloitte, HPE, and NGOs such as the World Economic Forum and supranational organizations such as the European Commission. You can see more reputable companies and resources referencing AIMultiple.

Throughout his career, Jam has worked as a tech consultant, tech buyer, and tech entrepreneur. For more than ten years, he advised enterprises on technology solutions at McKinsey & Company and Altman Solon. He also published a McKinsey report on digitization.

He led technology strategy and acquisitions for the phone company, reporting to the CEO. He also led the commercial growth of deep technology company Hypatos, achieving 7-figure annual recurring revenue and 9-figure valuation from 0 to 2 years. Jam’s work at Hypatos has been covered by leading technology publications like TechCrunch, Business Insider.

Jam regularly speaks at international technology conferences. He graduated from Bogazici University as a computer engineer and holds an MBA from Columbia Business School.