Meta’s insatiable urge for food for Nvidia AI chips might add as much as $7 billion price of H100 GPUs by the tip of the 12 months

Keep in mind once we reported some market knowledge indicating Meta had snapped up 150,000 of Nvidia’s beastly H100 AI chips? Now it seems Meta desires to have greater than double that at 350,000 of the massive silicon issues by the tip of the 12 months.

Most estimates put the value of an Nvidia H100 GPU at between $20,000 and $40,000. Meta together with Microsoft are Nvidia’s two greatest prospects for H100, so it is protected to imagine will probably be paying lower than most.

Nonetheless, some primary arithmetic round Nvidia’s reported AI revenues and its unit shipments of H100 reveals that Meta cannot be paying an terrible lot if any lower than the decrease finish of that worth vary, which works out to roughly $7 billion price of Nvidia AI chips over two years. For certain, then, on the very least we’re speaking about many billion of {dollars} being spent by Meta on Nvidia silicon. 

Meta has additionally now revealed some particulars (by way of ComputerBase) about how the H100s are applied. Apparently, they’re constructed into 24,576-strong clusters used for coaching language fashions. On the decrease finish per-unit H100 worth estimate, that works out at $480 million price of H100s per cluster. Meta has offered perception into two barely totally different variations of those clusters. 

To cite ComputerBase, one “depends on Distant Direct Reminiscence Entry (RDMA) over Converged Ethernet (RoCE) primarily based on the Arista 7800, Wedge400 and Minipack2 OCP parts, the opposite on Nvidia’s Quantum-2 InfiniBand resolution. Each clusters talk by way of 400 Gbps quick interfaces.”

In all candour, we could not inform our Arista 7800 arse from our Quantum-2 InfiniBand elbow. So, all that does not imply a lot to us past the realisation that there is much more in {hardware} phrases to knocking collectively some AI coaching {hardware} than simply shopping for a bunch of GPUs. Hooking them up would seem like a serious technical problem, too.

Nevertheless it’s the 350,000 determine that’s the actual eye opener. Meta was estimated to have purchased 150,000 H100s in 2023 and hitting that 350,000 goal means upping that to completely 200,000 extra of them this 12 months.

In keeping with the figures offered by analysis outfit Omnia late final 12 months, Meta’s solely rival for sheer quantity of H100 acquisition was Microsoft. Even Google was solely thought to have purchased 50,000 items in 2023. So it must ramp up its purchases by a really spectacular quantity to even come near what Meta is shopping for.

For what it is price, Omnia predicts that the general marketplace for these AI GPUs might be double the dimensions in 2027 that it was in 2023. So, Meta shopping for an enormous quantity of H100s in 2023 and but much more of them this 12 months matches the obtainable knowledge properly sufficient.

What this all means for the remainder of us is anybody’s guess. It is arduous to foretell all the pieces from the impression this large demand for AI GPUs can have on availability of humble graphics-rendering chips for PC gaming, not to mention the main points of how upcoming AI will advance over the following 12 months or two.

As if these numbers weren’t large enough, Omdia reckons Nvidia’s revenues for these huge knowledge GPUs will really be double in 2027 what it’s at present. They will not all be used for big language fashions and related AI functions, however numerous them will.

It is fascinating that Omdia is so bullish about Nvidia’s prospects on this market even supposing lots of Nvidia’s greatest prospects on this market are literally planning on constructing their very own AI chips. Certainly, Google and Amazon already do, which might be why they do not purchase as many Nvidia chips as Microsoft and Meta.

Robust competitors can be anticipated to come back from AMD’s MI300 GPU and its successors, too, and there are newer startups together with Tenstorrent, led by one of the crucial extremely regarded chip architects on the planet, Jim Keller, making an attempt to get in on the motion.

One factor is for certain, nevertheless, all of those numbers do moderately make the gaming graphics market look puny. We have created a monster, of us, and there is not any going again.