Broadcom CEO Predicts Hyperscalers Poised To Build Million-accelerator Clusters

Broadcom CEO Hock Tan has predicted his hyperscale semiconductor customers will continue building AI clusters for another three to five years, with each generation of machines to double in size.

Speaking to Jim Cramer of CNBC's Mad Money on Tuesday, Tan was asked to explain the recent performance of Broadcom stock. The share price dipped markedly after its Q3 results were felt by some to indicate its chips biz was off the boil as ardor for AI eased. It rebounded a week later, after other market signals indicated demand for custom AI silicon could be strong.

Tan liked that argument, telling Cramer he's aware of hyperscalers' roadmaps that suggest a three-to-five year plan "to build out these large clusters" that allow development of new large language models.

The Broadcom boss said those builds could be annual, and each will require two or three times the compute power of their predecessors.

That will create a larger "compute opportunity" that Tan predicted will be met by "XPUs" – AI acceleration silicon for the network and other components – that the chip shop has predicted will grow faster than GPUs in coming years. XPUs are also a product Broadcom will happily make for hyperscalers – in bulk.

Tan noted that the AI clusters hyperscalers are building today use 100,000 accelerators, but future rigs will require a million. Broadcom wants to design them just for hyperscale customers – a segment Tan feels is his best potential target.

And fair enough: Nvidia and AMD look set to fight it out for the GPU market and associated software stacks. And Meta has already signalled a desire to build custom AI silicon to run alongside 600,000 Nvidia GPUs.

Nvidia has promised its supply chain can deliver the GPUs the world wants, but has form of sputtering deliveries – as is reasonable during these boom times.

If Broadcom can help hyperscalers to design rigs that don't depend on known chokepoints, it could thrive.

• Oracle boasts zettascale 'AI supercomputer,' just don’t ask about precision
• Microsoft, BlackRock form fund to sink up to $100B into AI infrastructure
• Broadcom boss Hock Tan says public cloud gave IT departments PTSD
• Broadcom has brought VMware down to earth and that’s welcome

Tan also observed that the semiconductor market bottomed out this year, as part of the industry's normal cycles – But COVID-19 created an unusual upcycle that turned downwards in late 2023 and early 2024.

He predicted that 2025 and 2026 will be an upcycle for non-AI silicon.

Asked if that turnabout means sales of Wi-Fi and storage connectivity silicon will improve – after respectively achieving stasis and a slump last quarter – Tan said that will "absolutely" be the case.

"We are already seeing strong sequential growth from the bottom of Q2 this year," Tan said, thanks to demand from enterprises.

Tan was also asked about VMware, and opined that his reforms at the virtualization giant are going well.

"What we've done is improved the products and made it much more usable from just being compute virtualization to basically creating a virtualization of entire datacenter, on-prem, and creating the cloud experience, on-prem."

For what it's worth, VMware has offered compute, storage, and network virtualization for a decade. Tan was probably referring to Cloud Foundation 5.2, which for the first time allows virtual compute, storage, and networks to be brought under a single management umbrella. That represents around 80 percent of the work Broadcom says is needed to realize its vision of a fully virtualized datacenter – which it claims is coming, without committing to a timeline. ®