Facing A 30% Price Rise To Park Servers In A Colo? Blame AI

The rapid adoption of artificial intelligence, particularly by cloud providers, has strained datacenter capacity and led to increased hosting prices, according to a report from property services and investment management company JLL.

The firm’s North American Datacenter Report released on Sunday, asserts that a shortage of colocation space has driven up prices by between 20 and 30 percent year-over-year in primary markets.

"Major cloud service providers are growing rapidly to support new AI requirements and the need for more computing power," the report reads. "This has led to a significant surge in leasing in the second quarter of 2023."

Part of the problem is caused by the systems needed to run generative AI models requiring more power and space than is required for other applications.

As we've previously reported, the GPU clusters commonly deployed to run AI training and inference workloads are not only expensive — often requiring thousands of GPUs — but incredibly demanding in terms of power and heat management. A typical eight GPU node can easily consume 6-10kW depending on the accelerators and cooling technologies employed.

"With the increase of AI requirements, datacenter operators need to install infrastructure to accommodate high-power-density server clusters with some large requirements driving densities to 50-100kW per rack," the authors of JLL’s report wrote.

By way of comparison, kit in a typical hyperscale operators racks consume somewhere between 6kW and 10kW.

• Microsoft hiring a nuclear power program manager, because AI needs lots of 'leccy
• Schneider Electric warns that existing datacenters aren't buff enough for AI
• Nvidia's 900 tons of GPU muscle bulks up server market, slims down wallets
• The future of the cloud sure looks like it'll be paved in even more custom silicon

In the wake of the AI boom, several colocation providers have introduced facilities to accommodate the workload. In August, for example, Digital Realty announced support for deployments of up to 70kW per rack to support customers looking to deploy AI and high-performance workloads in its datacenters.

Doing so required the use of "air-assisted liquid cooling" which as The Register understands it amounts to using rear-door heat exchangers to extract heat from air-cooled servers. Other colocation providers that specialise in AI and high-performance computing have expanded support for direct liquid cooling. Santa Clara-based Colovore, for example, will support racks up to 250kW in a datacenter it plans to bring online in 2024.

Datacenter infrastructure vendors are also considering the challenge posed by dense and hot AI rigs. In a whitepaper published last week, French multinational Schneider Electric identified why AI workloads were so difficult to work with, namely that one way to manage the systems is to spread them among racks, but doing so creates network bottlenecks.

According to JLL, strong demand for AI products and services is also driving datacenter operators to build capacity further out to the edge. Typically, cloud providers and hyperscalers tend to build a handful of large campuses in a handful of key markets around the world.

However, in an effort to drive down latency, these companies are said to be building smaller edge datacenters in the range of 2-10 megawatts to put their models in closer proximity to customers.

JLL expects the datacenter supply crunch to persist through 2024.

"Most of the supply that will be delivered in the third and fourth quarter of 2023 has been preleased or is under exclusivity. Much of the anticipated 2024 supply will also be preleased," the report states.

As such the report's authors suggest secondary US cities such as Columbus, Salt Lake City, Reno, and Austin may benefit from the shortages in larger centres. ®