Tencent Said To Have Demoted And Fired Tech Bosses After Brief Outages

A late March outage of Tencent's two major social platforms, WeChat and QQ, resulted in the firing of managers and directors, thereby proving that the hand of justice at the Chinese tech giant is, well, rather strong.

According to Chinese media outlet Jiemian News, which sits on Beijing's list of approved media outlets, WeChat and QQ went down for a number of hours on March 29. The outage made functionalities like WeChat voice, WeChat payment, QQ file transfer, QQ mailbox and more unavailable until the fault was repaired "later in the day."

The root cause was a cooling system failure at Guangzhou Telecom Data Center. The news outlet said Tencent's management felt the accident exposed an imperfect disaster recovery and emergency plan and the hidden dangers associated.

To pay for these sins, a senior executive vice president and a vice president were reprimanded, two directors were reportedly knocked down to two out of five stars on their performance review, and two general managers and directors were "demoted and dismissed."

We have asked Tencent to confirm the dismissals.

WeChat is often ranked as the most used app in China. At the end of 2022, Statista clocked the platform as having over 1.3 billion monthly active users.

Not only is it used by everyday people, many businesses depend on it, which means there is the potential for significant social disruption when it is unavailable.

"This severe punishment is a microcosm of Tencent's emphasis on security in recent years, especially in Tencent's increased B-end business, where corporate customers place more emphasis on stability and security," said Jiemian News.

• Google, Oracle cloud servers wilt in UK heatwave, take down websites
• Tencent CEO Pony Ma reportedly bemoans internal corruption, inefficiency
• AWS expands footprint at site of infamously flaky US-EAST-1 region
• Outage rates fall, but major ones will cost more. Oh and don't bank on SLAs

The Reg is by no means suggesting that execs should not have any accountability for the organizations they run. However, outages do happen and rarely do such heads roll.

Cloud services and servers hosted by Google and Oracle in the UK were disrupted last July thanks to cooling issues amid a heatwave. Both were back online after about 11.5 hours of turmoil.

Meanwhile, AWS's oldest region, US-EAST-1, is notoriously flaky, experiencing an eight-hour brownout in September 2021 followed by two wobbles in December 2021 – one which saw some servers and Elastic Block Store (EBS) volumes become unrecoverable.

Last month, Uptime Institute issued a report on the frequency of IT infrastructure outages. It determined that although outages were not on the rise in terms of percentage of structures, the amount it was costing companies was.

More than two-thirds of all blackouts are now costing organizations more than $100,000, said the org. ®