A supercomputer in China ran a ‘brain-scale’ AI model with 174 trillion parameters

Scientists in China say they have been able to run an artificial intelligence model as sophisticated as a human brain on their most powerful supercomputer, a report from the South China Morning Post reveals.

According to the report, this puts China's Newest Generation Sunway supercomputer on the same level as the U.S. Department of Energy's Frontier, which was named the world's most powerful supercomputer earlier this month.

As a point of reference, Frontier is the first machine to have demonstrated it can perform more than one quintillion calculations per second.

Researchers say their new AI model rivals the human brain

The team of researchers from China used the Sunway supercomputer to train an AI model called 'bagualu,' which means "alchemist's pot." They presented their results at a virtual meeting of Principles and Practice of Parallel Programming 2022, an international conference hosted by the US-based Association for Computing Machinery (ACM).

They trained bagualu with a total of 174 trillion parameters, which according to SCMP rivals the number of synapses in the brain.

In truth, though the exact number of synapses in a brain is incredibly difficult to map, some estimates suggest the human brain contains up to 1,000 trillion synapses. That's not to say 174 trillion parameters isn't an incredibly impressive number when it comes to the field of artificial intelligence — last year, Google Brain was celebrated for developing an artificial intelligence language model with 1.6 trillion parameters.

A 'brain-scale' supercomputer

The Sunway supercomputer has a speed of a billion operations per second, or 5.3 floating-point operations per second (exaflops). According to the researchers, it has 37 million CPU cores — four times as many as Frontier — and nine petabytes of memory. They also claim the 96,000 semi-independent computer systems, called nodes, resemble the power of a human brain. Communications between these nodes take place at a speed of more than 23 petabytes per second.

The researchers, who refer to their model as a "brain-scale" AI model, say the new Sunway uses proprietary chips with unique features, including energy-saving and broad communication bandwidth.

The team behind the "brain-scale" AI model says their work could be used for autonomous vehicles, computer vision, facial recognition, and chemistry, among a number of other applications. Another Chinese supercomputer, called Tianhe, from the National Supercomputing Centre of Tianjin, was used to rapidly screen and discover pharmaceuticals during the early peak of the Covid-19 pandemic, giving a glimpse at the real-world use cases for these incredibly powerful machines.