AI Researchers Say Modern Computers Are Squeezed Dry

Modern computing has reached the end of the rope when it comes to scaling up AI.

| Jul 17, 2020 02:47 PM EST

Created: Jul 17, 2020 02:47 PM EST

Modern computers are out of their depth when it comes to deep learning and AI, according to recent research from MIT shared on a pre-print website.

Modern computers can't handle perpetual AI scaling

In essence, we've exhausted the computing potential of modern computers, and researchers say we'll soon exhaust economic and environmental ways to keep scaling up deep learning systems.

See Also

"Progress along current lines is rapidly becoming economically, technically, and environmentally unsustainable. Thus, continued progress in these applications will require dramatically more computationally-efficient methods, which will either have to come from changes to deep learning or from moving to other machine learning methods," said the researchers in their pre-print shared paper.

This is probably shocking to TensorFlow users and AI enthusiasts running top-tier neural networks on GPUs and home computers, but training increasingly scaled-up models is becoming too expensive and power-intensive a proposition.

For example, if someone wants to train a major state-of-the-art system, not unlike OpenAI's text generator — GPT-2 — it will cost a small fortune and could also do serious damage to the environment, reports The Next Web.

Performance Benchmarks MIT — The chart shows the required computation, carbon emissions, and economic costs for and of deep learning based on exponential and polynomial projection models. Source: MIT / arXiv

Rising AI costs in computing, finance, environment

In the chart above we see a screenshot from the MIT paper, and it shows how common deep learning systems like ImageNet cost us in multiple ways, including computational, financial, and environmental expenses.

Current trends suggest the approach of a moment when pushing further for computational benchmarks like reaching higher ImageNet-based accuracy will no longer be financially doable — at least in the current frame of mind, said the researchers.

Most Popular

In the world of modern computing, AI is sadly a field of excess that will soon exhaust the feasibility of its platform. With this in mind, we could say machine learning algorithms have been held back since the 1950s — and we've enjoyed an extra jump in performance for the last decade thanks to a handful of smart tricks. One that, despite losing their effect — made the twenty-teens the most exciting time for technology in the history of our species.