Modern computers are out of their depth when it comes to deep learning and AI, according to recent research from MIT shared on a pre-print website.
Modern computers can't handle perpetual AI scaling
In essence, we've exhausted the computing potential of modern computers, and researchers say we'll soon exhaust economic and environmental ways to keep scaling up deep learning systems.
"Progress along current lines is rapidly becoming economically, technically, and environmentally unsustainable. Thus, continued progress in these applications will require dramatically more computationally-efficient methods, which will either have to come from changes to deep learning or from moving to other machine learning methods," said the researchers in their pre-print shared paper.
This is probably shocking to TensorFlow users and AI enthusiasts running top-tier neural networks on GPUs and home computers, but training increasingly scaled-up models is becoming too expensive and power-intensive a proposition.
For example, if someone wants to train a major state-of-the-art system, not unlike OpenAI's text generator — GPT-2 — it will cost a small fortune and could also do serious damage to the environment, reports The Next Web.
Rising AI costs in computing, finance, environment
In the chart above we see a screenshot from the MIT paper, and it shows how common deep learning systems like ImageNet cost us in multiple ways, including computational, financial, and environmental expenses.
Current trends suggest the approach of a moment when pushing further for computational benchmarks like reaching higher ImageNet-based accuracy will no longer be financially doable — at least in the current frame of mind, said the researchers.
In the world of modern computing, AI is sadly a field of excess that will soon exhaust the feasibility of its platform. With this in mind, we could say machine learning algorithms have been held back since the 1950s — and we've enjoyed an extra jump in performance for the last decade thanks to a handful of smart tricks. One that, despite losing their effect — made the twenty-teens the most exciting time for technology in the history of our species.