DeepMind Gato and the Long, Uncertain Road to Artificial General Intelligence – The Wire Science

Photo: Possessed Photography/Unsplash


  • Last month, DeepMind, a subsidiary of tech giant Alphabet, rocked Silicon Valley when it announced Gato, arguably the most versatile AI model in existence.
  • To some computer experts, it’s proof that the industry is on the cusp of reaching a much-anticipated, much-hyped milestone: Artificial General Intelligence (AGI).
  • This would be huge for humanity. Think of everything you could achieve if you had a machine that could be physically adapted for any purpose.
  • But a host of experts and scientists have argued that something fundamental is missing from the grand plans to build Gato-like AI into full-fledged AGI machines.

Last month, DeepMind, a subsidiary of tech giant Alphabet, rocked Silicon Valley when it announced gato, arguably the most versatile artificial intelligence model out there. Billed as a “generalist agent”, Gato can perform over 600 different tasks. It can control a robot, caption images, identify objects in images, and more. It is probably the most advanced AI system in the world that is not dedicated to a single function. And for some computer experts, it’s proof that the industry is on the cusp of hitting a long-awaited, much-anticipated milestone: Artificial General Intelligence.

Unlike regular AI, artificial general intelligence (AGI) would not require massive amounts of data to learn a task. While ordinary artificial intelligence must be pre-trained or programmed to solve a specific set of problems, a general intelligence can learn through intuition and experience.

In theory, an AGI could learn everything a human being can do, if given the same access to information. If you place an AGI on a chip and then put that chip in a robot, the robot can learn to play tennis the same way you or I can: by swinging a racket and getting a feel for the game. That does not necessarily mean that the robot is conscious or capable of cognition. It wouldn’t have any thoughts or emotions, it would just be really good at learning how to do new tasks without human help.

This would be huge for humanity. Think of all you could achieve if you had a machine with the intellectual capacity of a human and the loyalty of a trusted canine companion – a machine that could be physically adapted to any purpose. That is AGI’s promise. To be C-3PO without the emotions, Lieutenant Commander Data without the curiosity, and Rosey the robot without the personality. In the hands of the right developers, it could embody the idea of: human-centered AI

But how close is AGI’s dream really? And does Gato really bring us closer?

For a particular group of scientists and developers (I call this group the “Scale-Uber-Everythingcrowd, using a term coined by world-renowned AI expert Gary Marcus) Gato and similar systems based on deep learning transformer models have already given us the blueprint for building AGI. Essentially, these transformers use massive databases and billions or trillions of adjustable parameters to predict what will happen in a sequence.

The Scaling-Uber-Everything crowd, which includes household names such as Ilya Sutskever of OpenAI and Alex Dimakis of the University of Texas at Austin, believe that transformers will inevitably lead to AGI; all that’s left is to make them bigger and faster. Like Nando de Freitas, a member of the team that created Gato, recently tweeted: “It’s all about scale now! The game is over! It’s about making these models bigger, more secure, more compute efficient, faster sampling, smarter memory…” De Freitas and the company understand that they need to create new algorithms and architectures to support this growth, but they also seem to believe that an AGI will come naturally as we continue to make models like Gato bigger.

Call me old-fashioned, but when a developer tells me their plan is to wait for an AGI to magically emerge from the mass of big data like a mudfish from the primordial soup, I tend to think they’re taking a few steps to skip. Apparently I’m not the only one. A host of experts and scientists, including Marcus, have argued that something fundamental is missing from grandiose plans to build Gato-like AI into full-fledged, generally intelligent machines.

I recently explained my thinking in a trilogy from essays in front of The next web‘s Neural vertical, where I am an editor. In short, an important principle of AGI is that it must be able to obtain its own data. But deep learning models, such as transformer AIs, are little more than machines designed to make inferences about the databases already delivered to them. They are librarians and as such they are only as good as their training libraries.

A general intelligence could theoretically figure things out even if it had a small database. It would be intuitive for the methodology to accomplish its task based on nothing more than its ability to choose what external data was and was not important, like a human deciding where to focus his attention.

Gato is cool and nothing beats that. But essentially it’s a smart package that arguably presents the illusion of a general purpose AI through expert use of big data. For example, the massive database likely contains datasets built on the full content of websites such as Reddit and Wikipedia. It’s amazing that people have been able to do so much with simple algorithms by forcing them to parse more data.

In fact, Gato is such an impressive way to fake general intelligence that I wonder if we might be barking at the wrong tree. Many of the tasks Gato is capable of today included: ever believed to be something only an AGI could do. It feels like the more we achieve with regular AI, the harder it seems to build a general agent.

For those reasons, I am skeptical that only deep learning is the way to AGI. I think we need more than bigger databases and extra parameters to tweak. We need a whole new conceptual approach to machine learning.

I think humanity will eventually succeed in the quest to build AGI. My best guess is that we’ll be knocking on AGI’s door sometime around the early to mid 2100s, and if we do, we’ll find that it looks very different from what DeepMind’s scientists envision.

But the beauty of science is that you have to show your work, and right now DeepMind is doing just that. It has every chance to prove me and the other naysayers wrong.

I really hope it works.

Tristan Greene is a futurist who believes in the power of human-centric technology. He is currently the editor of Neural, the futuristic vertical of The Next Web.

This article was first published by dark

Leave a Comment

Your email address will not be published.