Deep Neural Networks Don’t Lead Us Towards AGI
Machine learning techniques continue to evolve with increased efficiency for recognition problems. But, they still lack the critical element of intelligence, so we remain a long way from attaining AGI.
By Thuwarakesh Murallie, Data Scientist at Stax Inc.
Photo by Kindel Media from Pexels.
I was stunned by projects such as GitHub Copilot, Deepfake, and AI bots playing Alpha Go with complex strategies even professional players couldn’t understand.
I thought we were almost there to reach artificial general intelligence (AGI). I thought it was the great leap of AGI.
But I was wrong.
Today’s neural network lacks in many ways to human brains. Machine learning (including deep learning) can solve a particular recognition problem. Yet, intelligence is more of a generative problem.
We need to understand the actual problems of AI before trying to solve them with biologically inspired algorithms, like deep neural networks.
Why is machine learning not AI?
Many works of literature claim that machine learning is a subset of Artificial intelligence.
I’m okay with this in a general sense. But I trust that machine learning is only an approach of AI, not AI itself.
To understand this better, we must think about how children learn.
The way humans and animals learn is by creating a representation of the real world. Further learning occurs by improving this representation by merging other such virtual worlds.
It helps imagine things that do not exist yet. Problem-solving, in reality, is generating ideas from this internal representation and matching them with the real world.
This is how children learn. Because of this, they don’t need to necessarily see cats and dogs before they start to recognize them. Their parents and surroundings play a role in nurturing them to refine the internal representation of the world. Thus they generate ideas about dogs and cats, then match them with the natural world to classify when they see one.
Machine Learning (ML) algorithms don’t learn how to do things they weren’t programmed to do or be creative in any way. They follow the steps we taught them through data set examples.
Because of this, ML models have to start from scratch every time they’re exposed to a new type of data.
Even though deep neural networks have been taking over some pattern recognition tasks such as object detection, speech recognition, or translation — they lack generative capabilities.
In generative tasks, the model has to create something from nothing or randomness.
Related: How to Evaluate if Deep Learning Is Right For You?
What is machine learning if it’s not AI?
Machine learning algorithms are complex approximations of specific problems. They solve the problem by optimizing rather than generating ideas.
They need to see enough examples of the problem before they can recognize it.
This is why deep neural networks are limited in many ways. For example, they require vast quantities of data and labeled training samples. They don’t scale very well with complex tasks or problems with large classes/categories.
These limitations prevent them from solving problems that require reasoning, creativity, and abstract thinking — like those needed for artificial general intelligence.
In reality, if you know how to ride a bike, you should learn to drive a car with some degree of familiarity. But transferring knowledge from one machine learning model to a different one isn’t that straightforward.
Isn’t transfer learning a means of transferring knowledge?
One helpful technique researchers use is called transfer learning. Transfer learning allows us to use a deep learning model trained on one task as a starting point for a similar job.
Related: Transfer Learning: The Highest Leverage Deep Learning Skill You Can Learn
But even transfer learning is far from general knowledge transferability. In transfer learning, you need to comply with many requirements, such as maintaining a similar input structure.
Further, transfer learning is not a solution for the non-generative nature of machine learning models. It simply helps the neural network to converge much quickly for an adjustment problem.
Transfer learnings benefits include reduced training time and cost. But it’s still a recognition problem.
Google knows it.
Google knows that deep neural networks or machine learning, in general, are not capable of artificial general intelligence yet. They understand that a trained model is a weak form of representing real-world knowledge.
“Too often, machine learning systems overspecialize at individual tasks, when they could excel at many. “— Jeff Dean, Google Senior Fellow and SVP, Google Research
Google aims to tackle the problem with what they call the “Pathway.”
There hasn’t been much-published information on how Pathway works. But according to Jeff Dean’s blog post, Google Pathway is about to solve three of the common challenges towards AGI.
Pathway enables us to use a single model to solve thousands of different problems. This is different from conventional machine learning models as they are trained to solve a specific problem.
Also, Pathway uses multiple senses. Conventional models use inputs from a single sense. For example, computer vision problems take images, and speech recognition takes audio signals. The Pathway is expected to have them both and many others at once.
Further, Pathway promises to move model execution from dense mode to sparse. It means, to solve a problem, you don’t have to activate the entire neural network but a specific path in isolation.
In summary
This is not to say that machine learning doesn’t have its place in the field of AI.
It simply means we shouldn’t mistake ML for AGI or assume it’s leading us towards achieving something akin to human intelligence and general problem-solving capabilities.
Deep neural networks (and ML models in general) are suitable for recognition problems. But they don’t create a representation of the world which is essential to solving more general issues.
It isn’t clear how Google will solve these challenges towards AGI with Pathway yet. We’ll have to wait for some information on that front before we can comment further.
What are your thoughts? Do you think machine learning will lead us towards AGI?
Are you excited about the Pathway model and what it promises to bring in terms of solving general AI problems? Share your thoughts, ideas, opinions on this topic. I’d love to hear them all!
Bio: Thuwarakesh Murallie (@Thuwarakesh) is a data scientist at Stax Inc. and a top writer for artificial intelligence on Medium.
Related: