New deep learning models: Fewer neurons, more intelligence

Credit: Pixabay/CC0 Public Domain


An international research team from TU Wien (Vienna), IST Austria and MIT (USA) has developed a new system based on the brains of tiny animals, such as threadworms. This novel AI-system can control a vehicle with just a few artificial neurons. The team says that system has decisive advantages over previous models: It copes much better with noisy input, and, because of its simplicity, its mode of operation can be explained in detail. It does not have to be regarded as a complex "black box", but it can be understood by humans. This new deep learning model has now been published in the journal Nature Machine Intelligence.

Learning from nature


"For years, we have been investigating what we can learn from nature to improve deep learning," says Prof. Radu Grosu, head of the research group "Cyber-Physical Systems" at TU Wien. "The nematode C. elegans, for example, lives its life with an amazingly small number of neurons, and still shows interesting behavioral patterns. This is due to the efficient and harmonious way the nematode's nervous system processes information."

"Nature shows us that there is still lots of room for improvement," says Prof. Daniela Rus, director of MIT's Computer Science and Artificial Intelligence Laboratory (CSAIL). "Therefore, our goal was to massively reduce complexity and enhance interpretability of neural network models."

"Inspired by nature, we developed new mathematical models of neurons and synapses," says Prof. Thomas Henzinger, president of IST Austria.

"The processing of the signals within the individual follows different mathematical principles than previous deep learning models," says Dr. Ramin Hasani, postdoctoral associate at the Institute of Computer Engineering, TU Wien and MIT CSAIL. "Also, our networks are highly sparse—this means that not every cell is connected to every other cell. This also makes the network simpler."

Autonomous Lane Keeping


"Today, deep learning models with many millions of parameters are often used for learning complex tasks such as autonomous driving," says Mathias Lechner, TU Wien alumnus and Ph.D. student at IST Austria. "However, our new approach enables us to reduce the size of the networks by two orders of magnitude. Our systems only use 75,000 trainable parameters."

Alexander Amini, Ph.D. student at MIT CSAIL explains that the new system consists of two parts: The camera input is first processed by a so-called convolutional , which only perceives the visual data to extract structural features from incoming pixels. This network decides which parts of the camera image are interesting and important, and then passes signals to the crucial part of the network—a "control system" that then steers the vehicle.



Causality and Interpretability

The new deep learning model was tested on a real autonomous vehicle. "Our model allows us to investigate what the network focuses its attention on while driving. Our networks focus on very specific parts of the camera picture: The curbside and the horizon. This behavior is highly desirable, and it is unique among artificial intelligence systems," says Ramin Hasani. "Moreover, we saw that the role of every single cell at any driving decision can be identified. We can understand the function of and their behavior. Achieving this degree of interpretability is impossible for larger deep learning models."


"To test how robust NCPs are compared to previous deep models, we perturbed the input images and evaluated how well the agents can deal with the noise," says Mathias Lechner. "While this became an insurmountable problem for other deep neural networks, our NCPs demonstrated strong resistance to input artifacts. This attribute is a direct consequence of the novel neural model and the architecture."


Explore further

Artificial intelligence—parking a car with only 12 neurons

More information: Mathias Lechner et al, Neural circuit policies enabling auditable autonomy, Nature Machine Intelligence (2020). DOI: 10.1038/s42256-020-00237-3

Code Repository: github.com/mlech26l/keras-ncp

Journal information: Nature Machine Intelligence

Citation: New deep learning models: Fewer neurons, more intelligence (2020, October 13) retrieved 27 September 2021 from https://techxplore.com/news/2020-10-deep-neurons-intelligence.html

Feedback to editors