A "liquid" neural network modeled after the brains of tiny creatures was unveiled by MIT researchers last year. For practical, safety-critical tasks like driving and flying, we're talking about a class of robust, adaptive machine learning models that can learn on the job and adapt to changing conditions. The adaptability of these “liquid” neural networks strengthens the communication of our interconnected world, meaning better decision-making for a variety of time-series data-intensive tasks such as heart and brain monitoring, weather forecasting and stock pricing.
However, as the number of neurons and synapses in these models increases, they become computationally expensive and require cumbersome computer programs to solve the complex math at their core. And just like with many physical phenomena, solving all this arithmetic gets harder with size, which requires calculating lots of small steps to arrive at a solution.
The same team of scientists found a way to reduce this bottleneck by solving the differential equation underlying the connection of two neurons via synapses to reveal a new class of fast and efficient AI systems. While these modes are much faster and scalable than liquid neural networks, they share the same flexible, causal, robust and explainable features.
As a result, because they are small and flexible even after training – unlike many traditional models that are fixed – this type of neural network can be used for any task that involves gaining insights into data over time.
“Closed form continuous time” (CfC) neural network models outperformed their state-of-the-art counterparts in a variety of tasks including event-based sequential image processing, modeling the physical dynamics of a simulated walking robot, and human activity recognition from motion sensors. For example, the new models were 8.000 times faster on a sample of 220 patients for a medical prediction task.
According to MIT Professor Daniela Rus, director of the Computer Science and Artificial Intelligence Laboratory (CSAIL) and senior author of the new paper, “New machine learning models, which we call 'CfCs', are focused on numerical integration, replacing the differential equation that describes the neuron's computation with a closed-form approach. retains the beautiful properties of liquid meshes without the need for “CfC models are efficient, causal, condensed and explainable to train and predict. They open the door to reliable machine learning for applications essential for security.”
We can calculate the changing state of the world or a phenomenon over time using differential equations, but we can only do this step by step over time. The team rummaged through their bags of mathematical tricks to find the perfect solution.
A "closed form" solution that models the entire description of an entire system in a single computational step to model natural phenomena over time and understand past and present behavior, such as recognizing human activities or the path followed by a robot.
Their model allows this equation to be calculated at any point in the past or future. Not only that, the calculation is much faster since the differential equation does not need to be solved step-by-step.
Imagine an end-to-end neural network that uses a camera built into a car to provide driving input. The network is trained to produce outputs such as the steering angle of the car. In 2020, the team managed to create a car that could be driven by 19 neurons and a small sensing module using 19-node fluid neural networks. Each node in the system is described by a differential equation. Since the closed form solution is a good approximation of the actual dynamics of the system, changing it in this mesh will result in exactly the behavior you are looking for. As a result, they can solve the problem with even fewer neurons, making the process faster and less computationally expensive.
Source and Further Reading: techxplore.com/news/2022-11-brain-dynamics-flexible-machine-learning.html
Günceleme: 21/11/2022 14:03
Be the first to comment