Abstract
Despite significant recent advances, artificial agents are still far behind biological
agents in their abilities to adapt to novel and unexpected situations.
This thesis contributes to the field of adaptive artificial agents, taking inspiration
from biology to develop new methods that extend the capabilities of artificial agents controlled by neural networks. Several methods are introduced: (a) An algorithm, named Evolve & Merge, that progressively decreases the number of plasticity rules used to update the synapses of a neural network until the number of rules is orders of magnitude smaller than the number of rules. This is done without diminishing the agent’s performance and without extending the overall training time; (b) A parameterization of neurons in a neural network that makes each neuron a tiny dynamical system.
These neurons are shown to be expressive enough to solve several reinforcement learning (RL) tasks even when no synapses are optimized and only
the parameters of the neurons are evolved in random neural networks; (c) A
meta-learning framework that optimizes a network to provide a reward signal
for an RL agent. The evolved reward signal is shown to enhance the training
stability of the RL agent as well as enable the agent to maintain performance
in novel circumstances through continued optimization with the evolved reward
signal; (d) A demonstration of the minimal requirements for agents to become invariant to permutations of the input elements as well as the size of the input and output vectors; (e) A framework, named Structurally Flexible Adaptive Neural Networks (SFANN) that combines ideas from the earlier contributions of the thesis. SFANNs have a small number of plasticity rules, parameterized dynamic neurons, the ability to learn from rewards, and are flexible in the structure both when it comes to the input and output, and the hidden layers. This framework is put forward as a method that can be optimized in environments of different input and output dimensions to eventually allow a single set of parameters to serve as a general learner across many contexts.
agents in their abilities to adapt to novel and unexpected situations.
This thesis contributes to the field of adaptive artificial agents, taking inspiration
from biology to develop new methods that extend the capabilities of artificial agents controlled by neural networks. Several methods are introduced: (a) An algorithm, named Evolve & Merge, that progressively decreases the number of plasticity rules used to update the synapses of a neural network until the number of rules is orders of magnitude smaller than the number of rules. This is done without diminishing the agent’s performance and without extending the overall training time; (b) A parameterization of neurons in a neural network that makes each neuron a tiny dynamical system.
These neurons are shown to be expressive enough to solve several reinforcement learning (RL) tasks even when no synapses are optimized and only
the parameters of the neurons are evolved in random neural networks; (c) A
meta-learning framework that optimizes a network to provide a reward signal
for an RL agent. The evolved reward signal is shown to enhance the training
stability of the RL agent as well as enable the agent to maintain performance
in novel circumstances through continued optimization with the evolved reward
signal; (d) A demonstration of the minimal requirements for agents to become invariant to permutations of the input elements as well as the size of the input and output vectors; (e) A framework, named Structurally Flexible Adaptive Neural Networks (SFANN) that combines ideas from the earlier contributions of the thesis. SFANNs have a small number of plasticity rules, parameterized dynamic neurons, the ability to learn from rewards, and are flexible in the structure both when it comes to the input and output, and the hidden layers. This framework is put forward as a method that can be optimized in environments of different input and output dimensions to eventually allow a single set of parameters to serve as a general learner across many contexts.
| Original language | English |
|---|
| Place of Publication | Copenhagen |
|---|---|
| Publisher | IT University of Copenhagen |
| Number of pages | 225 |
| ISBN (Print) | 978-87-7949-408-4 |
| Publication status | Published - 23 Oct 2023 |
| Series | ITU-DS |
|---|---|
| Number | 212 |
| ISSN | 1602-3536 |