Evolving Neural Turing Machines for Reward-based Learning

Rasmus Boll Greve, Emil Juul Jacobsen, Sebastian Risi, Rasmus Boll Greve

Research output: Conference Article in Proceeding or Book/Report chapterArticle in proceedingsResearchpeer-review


An unsolved problem in neuroevolution (NE) is to evolve artificial neural networks (ANN) that can store and use information to change their behavior online. While plastic neural networks have shown promise in this context, they have difficulties retaining information over longer periods of time and integrating new information without losing previously acquired skills. Here we build on recent work by Graves et al. [5] who extended the capabilities of an ANN by combining it with an external memory bank trained through gradient descent. In this paper, we introduce an evolvable version of their Neural Turing Machine (NTM) and show that such an approach greatly simplifies the neural model, generalizes better, and does not require accessing the entire memory content at each time-step. The Evolvable Neural Turing Machine (ENTM) is able to solve a simple copy tasks and for the first time, the continuous version of the double T-Maze, a complex reinforcement-like learning problem. In the T-Maze learning task the agent uses the memory bank to display adaptive behavior that normally requires a plastic ANN, thereby suggesting a complementary and effective mechanism for adaptive behavior in NE.
Original languageUndefined/Unknown
Title of host publicationProceedings of the Genetic and Evolutionary Computation Conference 2016
Number of pages8
Place of PublicationNew York, NY, USA
PublisherAssociation for Computing Machinery
Publication date2016
ISBN (Print)978-1-4503-4206-3
Publication statusPublished - 2016

Cite this