HyperNTM: Evolving Scalable Neural Turing Machines Through HyperNEAT

Jakob Merrild, Mikkel Angaju Rasmussen, Sebastian Risi

    Publikation: Konference artikel i Proceeding eller bog/rapport kapitelKonferencebidrag i proceedingsForskningpeer review

    Abstract

    Recent developments in memory-augmented neural networks allowed sequential problems requiring long-term memory to be solved, which were intractable for traditional neural networks. However, current approaches still struggle to scale to large memory sizes and sequence lengths. In this paper we show how access to an external memory component can be encoded geometrically through a novel HyperNEAT-based Neural Turing Machine (HyperNTM). The indirect HyperNEAT encoding allows for training on small memory vectors in a bit vector copy task and then applying the knowledge gained from such training to speed up training on larger size memory vectors. Additionally, we demonstrate that in some instances, networks trained to copy nine bit vectors can be scaled to sizes of 1,000 without further training. While the task in this paper is simple, the HyperNTM approach could now allow memory-augmented neural networks to scale to problems requiring large memory vectors and sequence lengths.
    OriginalsprogEngelsk
    TitelInternational Conference on the Applications of Evolutionary Computation : EvoApplications 2018
    Antal sider17
    ForlagSpringer
    Publikationsdato2018
    Sider750-766
    ISBN (Elektronisk)978-3-319-77538-8
    DOI
    StatusUdgivet - 2018
    NavnLecture Notes in Computer Science
    Vol/bind10784
    ISSN0302-9743

    Emneord

    • Memory-augmented neural networks
    • HyperNEAT
    • Neural Turing Machine
    • Scalability
    • Memory vectors

    Fingeraftryk

    Dyk ned i forskningsemnerne om 'HyperNTM: Evolving Scalable Neural Turing Machines Through HyperNEAT'. Sammen danner de et unikt fingeraftryk.

    Citationsformater