Adaptive Learning of Linguistic Hierarchy in a Multiple Timescale Recurrent Neural Network

Stefan Heinrich, Cornelius Weber, Stefan Wermter

Research output: Conference Article in Proceeding or Book/Report chapterArticle in proceedingsResearchpeer-review

Abstract

Recent research has revealed that hierarchical linguistic structures can emerge in a recurrent neural network with a sufficient number of delayed context layers. As a representative of this type of network the Multiple Timescale Recurrent Neural Network (MTRNN) has been proposed for recognising and generating known as well as unknown linguistic utterances. However the training of utterances performed in other approaches demands a high training effort. In this paper we propose a robust mechanism for adaptive learning rates and internal states to speed up the training process substantially. In addition we compare the generalisation of the network for the adaptive mechanism as well as the standard fixed learning rates finding at least equal capabilities.
Original languageEnglish
Title of host publicationProceedings of the 22nd International Conference on Artificial Neural Networks (ICANN2012)
EditorsAlessandro E.P. Villa, Włodzisław Duch, Péter Érdi, Francesco Masulli, Günther Palm
Number of pages8
Volume7552
PublisherSpringer
Publication date1 Sept 2012
Pages555-562
DOIs
Publication statusPublished - 1 Sept 2012
Externally publishedYes
SeriesLecture Notes in Computer Science

Keywords

  • Adaptive Learning
  • MTRNN
  • Language

Fingerprint

Dive into the research topics of 'Adaptive Learning of Linguistic Hierarchy in a Multiple Timescale Recurrent Neural Network'. Together they form a unique fingerprint.

Cite this