Interactive Language Understanding with Multiple Timescale Recurrent Neural Networks

Stefan Heinrich; Stefan Wermter

doi:10.1007/978-3-319-11179-7_25

Interactive Language Understanding with Multiple Timescale Recurrent Neural Networks

Publikation: Konference artikel i Proceeding eller bog/rapport kapitel › Konferencebidrag i proceedings › Forskning › peer review

Abstract

Natural language processing in the human brain is complex and dynamic. Models for understanding, how the brain’s architecture acquires language, need to take into account the temporal dynamics of verbal utterances as well as of action and visual embodied perception. We propose an architecture based on three Multiple Timescale Recurrent Neural Networks (MTRNNs) interlinked in a cell assembly that learns verbal utterances grounded in dynamic proprioceptive and visual information. Results show that the architecture is able to describe novel dynamic actions with correct novel utterances, and they also indicate that multi-modal integration allows for a disambiguation of concepts.

Originalsprog	Engelsk
Titel	Proceedings of the 24th International Conference on Artificial Neural Networks (ICANN2014)
Redaktører	Stefan Wermter, Cornelius Weber, Włodzisław Duch, Timo Honkela, Petia Koprinkova-Hristova, Sven Magg, Günther Palm, Alessandro E.P. Villa
Antal sider	8
Vol/bind	8681
Forlag	Springer International Publishing, Switzerland
Publikationsdato	1 sep. 2014
Sider	193-200
ISBN (Trykt)	978-3-319-11178-0
DOI	https://doi.org/10.1007/978-3-319-11179-7_25
Status	Udgivet - 1 sep. 2014
Udgivet eksternt	Ja

Navn	Lecture Notes in Computer Science

Adgang til dokumentet

10.1007/978-3-319-11179-7_25

https://stefanheinrich.net/files/2014_Heinrich_ICANN.pdfLicens: Ikke-specificeret

Citationsformater

Heinrich, S., & Wermter, S. (2014). Interactive Language Understanding with Multiple Timescale Recurrent Neural Networks. I S. Wermter, C. Weber, W. Duch, T. Honkela, P. Koprinkova-Hristova, S. Magg, G. Palm, & A. E. P. Villa (red.), Proceedings of the 24th International Conference on Artificial Neural Networks (ICANN2014) (Bind 8681, s. 193-200). Springer International Publishing, Switzerland. https://doi.org/10.1007/978-3-319-11179-7_25

Heinrich, Stefan ; Wermter, Stefan. / Interactive Language Understanding with Multiple Timescale Recurrent Neural Networks. Proceedings of the 24th International Conference on Artificial Neural Networks (ICANN2014). red. / Stefan Wermter ; Cornelius Weber ; Włodzisław Duch ; Timo Honkela ; Petia Koprinkova-Hristova ; Sven Magg ; Günther Palm ; Alessandro E.P. Villa. Bind 8681 Springer International Publishing, Switzerland, 2014. s. 193-200 (Lecture Notes in Computer Science).

@inproceedings{4f76212e3d84441abc63cbc576c03d0d,

title = "Interactive Language Understanding with Multiple Timescale Recurrent Neural Networks",

abstract = "Natural language processing in the human brain is complex and dynamic. Models for understanding, how the brain{\textquoteright}s architecture acquires language, need to take into account the temporal dynamics of verbal utterances as well as of action and visual embodied perception. We propose an architecture based on three Multiple Timescale Recurrent Neural Networks (MTRNNs) interlinked in a cell assembly that learns verbal utterances grounded in dynamic proprioceptive and visual information. Results show that the architecture is able to describe novel dynamic actions with correct novel utterances, and they also indicate that multi-modal integration allows for a disambiguation of concepts.",

author = "Stefan Heinrich and Stefan Wermter",

year = "2014",

month = sep,

day = "1",

doi = "10.1007/978-3-319-11179-7_25",

language = "English",

isbn = "978-3-319-11178-0",

volume = "8681",

series = "Lecture Notes in Computer Science",

pages = "193--200",

editor = "Stefan Wermter and Cornelius Weber and W{\l}odzis{\l}aw Duch and Timo Honkela and Petia Koprinkova-Hristova and Sven Magg and G{\"u}nther Palm and Villa, {Alessandro E.P.}",

booktitle = "Proceedings of the 24th International Conference on Artificial Neural Networks (ICANN2014)",

publisher = "Springer International Publishing, Switzerland",

}

Heinrich, S & Wermter, S 2014, Interactive Language Understanding with Multiple Timescale Recurrent Neural Networks. i S Wermter, C Weber, W Duch, T Honkela, P Koprinkova-Hristova, S Magg, G Palm & AEP Villa (red), Proceedings of the 24th International Conference on Artificial Neural Networks (ICANN2014). bind 8681, Springer International Publishing, Switzerland, Lecture Notes in Computer Science, s. 193-200. https://doi.org/10.1007/978-3-319-11179-7_25

Interactive Language Understanding with Multiple Timescale Recurrent Neural Networks. / Heinrich, Stefan; Wermter, Stefan.
Proceedings of the 24th International Conference on Artificial Neural Networks (ICANN2014). red. / Stefan Wermter; Cornelius Weber; Włodzisław Duch; Timo Honkela; Petia Koprinkova-Hristova; Sven Magg; Günther Palm; Alessandro E.P. Villa. Bind 8681 Springer International Publishing, Switzerland, 2014. s. 193-200 (Lecture Notes in Computer Science).

Publikation: Konference artikel i Proceeding eller bog/rapport kapitel › Konferencebidrag i proceedings › Forskning › peer review

TY - GEN

T1 - Interactive Language Understanding with Multiple Timescale Recurrent Neural Networks

AU - Heinrich, Stefan

AU - Wermter, Stefan

PY - 2014/9/1

Y1 - 2014/9/1

N2 - Natural language processing in the human brain is complex and dynamic. Models for understanding, how the brain’s architecture acquires language, need to take into account the temporal dynamics of verbal utterances as well as of action and visual embodied perception. We propose an architecture based on three Multiple Timescale Recurrent Neural Networks (MTRNNs) interlinked in a cell assembly that learns verbal utterances grounded in dynamic proprioceptive and visual information. Results show that the architecture is able to describe novel dynamic actions with correct novel utterances, and they also indicate that multi-modal integration allows for a disambiguation of concepts.

AB - Natural language processing in the human brain is complex and dynamic. Models for understanding, how the brain’s architecture acquires language, need to take into account the temporal dynamics of verbal utterances as well as of action and visual embodied perception. We propose an architecture based on three Multiple Timescale Recurrent Neural Networks (MTRNNs) interlinked in a cell assembly that learns verbal utterances grounded in dynamic proprioceptive and visual information. Results show that the architecture is able to describe novel dynamic actions with correct novel utterances, and they also indicate that multi-modal integration allows for a disambiguation of concepts.

U2 - 10.1007/978-3-319-11179-7_25

DO - 10.1007/978-3-319-11179-7_25

M3 - Article in proceedings

SN - 978-3-319-11178-0

VL - 8681

T3 - Lecture Notes in Computer Science

SP - 193

EP - 200

BT - Proceedings of the 24th International Conference on Artificial Neural Networks (ICANN2014)

A2 - Wermter, Stefan

A2 - Weber, Cornelius

A2 - Duch, Włodzisław

A2 - Honkela, Timo

A2 - Koprinkova-Hristova, Petia

A2 - Magg, Sven

A2 - Palm, Günther

A2 - Villa, Alessandro E.P.

PB - Springer International Publishing, Switzerland

ER -

Heinrich S, Wermter S. Interactive Language Understanding with Multiple Timescale Recurrent Neural Networks. I Wermter S, Weber C, Duch W, Honkela T, Koprinkova-Hristova P, Magg S, Palm G, Villa AEP, red., Proceedings of the 24th International Conference on Artificial Neural Networks (ICANN2014). Bind 8681. Springer International Publishing, Switzerland. 2014. s. 193-200. (Lecture Notes in Computer Science). doi: 10.1007/978-3-319-11179-7_25

Interactive Language Understanding with Multiple Timescale Recurrent Neural Networks

Abstract

Adgang til dokumentet

Fingeraftryk

Citationsformater