We study the issue of catastrophic forgetting in the context of neural multimodal approaches to Visual Question Answering (VQA). Motivated by evidence from psycholinguistics, we devise a set of linguistically-informed VQA tasks, which differ by the types of questions involved (Wh-questions and polar questions). We test what impact task difficulty has on continual learning, and whether the order in which a child acquires question types facilitates computational models. Our results show that dramatic forgetting is at play and that task difficulty and order matter. Two well-known current continual learning methods mitigate the problem only to a limiting degree.
Title of host publication
Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics
This page is printed from https://en.itu.dk/research/portalplaceholder?layoutfraction=top&langRef=https://pure.itu.dk/portal/da/journals/spiele-des-sudens(d7df1ea7-5c38-436e-8c2a-bab5a3069f6f)/publications.html?ordering=researchOutputOrderByType&pageSize=50&page=0&descending=false