Revealing the Dark Secrets of BERT

Olga Kovaleva, Alexey Romanov, Anna Rogers, Anna Rumshisky

    Research output: Conference Article in Proceeding or Book/Report chapterArticle in proceedingsResearchpeer-review


    BERT-based architectures currently give state-of-the-art performance on many NLP tasks, but little is known about the exact mechanisms that contribute to its success. In the current work, we focus on the interpretation of self-attention, which is one of the fundamental underlying components of BERT. Using a subset of GLUE tasks and a set of handcrafted features-of-interest, we propose the methodology and carry out a qualitative and quantitative analysis of the information encoded by the individual BERT's heads. Our findings suggest that there is a limited set of attention patterns that are repeated across different heads, indicating the overall model overparametrization. While different heads consistently use the same attention patterns, they have varying impact on performance across different tasks. We show that manually disabling attention in certain heads leads to a performance improvement over the regular fine-tuned BERT models.
    Original languageEnglish
    Title of host publicationProceedings of EMNLP-IJCNLP)
    Number of pages10
    Place of PublicationHong Kong, China
    PublisherAssociation for Computational Linguistics
    Publication date2019
    Publication statusPublished - 2019


    • BERT-based architectures
    • self-attention mechanisms
    • NLP tasks performance
    • GLUE tasks analysis
    • attention pattern repetition


    Dive into the research topics of 'Revealing the Dark Secrets of BERT'. Together they form a unique fingerprint.

    Cite this