Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Training

Computer Science

• Share Information 100%
• Language Modeling 100%
• Natural Language Processing 50%
• Multitask Learning 50%
• Syntactics 50%
• Domain Knowledge 50%
• Interpretability 50%
• Task Performance 50%
• Performance Improvement 50%
• Reasoning Task 50%
• Learning Phase 50%

Keyphrases

• Semantic Tasks 50%
• Full Training 50%
• Learning Phase 50%
• Open-domain 50%
• Critical Learning 50%
• Reasoning Tasks 50%
• Syntactic Knowledge 50%
• Representational Space 50%

Psychology

• Domain Knowledge 100%
• Representational Space 100%
• Performance Task 100%