Subspace Chronicles: How Linguistic Information Emerges, Shifts and Interacts during Language Model Training

Keyphrases

• Training Model 100%
• Information Sharing 100%
• Language Model 100%
• Linguistic Information 100%
• Natural Language Processing 50%
• Multi-task Learning 50%
• Syntax-semantics 50%
• Information Theory 50%
• Performance Improvement 50%
• Task Performance 50%
• Language Modeling 50%
• Training Step 50%
• Task Similarity 50%
• Cross-task 50%
• Model Interpretability 50%
• Semantic Tasks 50%
• Full Training 50%
• Learning Phase 50%
• Open-domain 50%
• Critical Learning 50%
• Reasoning Tasks 50%
• Syntactic Knowledge 50%
• Representational Space 50%

Computer Science

• Share Information 100%
• Language Modeling 100%
• Natural Language Processing 50%
• Multitask Learning 50%
• Syntactics 50%
• Domain Knowledge 50%
• Interpretability 50%
• Task Performance 50%
• Performance Improvement 50%
• Reasoning Task 50%
• Learning Phase 50%

Psychology

• Domain Knowledge 100%
• Representational Space 100%