Abstract
There are some important problems in the evaluation of word embeddings using standard word analogy tests. In particular, in virtue of the assumptions made by systems generating the embeddings, these remain tests over randomness. We show that even supposing there were such word analogy regularities that
should be detected in the word embeddings obtained via unsupervised means, standard word analogy test implementation practices provide distorted or contrived results. We raise concerns regarding the use of Principal Component Analysis to 2 or 3 dimensions as a provision of visual evidence for the existence of word analogy relations in embeddings. Finally, we propose some solutions to these problems.
should be detected in the word embeddings obtained via unsupervised means, standard word analogy test implementation practices provide distorted or contrived results. We raise concerns regarding the use of Principal Component Analysis to 2 or 3 dimensions as a provision of visual evidence for the existence of word analogy relations in embeddings. Finally, we propose some solutions to these problems.
| Original language | English |
|---|---|
| Title of host publication | Proceedings of the 2018 Conference of the North American Chapter of the Association for Computational Linguistics: Human Language Technologies : Volume 2 (Short Papers) |
| Number of pages | 5 |
| Publisher | Association for Computational Linguistics |
| Publication date | 1 Jun 2018 |
| Pages | 242–246 |
| ISBN (Electronic) | 978-1-948087-29-2 |
| DOIs | |
| Publication status | Published - 1 Jun 2018 |
Keywords
- Word Embeddings
- Word Analogy Tests
- Randomness in Embeddings
- Principal Component Analysis
- Evaluation Methodologies