Investigating Different Syntactic Context Types and Context Representations for Learning Word Embeddings

Bofang Li; Tao Liu; Zhe Zhao; Buzhou Tang; Aleksandr Drozd; Anna Rogers; Xiaoyong Du

Investigating Different Syntactic Context Types and Context Representations for Learning Word Embeddings

Bofang Li, Tao Liu, Zhe Zhao, Buzhou Tang, Aleksandr Drozd, Anna Rogers, Xiaoyong Du

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

Abstract

The number of word embedding models is growing every year. Most of them are based on the co-occurrence information of words and their contexts. However, it is still an open question what is the best definition of context. We provide a systematical investigation of 4 different syntactic context types and context representations for learning word embeddings. Comprehensive experiments are conducted to evaluate their effectiveness on 6 extrinsic and intrinsic tasks. We hope that this paper, along with the published code, would be helpful for choosing the best context type and representation for a given task.

Original language	English
Title of host publication	Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing
Number of pages	11
Place of Publication	Copenhagen, Denmark, September 71, 2017
Publication date	2017
Pages	2411-2421
Publication status	Published - 2017

Keywords

Word Embedding Models
Syntactic Context Types
Context Representations
Co-occurrence Information
Extrinsic and Intrinsic Evaluation

Access to Document

http://aclweb.org/anthology/D17-1257

Cite this

@inproceedings{fe14e6a2e70f4b28ac394d5e1099334d,

title = "Investigating Different Syntactic Context Types and Context Representations for Learning Word Embeddings",

abstract = "The number of word embedding models is growing every year. Most of them are based on the co-occurrence information of words and their contexts. However, it is still an open question what is the best definition of context. We provide a systematical investigation of 4 different syntactic context types and context representations for learning word embeddings. Comprehensive experiments are conducted to evaluate their effectiveness on 6 extrinsic and intrinsic tasks. We hope that this paper, along with the published code, would be helpful for choosing the best context type and representation for a given task.",

keywords = "Word Embedding Models, Syntactic Context Types, Context Representations, Co-occurrence Information, Extrinsic and Intrinsic Evaluation, Word Embedding Models, Syntactic Context Types, Context Representations, Co-occurrence Information, Extrinsic and Intrinsic Evaluation",

author = "Bofang Li and Tao Liu and Zhe Zhao and Buzhou Tang and Aleksandr Drozd and Anna Rogers and Xiaoyong Du",

year = "2017",

language = "English",

pages = "2411--2421",

booktitle = "Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing",

}

Investigating Different Syntactic Context Types and Context Representations for Learning Word Embeddings. / Li, Bofang; Liu, Tao; Zhao, Zhe et al.
Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing. Copenhagen, Denmark, September 71, 2017, 2017. p. 2411-2421.

Research output: Conference Article in Proceeding or Book/Report chapter › Article in proceedings › Research › peer-review

TY - GEN

T1 - Investigating Different Syntactic Context Types and Context Representations for Learning Word Embeddings

AU - Li, Bofang

AU - Liu, Tao

AU - Zhao, Zhe

AU - Tang, Buzhou

AU - Drozd, Aleksandr

AU - Rogers, Anna

AU - Du, Xiaoyong

PY - 2017

Y1 - 2017

N2 - The number of word embedding models is growing every year. Most of them are based on the co-occurrence information of words and their contexts. However, it is still an open question what is the best definition of context. We provide a systematical investigation of 4 different syntactic context types and context representations for learning word embeddings. Comprehensive experiments are conducted to evaluate their effectiveness on 6 extrinsic and intrinsic tasks. We hope that this paper, along with the published code, would be helpful for choosing the best context type and representation for a given task.

AB - The number of word embedding models is growing every year. Most of them are based on the co-occurrence information of words and their contexts. However, it is still an open question what is the best definition of context. We provide a systematical investigation of 4 different syntactic context types and context representations for learning word embeddings. Comprehensive experiments are conducted to evaluate their effectiveness on 6 extrinsic and intrinsic tasks. We hope that this paper, along with the published code, would be helpful for choosing the best context type and representation for a given task.

KW - Word Embedding Models

KW - Syntactic Context Types

KW - Context Representations

KW - Co-occurrence Information

KW - Extrinsic and Intrinsic Evaluation

KW - Word Embedding Models

KW - Syntactic Context Types

KW - Context Representations

KW - Co-occurrence Information

KW - Extrinsic and Intrinsic Evaluation

M3 - Article in proceedings

SP - 2411

EP - 2421

BT - Proceedings of the 2017 Conference on Empirical Methods in Natural Language Processing

CY - Copenhagen, Denmark, September 71, 2017

ER -

Investigating Different Syntactic Context Types and Context Representations for Learning Word Embeddings

Abstract

Keywords

Access to Document

Fingerprint

Cite this