References & Citations
Computer Science > Computation and Language
Title: HittER: Hierarchical Transformers for Knowledge Graph Embeddings
(Submitted on 28 Aug 2020 (v1), last revised 6 Oct 2021 (this version, v2))
Abstract: This paper examines the challenging problem of learning representations of entities and relations in a complex multi-relational knowledge graph. We propose HittER, a Hierarchical Transformer model to jointly learn Entity-relation composition and Relational contextualization based on a source entity's neighborhood. Our proposed model consists of two different Transformer blocks: the bottom block extracts features of each entity-relation pair in the local neighborhood of the source entity and the top block aggregates the relational information from outputs of the bottom block. We further design a masked entity prediction task to balance information from the relational context and the source entity itself. Experimental results show that HittER achieves new state-of-the-art results on multiple link prediction datasets. We additionally propose a simple approach to integrate HittER into BERT and demonstrate its effectiveness on two Freebase factoid question answering datasets.
Submission history
From: Sanxing Chen [view email][v1] Fri, 28 Aug 2020 18:58:15 GMT (4937kb,D)
[v2] Wed, 6 Oct 2021 04:52:07 GMT (6168kb,D)
Link back to: arXiv, form interface, contact.