Knowledge graph embedding

[1][2][3] Leveraging their embedded representation, knowledge graphs (KGs) can be used for various applications such as link prediction, triple classification, entity recognition, clustering, and relation extraction.

[6] However, nowadays, people have to deal with the sparsity of data and the computational inefficiency to use them in a real-world application.

[3][7] The embedding of a knowledge graph is a function that translates each entity and each relation into a vector of a given dimension

[7] Usually, the stop condition depends on the overfitting of the training set.

[7] At the end, the learned embeddings should have extracted semantic meaning from the training triples and should correctly predict unseen true facts in the knowledge graph.

The simplicity of the indexes makes them very suitable for evaluating the performance of an embedding algorithm even on a large scale.

as the set of all ranked predictions of a model, it is possible to define three different performance indexes: Hits@K, MR, and MRR.

[10] Hits@K reflects the accuracy of an embedding model to predict the relation between two given triples correctly.

[10] Mean reciprocal rank measures the number of triples predicted correctly.

[10] Mean reciprocal rank is generally used to quantify the effect of search algorithms.

[11] In particular, this technique completes a triple inferring the missing entity or relation.

[11] The decision is made with the model score function and a given threshold.

[11] Clustering is another application that leverages the embedded representation of a sparse knowledge graph to condense the representation of similar semantic entities close in a 2D space.

[4] The use of knowledge graph embedding is increasingly pervasive in many applications.

In the case of recommender systems, the use of knowledge graph embedding can overcome the limitations of the usual reinforcement learning.

[12][13] Training this kind of recommender system requires a huge amount of information from the users; however, knowledge graph techniques can address this issue by using a graph already constructed over a prior knowledge of the item correlation and using the embedding to infer from it the recommendation.

[14] It is possible to use the task of link prediction to infer a new connection between an already existing drug and a disease by using a biomedical knowledge graph built leveraging the availability of massive literature and biomedical databases.

[14] Knowledge graph embedding can also be used in the domain of social politics.

[5] In particular, these models use a third-order (3D) tensor, which is then factorized into low-dimensional vectors that are the embeddings.

[5][17] A third-order tensor is suitable for representing a knowledge graph because it records only the existence or absence of a relation between entities,[17] and so is simple, and there is no need to know a priori the network structure,[15] making this class of embedding models light, and easy to train even if they suffer from high-dimensionality and sparsity of data.

[5][17] This family of models uses a linear equation to embed the connection between the entities through a relation.

This class of models is inspired by the idea of translation invariance introduced in word2vec.

[7] A pure translational model relies on the fact that the embedding vector of the entities are close to each other after applying a proper relational translation in the geometric space in which they are defined.

[5] The closeness of the entities embedding is given by some distance measure and quantifies the reliability of a fact.

[17]It is possible to associate additional information to each element in the knowledge graph and their common representation facts.

[5] This family of models, in addition or in substitution of a translation they employ a rotation-like transformation.

[5] This group of embedding models uses deep neural network to learn patterns from the knowledge graph that are the input data.

[1][5] This family of models, instead of using fully connected layers, employs one or more convolutional layers that convolve the input data applying a low-dimensional filter capable of embedding complex structures with few parameters by learning nonlinear features.

[1][5][18] This family of models uses capsule neural networks to create a more stable representation that is able to recognize a feature in the input without losing spatial information.

[5] The advantage of this architecture is to memorize a sequence of fact, rather than just elaborate single events.

Embedding of a knowledge graph. The vector representation of the entities and relations can be used for different machine learning applications.
Publication timeline of some knowledge graph embedding models. In red the tensor decomposition models, in blue the geometric models, and in green the deep learning models. RESCAL [ 15 ] (2011) was the first modern KGE approach. In [ 16 ] it was applied to the YAGO knowledge graph. This was the first application of KGE to a large scale knowledge graph.
TransE embedding model. The vector representation (embedding) of the head plus the vector representation of the relation should be equal to the vector representation of the tail entity.