Probabilistic latent semantic analysis

Probabilistic latent semantic analysis (PLSA), also known as probabilistic latent semantic indexing (PLSI, especially in information retrieval circles) is a statistical technique for the analysis of two-mode and co-occurrence data.

In effect, one can derive a low-dimensional representation of the observed variables in terms of their affinity to certain hidden variables, just as in latent semantic analysis, from which PLSA evolved.

Compared to standard latent semantic analysis which stems from linear algebra and downsizes the occurrence tables (usually via a singular value decomposition), probabilistic latent semantic analysis is based on a mixture decomposition derived from a latent class model.

of words and documents, PLSA models the probability of each co-occurrence as a mixture of conditionally independent multinomial distributions: with

Note that the number of topics is a hyperparameter that must be chosen in advance and is not estimated from the data.

, a latent class is chosen conditionally to the document according to

Although we have used words and documents in this example, the co-occurrence of any couple of discrete variables may be modelled in exactly the same way.

[1] PLSA has applications in information retrieval and filtering, natural language processing, machine learning from text, bioinformatics,[2] and related areas.

It is reported that the aspect model used in the probabilistic latent semantic analysis has severe overfitting problems.

[3] This is an example of a latent class model (see references therein), and it is related[6][7] to non-negative matrix factorization.

The present terminology was coined in 1999 by Thomas Hofmann.

Plate notation representing the PLSA model ("asymmetric" formulation). is the document index variable, is a word's topic drawn from the document's topic distribution, , and is a word drawn from the word distribution of this word's topic, . The and are observable variables , the topic is a latent variable .