Classifier chains is a machine learning method for problem transformation in multi-label classification.
It combines the computational efficiency of the binary relevance method while still being able to take the label dependencies into account for classification.
[1] Several problem transformation methods exist.
One of them is the Binary Relevance method (BR).
and a data set with instances of the form
is a set of labels assigned to the instance.
BR transforms the data set into
During this process the information about dependencies between labels is not preserved.
This can lead to a situation where a set of labels is assigned to an instance although these labels never co-occur together in the data set.
Loss of this information can in some cases lead to a decrease in classification performance.
[2] Another approach, which takes into account label correlations, is the Label Powerset method (LP).
is the power set of all labels in
The main drawback of this approach is that the number of label combinations grows exponentially with the number of labels.
For example, a multi-label data set with 10 labels can have up to
The Classifier Chains method is based on the BR method and it is efficient even on a big number of labels.
Furthermore, it considers dependencies between labels.
the Classifier Chain model (CC) learns
classifiers as in the Binary Relevance method.
All classifiers are linked in a chain through feature space.
-th label was assigned to the instance then
Thus, classifiers build a chain where each of them learns binary classification of a single label.
The features given to each classifier are extended with binary values that indicate which of previous labels were assigned to the instance.
by passing label information between classifiers through the feature space.
However, the result can vary for different order of chains.
In order to solve this problem and increase accuracy it is possible to use ensemble of classifiers.
[3] In Ensemble of Classifier Chains (ECC) several CC classifiers can be trained with random order of chains (i.e. random order of labels) on a random subset of data set.
Labels of a new instance are predicted by each classifier separately.
After that, the total number of predictions or "votes" is counted for each label.
The label is accepted if it was predicted by a percentage of classifiers that is bigger than some threshold value.
There is also regressor chains, which themselves can resemble vector autoregression models if the order of the chain makes sure temporal order is respected.