Video matting

The technique is very popular in video editing because it allows to substitute the background, or process the layers individually.

The matte can serve as a binary mask, defining which of the image parts are visible.

Video SnapCut,[2] which later was incorporated in Adobe After Effects as Roto Brush tool, was developed in 2009.

The method makes use of local classifiers for binary image segmentation near the target object's boundary.

The results of the segmentation are propagated to the next frame using optical flow, and an image matting algorithm [3] is applied.

The propagation of trimap with optical flow was enhanced with control points along the object edge.

The benchmark consists of a dataset with test sequences and a result comparison methodology.

Currently there exists one major video matting online benchmark,[6] which uses chroma keying and stop motion for ground truth estimation.

As objective metrics do not represent human perception of quality, a subjective survey is necessary to provide adequate comparison.

There are several software implementations: To enhance the speed and quality of matting, some methods use additional data.

A Zoom plugin had been developed,[13] and Skype announced Background Replace in June 2020.

Video matting is crucial in 2D to 3D conversion, where the alpha matte is used to correctly process transparent objects.

Left to right: input image, background, foreground, and alpha matte.
The trimap (bottom) is used as a guide for estimating the alpha matte. White pixels are foreground, black pixels are background and grey pixels are yet to be estimated. Matting algorithms take the complete frame (top) and the trimap as input to produce the alpha matte (middle)