Proximal gradient methods are a generalized form of projection used to solve non-differentiable convex optimization problems.
are possibly non-differentiable convex functions.
The lack of differentiability rules out conventional smooth optimization techniques like the steepest descent method and the conjugate gradient method, but proximal gradient methods can be used instead.
Proximal gradient methods starts by a splitting step, in which the functions
are used individually so as to yield an easily implementable algorithm.
They are called proximal because each non-differentiable function among
Iterative shrinkage thresholding algorithm,[1] projected Landweber, projected gradient, alternating projections, alternating-direction method of multipliers, alternating split Bregman are special instances of proximal algorithms.
[2] For the theory of proximal gradient methods from the perspective of and with applications to statistical learning theory, see proximal gradient methods for learning.
This algorithm is employed to recover/synthesize a signal satisfying simultaneously several convex constraints.
be the indicator function of non-empty closed convex set
This reduces to convex feasibility problem, which require us to find a solution such that it lies in the intersection of all convex sets
is updated as However beyond such problems projection operators are not appropriate and more general operators are required to tackle them.
Among the various generalizations of the notion of a convex projection operator that exist, proximal operators are best suited for other purposes.
Special instances of Proximal Gradient Methods are