The Moreau envelope (or the Moreau-Yosida regularization)
of a proper lower semi-continuous convex function
is a smoothed version of
It was proposed by Jean-Jacques Moreau in 1965.
[1] The Moreau envelope has important applications in mathematical optimization: minimizing over
and minimizing over
are equivalent problems in the sense that the sets of minimizers of
However, first-order optimization algorithms can be directly applied to
may be non-differentiable while
is always continuously differentiable.
Indeed, many proximal gradient methods can be interpreted as a gradient descent method over
The Moreau envelope of a proper lower semi-continuous convex function
from a Hilbert space
is defined as[2]
inf
Given a parameter
, the Moreau envelope of
is also called as the Moreau envelope of
with parameter
{\displaystyle \nabla M_{\lambda f}(x)={\frac {1}{\lambda }}(x-\mathrm {prox} _{\lambda f}(x))}
By defining the sequence
{\displaystyle x_{k+1}=\mathrm {prox} _{\lambda f}(x_{k})}
and using the above identity, we can interpret the proximal operator as a gradient descent algorithm over the Moreau envelope.
denotes the convex conjugate of
Since the subdifferential of a proper, convex, lower semicontinuous function on a Hilbert space is inverse to the subdifferential of its convex conjugate, we can conclude that if
is the maximizer of the above expression, then
is the minimizer in the primal formulation and vice versa.