Necessary Conditions by Majorization

Suppose on majorizes on . We show that a necessary condition for to be a minimizer of on is that minimizes the majorization function on .

Theorem: If then

Proof: Suppose is such that Then which contradicts that minimizes . QED

As an example, suppose that we have a quadratic majorization of the form with positive definite. If minimizes over , then we must have with . Thus must be the weighted least squares projection of on . If is all of then we must have , which means .

For a concave function on a bounded set we have , and thus the necessary condition for a minimum is