Nondiagonal Weights in Least Squares
An even simpler example of quadratic majorization of a quadratic function is the following. Suppose we want to solve the problem of minimizing ϕ(ω)=(y−ω)′W(y−ω), over ω∈Ω, where Ω is the cone of isotonic vectors. This problem can be solved by general quadratic programming techniques (compare, for example, \cite{lawhan}), but it is easier in many respects to use iterated monotone regression.
Suppose we can find a diagonal D such that W≲D. A simple choice would be D=λ+I, with λ+ the largest eigenvalue of W, but sometimes other choices may be more appropriate.