Nondiagonal Weights in Least Squares

An even simpler example of quadratic majorization of a quadratic function is the following. Suppose we want to solve the problem of minimizing $$\phi(\omega)= (y-\omega)'W(y-\omega),$$ over $\omega\in\Omega,$ where $\Omega$ is the cone of isotonic vectors. This problem can be solved by general quadratic programming techniques (compare, for example, \cite{lawhan}), but it is easier in many respects to use iterated monotone regression.

Suppose we can find a diagonal $D$ such that $W\lesssim D.$ A simple choice would be $D=\lambda_+I,$ with $\lambda_+$ the largest eigenvalue of $W,$ but sometimes other choices may be more appropriate.