Aspects of Correlation Matrices

Suppose $\underline{x}_j,\cdots,\underline{x}_m$ are random variables, and $\mathcal{K}_j$ are convex cones of Borel-measurable real-valued functions of $\underline{x}_j$ with finite variance. The elements of $\mathcal{K}_j$ are called transformations of the variable $\underline{x}_j$.

For instance, $\mathcal{K}_j$ can be the cone of monotone transformations, or the subspace of splines with given knots, or the subspace of quantifications of a categorical variable

A transformation $\kappa\in\mathcal{K}$ is standardized if $\mathbf{E}(\kappa(\underline{x}))=0$ and $\mathbf{E}(\kappa^2(\underline{x}))=1$. Standardized transformations define a sphere $\mathcal{S}_j$.

Now suppose $f$ is a concave and differentiable function defined on the space of all correlation matrices $R$ between $m$ random variables. Suppose we want to minimize $$g(\kappa_1,\cdots,\kappa_m)\mathop{=}\limits^{\Delta}f(R(\kappa_1(\underline{x}_1),\cdots,\kappa_m(\underline{x}_m)))$$ over all transformations $\kappa_j\in\mathcal{K}_j\cap\mathcal{S}_j$.

Because $f$ is concave $f(R)\leq f(S)+\hbox{ tr }\nabla f(S)(R-S).$ Collect the partials in the matrix $G.$ A majorization algorithm can minimize $\sum_{i=1}^m\sum_{j=1}^m g_{ij}(S)\mathbf{E}(\kappa_i\kappa_j),$ over all standardized transformations, which we do with block relaxation using $m$ blocks. In each block we must maximize a linear function on a cone, under a quadratic constraint, which is usually not hard to do.

This algorithm generalizes ACE, CA, and many other forms of MVA with OS. It was proposed first by De Leeuw [1988a], with additional theretical results in De Leeuw [1988b]. The function $f$ can be based on multiple correlations, eigenvalues, determinants, and so on.