I.3.7.3: Optimal Scaling with LINEALS

Suppose we have $m$ categorical variables, where variable $j$ has $k_j$ categories. Also suppose $C_{j\ell}$ are the $k_j\times k_\ell$ cross tables and $D_j$ are the diagonal matrices with univariate marginals. Both the $C_{jl}$ and the $D_j$ are normalized so they add up to one.

A quantification of variable $j$ is a $k_j$ element vector $y_j$, normalized by $e'D_jy_j=0$ and $y_j'D_j^{\ }y_j^{\ }=1$. If we replace the categories of a variable by the corresponding elements of the quantification vector then the correlation between quantified variables $j$ and $\ell$ is $$\rho_{j\ell}(y_1,\cdots,y_m)\mathop{=}\limits^{\Delta}y_j'C_{j\ell}^{\ }y_\ell^{\ }.$$ Of course $\rho_{j\ell}(y_1,\cdots,y_m)=\rho_{\ell j}(y_1,\cdots,y_m)$ for all $j$ and $\ell$, and $\rho_{jj}(y_1,\cdots,y_m)=1$ for all $j$.

The correlation ratio between variables $j$ and $\ell$ is $$\eta^2_{j\ell}(y_1,\cdots,y_m)\mathop{=}\limits^{\Delta}y_j'C_{jl}^{\ }D_\ell^{-1}C_{\ell j}^{\ }y_j^{\ }.$$ In general $\eta^2_{j\ell}(y_1,\cdots,y_m)\not=\eta^2_{\ell j}(y_1,\cdots,y_m)$, but still $\eta^2_{jj}(y_1,\cdots,y_m)=1$.

Statistical theory, and the Cauchy-Schwartz inequality, tell us that $$\begin{align*} \rho_{j\ell}^2(y_1,\cdots,y_m)&\leq \eta_{j\ell}^2(y_1,\cdots,y_m),\\ \rho_{j\ell}^2(y_1,\cdots,y_m)&\leq\eta_{\ell j}^2(y_1,\cdots,y_m), \end{align*}$$ with equality if and only if $$\begin{align*} C_{\ell j}y_j&=\rho_{j\ell}(y_1,\cdots,y_m)D_\ell y_\ell,\\ C_{j\ell}y_\ell&=\rho_{j\ell}(y_1,\cdots,y_m)D_j y_j, \end{align*}$$ i.e. if and only if the regressions between the quantified variables are both linear.

De Leeuw [1988] has suggested to find standardized quantifications in such a way that the loss function $$f(y_1,\cdots,y_m)\sum_{j=1}^m\sum_{\ell=1}^m(\eta_{j\ell}^2(y_1,\cdots,y_m)-\rho_{j\ell}^2(y_1,\cdots,y_m))$$ is minimized. Thus we try to find quantifications of the variables that linearize all bivariate regressions. A block relaxation method to do just this is implemented in the lineals function of the R package aspect [Mair and De Leeuw, 2010]. In lineals there is the additional option of requiring that the elements of the $y_j$ are increasing or decreasing.

If we change quantification $y_j$ while keeping all $y_\ell$ with $\ell\not= j$ at their current values, then we have to minimize $$y_j'\left\{\sum_{\ell\not= j}C_{jl}^{\ }\left[D_\ell^{-1}-y_\ell^{\ }y_\ell'\right]C_{\ell j}^{\ }\right\}y_j^{\ }\tag{1}$$ over all $y_j$ with $e'D_jy_j=0$ and $y_j'D_j^{\ }y_j^{\ }=1$. Thus each step in the cycle amounts to finding the eigenvector corresponding with the smallest eigenvalue of the matrix in $\text{(1)}$.