Eigenvalues and Eigenvectors of Asymmetric Matrices

If $A$ is a square but asymmetric real matrix the eigenvector-eigenvalue situation becomes quite different from the symmetric case. We gave a variational treatment of the symmetric case, using the connection between eigenvalue problems and quadratic forms (or ellipses and other conic sections, if you have a geometric mind).That connection, howver, is lost in the asymmetric case, and there is no obvious variational problem associated with eigenvalues and eigenvectors.

Let us first define eigenvalues and eigenvectors in the asymmetric case. As before, an eigen-pair $(x,\lambda)$ is a solution to the equation $Ax=\lambda x$ with $x\not= 0$. This can also be written as $(A-\lambda I)x=0$, which shows that the eigenvalues are the solutions of the equation $\pi_A(\lambda)=\mathbf{det}(A-\lambda I)=0$. Now the function $\pi_A$ is the characteristic polynomial of $A$. It is a polynomial of degree $n$, and by the fundamental theorem of algebra there are $n$ real and complex roots, counting multiplicities. Thus $A$ has $n$ eigenvalues, as before, although some of them can be complex

A first indication that something may be wrong, or least fundamentally different, is the matrix $$A=\begin{bmatrix}0&1\\0&0\end{bmatrix}.$$ The characteristic equation $\pi_A(\lambda)=\lambda^2=0$ has the root $\lambda=0$, with multiplicity 2. Thus an eigenvector should satisfy $$\begin{bmatrix}0&1\\0&0\end{bmatrix}\begin{bmatrix}x_1\\x_2\end{bmatrix}=\begin{bmatrix}0\\0\end{bmatrix},$$ which merely says $x_2=0$. Thus $A$ does not have two linearly independent, let alone orthogonal, eigenvectors.

A second problem is illustrated by the anti-symmetric matrix $$A=\begin{bmatrix}0&1\\-1&0\end{bmatrix}$$ for which the characteristic polynomial is $\pi_A(\lambda)=\lambda^2+1$. The characteristic equations has the two complex roots $+\sqrt{-1}$ and $-\sqrt{-1}$. The corresponding eigenvectors are the columns of $$\begin{bmatrix}1&1\\\sqrt{-1}&-\sqrt{-1}\end{bmatrix}.$$ Thus both eigenvalues and eigenvectors may be complex. In fact if we take complex conjugates on both sides of $Ax=\lambda x$, and remember that $A$ is real, we see that $\overline{Ax}=A\overline x=\overline{\lambda}\overline{x}.$ Thus $(x,\lambda)$ is an eigen-pair if and only if $(\overline{x},\overline{\lambda})$ is. If $A$ is real and of odd order it always has at least one real eigenvalue. If an eigenvalue $\lambda$ is real and of multiplicity $m$, then there are $m$ corresponding real and linearly independent eigenvectors. They are simply a basis for the null space of $A-\lambda I$.

A third problem, which by definition did not come up in the symmetric case, is that we now have an eigen problem for both $A$ and its transpose $A'$. Since for all $\lambda$ we have $\mathbf{det}(A-\lambda I)=\mathbf{det}(A'-\lambda I)$ it follows that $A$ and $A'$ have the same eigenvalues. We say that $(x,\lambda)$ is a right eigen-pair of $A$ if $Ax=\lambda x$, and $(y,\lambda)$ is a left eigen-pair of $A$ if $y'A=\lambda y'$, which is of course the same as $A'y=\lambda y$.

A matrix $A$ is diagonalizable if there exists a non-singular $X$ such that $X^{-1}AX=\Lambda$, with $\Lambda$ diagonal. Instead of the spectral decomposition of symmetric matrices we have the decomposition $A=X\Lambda X^{-1}$ or $X^{-1}AX=\Lambda$. A matrix that is not diagonalizable is called defective.

Result: A matrix $A$ is diagonalizable if and only if it has $n$ linearly independent right eigenvectors if and only if it has $n$ linearly independent left eigenvectors. We show this for right eigenvectors. Collect them in the columns of a matrix $X$. Thus $AX=X\Lambda$, with $X$ non-singular. This implies $X^{-1}A=\Lambda X^{-1}$, and thus the rows of $Y=X^{-1}$ are $n$ linearly independent left eigenvalues. Also $X^{-1}AX=\Lambda$. Conversely if $X^{-1}AX=\Lambda$ then $AX=X\Lambda$ and $A'X^{-1}=X^{-1}\Lambda$, so we have linearly independent left and right eigenvectors.

Result: If the $n$ eigenvalues $\lambda_j$ of $A$ are all diferent then the eigenvectors $x_j$ are linearly independent. We show this by contradiction. Select a maximally linearly independent subset from the $x_j$. Suppose there are $p<m$, so the eigenvectors are linearly dependent. Without loss of generality the maximally linearly independent subset can be taken as the first $p$. Then for all $j>p$ there exist $\alpha_{js}$ such that $$x_j=\sum_{s=1}^p\alpha_{js} x_s.\tag{1}$$ Premultiply $(1)$ with $\lambda_j$ to get $$\lambda_jx_j=\sum_{s=1}^p\alpha_{js} \lambda_jx_s.\tag{2}$$ Premultiply $(1)$ by $A$ to get $$\lambda_jx_j=\sum_{s=1}^p\alpha_{js} \lambda_sx_s.\tag{3}$$ Subtract $(2)$ from $(3)$ to get $$\sum_{s=1}^p\alpha_{js}(\lambda_s-\lambda_j)x_s = 0,$$ which implies that $\alpha_{js}(\lambda_s-\lambda_j)=0,$ because the $x_s$ are linearly independent. Since the eigenvalues are unequal, this implies $a_{js}=0$ and thus $x_j=0$ for all $j>p$, contradicting that the $x_j$ are eigenvectors. Thus $p=m$ and the $x_j$ are linearly independent.

Note 030615 Add small amount on defective matrices. Add stuff on characteristic and minimal polynomials. Take about using the SVD instead.