Matrix Derivatives

A matrix, of course, is just an element of a finite dimensional linear vector space. We write , and we use the inner product and corresponding norm Thus derivatives of real-valued function of matrices, or derivatives of matrix-valued functions of matrices, are covered by the usual definitions and formulas. Nevertheless there is a surprisingly huge literature on differential calculus for real-valued functions of matrices, and matrix-valued functions of matrices.

One of the reason for the proliferation of publications is that a matrix-valued function of matrices can be thought of a function of for matrix space to matrix-space , but also as a function of vector space to vector space . There are obvious isomorphisms between the two representations, but they naturally lead to different notations. We will consistently choose the matrix-space formulation, and consequently minimize the role of the operator and the special constructs such as the commutation and duplication matrix.

The other choice

Nevertheless having a compendium of the standard real-valued and matrix-valued functions available is of some interest. The main reference is the book by Magnus and Neudecker [1999]. We will avoid using differentials and the operator.

Suppose is a matrix valued function of a single variable . In other words is a matrix of functions, as in Now the derivatives of any order of , if they exist, are also matrix valued functions If is a function of a vector then partial derivatives are defined similarly, as in with The notation becomes slightly more complicated if is a function of a matrix , i.e. an element of . It then makes sense to write the partials as where and