Handy Formulas for Gradients and Jocobians

The following are some handy formulas for computing the gradient or Jacobian of a (vector) function expressed in terms of matrices:

 

$$\nabla_{\bf x}({\bf x}^T{\bf y}) = \nabla_{\bf x}({\bf y}^T{\bf x}) = {\bf y}$$ (1)

 

$$\nabla_{\bf x}({\bf x}^T{\bf x}) = 2{\bf x}$$ (2)

 

$$\nabla_{\bf x}({\bf x}^T{\bf A}{\bf y}) = {\bf A}{\bf y}$$ (3)

 

$$\nabla_{\bf x}({\bf y}^T{\bf A}{\bf x}) = {\bf A}^T{\bf y}$$ (4)

 

$$\nabla_{\bf x}({\bf x}^T{\bf A}{\bf x}) = \left\{ \begin{ar... ...{\bf A}+ {\bf A}^T){\bf x}, \mbox{otherwise} \end{array}\right.$$ (5)

 

$$\nabla_{\bf x}[{\bf f}^T({\bf x}){\bf g}({\bf x})] = \nabla... ...{\bf x})] = {\bf J}_{\bf f}^T{\bf g}+ {\bf J}_{\bf g}^T{\bf f}$$ (6)

$${\bf J}({\bf f}({\bf x})) = {\bf J}_{\bf f}= \left[ \begin{a... ...f_1({\bf x})\\ \vdots\\ \nabla^T f_m({\bf x})\end{array}\right]$$ (7)

$${\bf J}({\bf q}f({\bf x})) = {\bf q}\nabla^T f({\bf x})$$ (8)

$${\bf J}({\bf Q}{\bf f}({\bf x})) = {\bf J}(\sum_i {\bf q}_i ... ...sum_i {\bf q}_i \nabla^T f_i({\bf x})) = {\bf Q}{\bf J}_{\bf f}$$ (9)

$$\nabla_{\bf x}[{\bf g}^T({\bf x}) {\bf Q}{\bf g}({\bf x})] =... ...\bf Q}^T) {\bf g}({\bf x}), \mbox{ otherwise}\end{array}\right.$$ (10)

 

$$\nabla_{\bf x}[{\bf f}^T({\bf x}) {\bf Q}{\bf g}({\bf x})] =... ...bf g}({\bf x}) + {\bf J}_{{\bf g}}^T {\bf Q}^T {\bf f}({\bf x})$$ (11)