Math Formulary

Keeping mathematical formulary in memory is mighty difficult for me, so I wrote this page as a little compendium (math rendering by KateX).

Calculus

Notation: $$\partial$$ partial derivative, nabla $$\nabla$$ all partial derivatives, laplacian $$\Delta = \nabla^2$$ all second derivatives.

• Chain rule: for $$f(g(x))$$ with $$z=f(y)$$ and $$y=g(x)$$, derive $$\frac{\partial f(g(x))}{\partial x} = \frac{\partial f(g(x))}{\partial y} \cdot \frac{\partial g(x)}{\partial x}$$, or short $$(f \circ g)' = (f' \circ g) \cdot g'$$
• Product rule: $$\frac{\partial}{\partial x} (f(x) \cdot g(x)) = \frac{\partial}{\partial x} f(x) \cdot g(x) + f(x) \cdot \frac{\partial}{\partial x} g(x)$$, or short $$(f \cdot g)' = f' \cdot g + f \cdot g'$$
• Quotient rule: $$\frac{\partial}{\partial x} \left ( f(x) \cdot {g(x)^{-1}} \right ) = g(x)^{-2} \left ( \frac{\partial}{\partial x} f(x) \cdot g(x) - f(x) \cdot \frac{\partial}{\partial x} g(x) \right )$$, or short $$(\frac{f}{g})' = \frac{f' \cdot g - f \cdot g'}{g^2}$$
• Reciprocal rule: $$\frac{\partial}{\partial x} f(x)^{-1} = -\frac{\frac{\partial}{\partial x} f(x)}{f(x)^2}$$, or short $$(f^{-1})' = -f' \cdot f^{-2}$$
• Sum rule: $$\frac{\partial}{\partial x} (f(x)+g(x)) = \frac{\partial}{\partial x} f(x) + \frac{\partial}{\partial x} g(x)$$, or short $$(f+g)' = f'+g'$$
• Powers: $$\frac{\partial}{\partial x} x^n = nx^{n-1}$$
• Exponential: $$\frac{\partial}{\partial x} a^x = a^{x} \ln(a)$$ with $$\frac{\partial}{\partial x} e^x = e^{x}$$ for special case $$a=e$$, and $$\frac{\partial}{\partial x} \ln(x) = x^{-1}$$
• Trigonometric: $$\frac{\partial}{\partial x} \sin(x) = \cos(x)$$, $$\frac{\partial}{\partial x} \cos(x) = -\sin(x)$$, and $$\frac{\partial}{\partial x} \tan(x) = 1+\tan^2(x)$$
• Intuition: 3Blue1Brown Calculus series

Linear algebra

Notation: $$\vec{u}\cdot\vec{v}$$ or $$\left \langle u, v \right \rangle$$ dot product, $$A \vec{u}$$ or $$AB$$ matrix multiplication, $$\vec{u} \times \vec{v}$$ cross product, $$A_{ij}$$ i-th row then j-th column of a matrix, $$\vec{v}_i$$ i-th row (entry) of a vector.

• Distributivity: $$A(\vec{u}+\vec{v}) = A\vec{u}+A\vec{v}$$, and $$A(B+C) = AB + AC$$
• Associativity of scalar and field multiplication: $$A(\alpha\vec{u}) = \alpha (A\vec{u}$$)
• Non-commutative except for diagonal matrixes: $$AB \neq BA$$ except when $$A, B$$ are zero outside their diagonal.
• Transpose: $$(A B)^T = B^T A^T$$, and $$(A^T)^{-1} = (A^{-1})^T$$, as well as $$(A + B)^T = A^T + B^T$$; special case $$A^T = A^{-1}$$ for orthogonal matrix.
• Invertibility: $$A^{-1} A = I$$ and $$(A B)^{-1} = B^{-1} A^{-1}$$ iff square and $$A = \left [ \vec{e}_1, \vec{e}_1, .., \vec{e}_{n} \right ]$$ for linearly independent basis vectors, , i.e. $$\det(A) \neq 0$$
• Eigenvector: A $$\vec{v}$$ that does not change direction under transformation $$A$$, implying $$(A - \alpha I) \vec{v} = 0$$, or short $$A \vec{v} = \alpha \vec{v}$$
• Intuition: 3Blue1Brown Linear algebra series

Matrix Calculus

Notation as in calculus and linear algebra above.

• Jacobian of $$\vec{y} = \vec{f}_{1..m}(\vec{x}_{1..n}) = \begin{bmatrix} f_1(\vec{x})\\ \vdots\\ f_m(\vec{x}) \end{bmatrix}: \mathbb{R}^n \to \mathbb{R}^m$$, is $$J = \nabla \vec{f} = \frac{\partial \vec{f}}{\partial \vec{x}} = \begin{bmatrix} \frac{\partial \vec{f}}{\partial \vec{x}_1} & \dots & \frac{\partial \vec{f}}{\partial \vec{x}_n} \end{bmatrix} = \begin{bmatrix} \nabla^T \vec{f}_1\\ \vdots\\ \nabla^T \vec{f}_m \end{bmatrix} = \begin{bmatrix} \frac{\partial f_1}{\partial x_1} & \dots & \frac{\partial f_1}{\partial x_n}\\ \vdots & \ddots & \vdots \\ \frac{\partial f_n}{\partial x_1} & \dots & \frac{\partial f_n}{\partial x_n} \end{bmatrix}$$
• Hessian of $$y = f(\vec{x}_{1..n}): \mathbb{R}^n \to \mathbb{R}^1$$, is $$H = \Delta f = \begin{bmatrix} \frac{\partial^2 f}{\partial x_1 \partial x_1} & \dots & \frac{\partial^2 f}{\partial x_1 \partial x_n}\\ \vdots & \ddots & \vdots \\ \frac{\partial^2 f}{\partial x_n \partial x_1} & \dots & \frac{\partial^2 f}{\partial x_n \partial x_n} \end{bmatrix}$$

Conclusion

And that is it! Hope you found something worth noting. Have a good day to remember!