Dual numbers for automatic differentiation

Project Description

Calculations with dual numbers for computing derivatives.

If I is the identity matrix and J is the matrix with 1's above the main diagonal and 0's everywhere else, then a dual number has the form

a = a₀ I + a₁ J + a₂ J² + ...

For sufficiently smooth functions

f(xI + J) = f(x) I + f'(x) J + f"(x)/2! J² + ...

This allows us compute derivatives without taking limits of difference quotients.

(xI + J)³ = x³ I + 3 x² J + 3 x J² + J³.

We can read off the derivatives of f(x) = x³: f'(x) = 3 x² and f"(x)/2! = 3 x, so f"(x) = 6x and the third derivative is 3!.