Automatic differentiation is also called algorithmic differentiation, which is why the adjoint method is abbreviated AAD. Automatic differentiation has a forward mode (tangent linear mode), which passes derivatives along in the same direction as the computation, and a reverse mode (adjoint mode), which works backward from the output to the inputs1. The forward mode needs one run per input direction, while the reverse mode obtains the derivatives with respect to all inputs in one run for a single output.

Some tools generate code that computes the derivatives from the source code, and others replace the operations with a type that pairs a value with its derivative (operator overloading)2. How it works and where it is used are covered in the page "Adjoint Algorithmic Differentiation (AAD)".

Notes

  1. Homescu, "Adjoints and automatic (algorithmic) differentiation in computational finance" (arXiv, 2011), page 3 (section 1). ↩

  2. Giles, "Advanced Monte Carlo Methods: Computing Greeks" (Oxford University, lecture slides), page 19. ↩