Adjoint algorithmic differentiation

In theory, the cost of the reverse mode is less than 5 times that of the original computation, and it does not depend on the number of inputs for which sensitivities are needed¹. Bumping, which moves the inputs one at a time and recomputes, takes more work in proportion to the number of sensitivities².

On the other hand, one reverse run returns the sensitivities of a single output, so several outputs need one run each. How it works, a worked example, and the limits are covered in the page "Adjoint Algorithmic Differentiation (AAD)".

Notes

1. Homescu, "Adjoints and automatic (algorithmic) differentiation in computational finance" (arXiv, 2011), page 3 (section 1). ↩

2. Capriotti and Giles, "Fast Correlation Greeks by Adjoint Algorithmic Differentiation" (arXiv, 2010), first two paragraphs. ↩