From b95a522539f7883013060e656bac2fb4aef84409 Mon Sep 17 00:00:00 2001 From: "Bowen S. Zhu" Date: Wed, 20 Apr 2022 14:47:35 -0400 Subject: [PATCH 1/4] Fic typo of sign in Taylor series --- _weave/lecture08/automatic_differentiation.jmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_weave/lecture08/automatic_differentiation.jmd b/_weave/lecture08/automatic_differentiation.jmd index 2f6f6805..46abc5b7 100644 --- a/_weave/lecture08/automatic_differentiation.jmd +++ b/_weave/lecture08/automatic_differentiation.jmd @@ -102,7 +102,7 @@ just happens to also be complex analytic when extended to the complex plane. Thus it has a Taylor series, and let's see what happens when we expand out this Taylor series purely in the complex direction: -$$f(x+ih) = f(x) + f'(x)ih + \frac{1}{2}f''(x)h^2 + \mathcal{O}(h^3)$$ +$$f(x+ih) = f(x) + f'(x)ih - \frac{1}{2}f''(x)h^2 + \mathcal{O}(h^3)$$ which we can re-arrange as: From 258df225035863572ce6ec1b7b8810f0e38e651d Mon Sep 17 00:00:00 2001 From: "Bowen S. Zhu" Date: Wed, 20 Apr 2022 14:57:25 -0400 Subject: [PATCH 2/4] Fix typo 'recal' -> 'recall' --- _weave/lecture08/automatic_differentiation.jmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_weave/lecture08/automatic_differentiation.jmd b/_weave/lecture08/automatic_differentiation.jmd index 46abc5b7..e9eb7273 100644 --- a/_weave/lecture08/automatic_differentiation.jmd +++ b/_weave/lecture08/automatic_differentiation.jmd @@ -117,7 +117,7 @@ since $Im(f(x)) = 0$ (since it's real valued, the next order term cancels for the same reason). Thus with a sufficiently small choice of $h$, this is the *complex step differentiation* formula for calculating the derivative. -But to understand the computational advantage, recal that $x$ is pure real, and +But to understand the computational advantage, recall that $x$ is pure real, and thus $x+ih$ is an imaginary number where **the $h$ never directly interacts with $x$** since a complex number is a two dimensional number where you keep the two pieces separate. Thus there is no numerical cancellation by using a small value From c0e09356a552996ddacb1da082f59caa7a70cd5d Mon Sep 17 00:00:00 2001 From: "Bowen S. Zhu" Date: Wed, 20 Apr 2022 14:58:57 -0400 Subject: [PATCH 3/4] Change 'imaginary' to 'complex' --- _weave/lecture08/automatic_differentiation.jmd | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/_weave/lecture08/automatic_differentiation.jmd b/_weave/lecture08/automatic_differentiation.jmd index e9eb7273..93423b69 100644 --- a/_weave/lecture08/automatic_differentiation.jmd +++ b/_weave/lecture08/automatic_differentiation.jmd @@ -118,7 +118,7 @@ the same reason). Thus with a sufficiently small choice of $h$, this is the *complex step differentiation* formula for calculating the derivative. But to understand the computational advantage, recall that $x$ is pure real, and -thus $x+ih$ is an imaginary number where **the $h$ never directly interacts with +thus $x+ih$ is a complex number where **the $h$ never directly interacts with $x$** since a complex number is a two dimensional number where you keep the two pieces separate. Thus there is no numerical cancellation by using a small value of $h$, and thus, due to the relative precision of floating point numbers, both From d8662bda1ca3deb9594a0a73741a9fbc3a898179 Mon Sep 17 00:00:00 2001 From: "Bowen S. Zhu" Date: Wed, 20 Apr 2022 16:38:49 -0400 Subject: [PATCH 4/4] Fix LaTeX math code --- _weave/lecture08/automatic_differentiation.jmd | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/_weave/lecture08/automatic_differentiation.jmd b/_weave/lecture08/automatic_differentiation.jmd index 93423b69..6559d17c 100644 --- a/_weave/lecture08/automatic_differentiation.jmd +++ b/_weave/lecture08/automatic_differentiation.jmd @@ -592,13 +592,16 @@ dual numbers with vectors for the components. But if there are vectors for the components, then we can think of the grouping of dual components as a matrix. Thus define our multidimensional multi-partial dual number as: -$$D0 = [d1,d2,d3,...,dn]$$ - -$$Sigma = [d11 d21 d31 - d12 ... - ...]$$ +$$D_0 = [d_1,d_2,d_3,\ldots,d_n]$$ + +$$\Sigma = \begin{bmatrix} + d_{11} & d_{12} & \cdots & d_{1n} \\ + d_{21} & d_{22} & & \vdots \\ + \vdots & & \ddots & \vdots \\ + d_{m1} & \hdots & \hdots & d_{mn} + \end{bmatrix}$$ -$$epsilon = [epsilon_1,epsilon_2,...,epsilon_m]$$ +$$\epsilon=[\epsilon_1,\epsilon_2,\ldots,\epsilon_m]$$ $$D = D_0 + \Sigma \epsilon$$