Prev Next

exp_eps: First Order Reverse Sweep

Purpose
First order reverse mode uses the operation sequence , and zero order forward sweep values, to compute the first order derivative of one dependent variable with respect to all the independent variables. The computations are done in reverse of the order of the computations in the original algorithm.

Mathematical Form
Suppose that we use the algorithm exp_eps.hpp to compute exp_eps(xepsilon) with x is equal to .5 and epsilon is equal to .2. For this case, the mathematical function for the operation sequence corresponding to exp_eps is  \[
f ( x , \varepsilon ) =   1 + x + x^2 / 2  
\] 
The corresponding partial derivatives, and the value of the derivatives, are  \[
\begin{array}{rcl}
\partial_x f ( x , \varepsilon ) & = &  1 + x  = 1.5
\\
\partial_\varepsilon f ( x , \varepsilon ) & = &  0
\end{array}
\] 


epsilon
Since  \varepsilon is an independent variable, it could included as an argument to all of the  f_j functions below. The result would be that all the partials with respect to  \varepsilon would be zero and hence we drop it to simplify the presentation.

f_7
In reverse mode we choose one dependent variable and compute its derivative with respect to all the independent variables. For our example, we chose the value returned by exp_eps.hpp which is  v_7 . We begin with the function  f_7 where  v_7 is both an argument and the value of the function; i.e.,  \[
\begin{array}{rcl}
f_7 ( v_1 , v_2 , v_3 , v_4 , v_5 , v_6 , v_7 ) & = & v_7 
\\
\D{f_7}{v_7} & = & 1
\end{array}
\] 
All the other partial derivatives of  f_7 are zero.

Index 7: f_6
The last operation has index 7,  \[
     v_7 =   v_4 + v_6  
\] 
We define the function  f_6 ( v_1 , v_2 , v_3 , v_4 , v_5 , v_6 )  as equal to  f_7 except that  v_7 is eliminated using this operation; i.e.  \[
f_6  = 
f_7 [ v_1 , v_2 , v_3 , v_4 , v_5 , v_6 , v_7 ( v_4 , v_6 ) ]
\] 
It follows that  \[
\begin{array}{rcll}
\D{f_6}{v_4} 
& = & \D{f_7}{v_4} + 
     \D{f_7}{v_7} * \D{v_7}{v_4} 
& = 1
\\
\D{f_6}{v_6} 
& = & \D{f_7}{v_6} + 
     \D{f_7}{v_7} * \D{v_7}{v_6} 
& = 1
\end{array}
\] 
All the other partial derivatives of  f_6 are zero.

Index 6: f_5
The previous operation has index 6,  \[
     v_6 = v_5 / 2
\] 
We define the function  f_5 (  v_1 , v_2 , v_3 , v_4 , v_5 )  as equal to  f_6  except that  v_6  is eliminated using this operation; i.e.,  \[
f_5 = 
f_6 [  v_1 , v_2 , v_3 , v_4 , v_5 , v_6 ( v_5 ) ]
\] 
It follows that  \[
\begin{array}{rcll}
\D{f_5}{v_4} 
& = & \D{f_6}{v_4} 
& = 1
\\
\D{f_5}{v_5} 
& = & \D{f_6}{v_5} + 
     \D{f_6}{v_6} * \D{v_6}{v_5} 
& = 0.5
\end{array}
\] 
All the other partial derivatives of  f_5 are zero.

Index 5: f_4
The previous operation has index 5,  \[
     v_5 = v_3 * v_1
\] 
We define the function  f_4 (  v_1 , v_2 , v_3 , v_4 )  as equal to  f_5  except that  v_5  is eliminated using this operation; i.e.,  \[
f_4 = 
f_5 [  v_1 , v_2 , v_3 , v_4 , v_5 ( v_3 , v_1 )  ]
\] 
Given the information from the forward sweep, we have  v_3 =  0.5  and  v_1 = 0.5  . It follows that  \[
\begin{array}{rcll}
\D{f_4}{v_1} 
& = & \D{f_5}{v_1} + 
     \D{f_5}{v_5} * \D{v_5}{v_1} 
& =  0.25
\\
\D{f_4}{v_2} & = & \D{f_5}{v_2}  & = 0
\\
\D{f_4}{v_3} 
& = & \D{f_5}{v_3} + 
     \D{f_5}{v_5} * \D{v_5}{v_3} 
& =  0.25
\\
\D{f_4}{v_4}
& = & \D{f_5}{v_4} 
& = 1
\end{array}
\] 


Index 4: f_3
The previous operation has index 4,  \[
     v_4 = 1 + v_3
\] 
We define the function  f_3 (  v_1 , v_2 , v_3 )  as equal to  f_4  except that  v_4  is eliminated using this operation; i.e.,  \[
f_3 = 
f_4 [  v_1 , v_2 , v_3 , v_4 ( v_3 )  ]
\] 
It follows that  \[
\begin{array}{rcll}
\D{f_3}{v_1} 
& = & \D{f_4}{v_1} 
& =  0.25
\\
\D{f_3}{v_2} & = & \D{f_4}{v_2}  & = 0
\\
\D{f_3}{v_3} 
& = & \D{f_4}{v_3} + 
     \D{f_4}{v_4} * \D{v_4}{v_3} 
& =  1.25
\end{array}
\] 


Index 3: f_2
The previous operation has index 3,  \[
     v_3 = v_2 / 1
\] 
We define the function  f_2 (  v_1 , v_2 )  as equal to  f_3  except that  v_3  is eliminated using this operation; i.e.,  \[
f_2 = 
f_4 [  v_1 , v_2 , v_3 ( v_2 )  ]
\] 
It follows that  \[
\begin{array}{rcll}
\D{f_2}{v_1} 
& = & \D{f_3}{v_1} 
& =  0.25
\\
\D{f_2}{v_2} & = & \D{f_3}{v_2}  +
     \D{f_3}{v_3} * \D{v_3}{v_2}
& = 1.25
\end{array}
\] 


Index 2: f_1
The previous operation has index 1,  \[
     v_2 = 1 * v_1
\] 
We define the function  f_1 (  v_1 )  as equal to  f_2  except that  v_2  is eliminated using this operation; i.e.,  \[
f_1 = 
f_2 [  v_1 , v_2 ( v_1 )  ]
\] 
It follows that  \[
\begin{array}{rcll}
\D{f_1}{v_1} & = & \D{f_2}{v_1}  +
     \D{f_2}{v_2} * \D{v_2}{v_1}
& = 1.5
\end{array}
\] 
Note that  v_1 is equal to  x , so the derivative of exp_eps(xepsilon) at x equal to .5 and epsilon equal .2 is 1.5 in the x direction and zero in the epsilon direction. We also note that forward forward mode gave the same result for the partial in the x direction.

Verification
The file exp_eps_rev1.cpp contains a routine that verifies the values computed above. It returns true for success and false for failure. It only tests the partial derivatives of  f_j that might not be equal to the corresponding partials of  f_{j+1} ; i.e., the other partials of  f_j must be equal to the corresponding partials of  f_{j+1} .

Exercises
  1. Consider the case where  x = .1 and we first preform a zero order forward mode sweep for the operation sequence used above (in reverse order). What are the results of a first order reverse mode sweep; i.e., what are the corresponding values for  \D{f_j}{v_k} for all  j, k such that  \D{f_j}{v_k} \neq 0 .
  2. Create a modified version of exp_eps_rev1.cpp that verifies the values you obtained for the previous exercise. Also create and run a main program that reports the result of calling the modified version of exp_eps_rev1.cpp .

Input File: introduction/exp_apx/exp_eps.omh