Commit 5e51032d authored by Erik Strand's avatar Erik Strand

Add notes on parabolic interpolation

parent 9ea9cf49
......@@ -26,5 +26,6 @@ print(sp.latex(sol))
q = (x_2 - x_3) * (y_2 - y_1)
r = (x_2 - x_1) * (y_2 - y_3)
alt = x_2 - ((x_2 - x_3) * q - (x_2 - x_1) * r) / 2 / (q - r)
print(sp.latex(alt))
print(sp.simplify(sol - alt))
......@@ -2,6 +2,49 @@
title: Optimization Algorithms
---
## Parabolic Interpolation
A task that comes up frequently in optimization problems is guessing a function minimum based on a
parabola fit between three points. So I'll derive this technique here.
First, we need a parabola that goes through three points, say $$(x_1, y_1)$$, $$(x_1, y_1)$$, and
$$(x_3, y_3)$$. In the interest of generality, I'll construct it using Lagrange polynomials.
$$
p(x) = \frac{y_1 (x - x_2) (x - x_3)}{(x_1 - x_2) (x_1 - x_3)}
+ \frac{y_2 (x - x_1) (x - x_3)}{(x_2 - x_1) (x_2 - x_3)}
+ \frac{y_3 (x - x_1) (x - x_2)}{(x_3 - x_1) (x_3 - x_2)}
$$
Differentiating, we find that the derivative is equal to the following.
$$
\begin{aligned}
p'(x) &=
\frac{y_{1} \left(x - x_{2}\right)}{\left(x_{1} - x_{2}\right) \left(x_{1} - x_{3}\right)}
+ \frac{y_{1} \left(x - x_{3}\right)}{\left(x_{1} - x_{2}\right) \left(x_{1} - x_{3}\right)} \\
&+ \frac{y_{2} \left(x - x_{1}\right)}{\left(- x_{1} + x_{2}\right) \left(x_{2} - x_{3}\right)}
+ \frac{y_{2} \left(x - x_{3}\right)}{\left(- x_{1} + x_{2}\right) \left(x_{2} - x_{3}\right)} \\
&+ \frac{y_{3} \left(x - x_{1}\right)}{\left(- x_{1} + x_{3}\right) \left(- x_{2} + x_{3}\right)}
+ \frac{y_{3} \left(x - x_{2}\right)}{\left(- x_{1} + x_{3}\right) \left(- x_{2} + x_{3}\right)}
\end{aligned}
$$
Setting this to zero, we find that the solution is surprisingly simple.
$$
x_\text{min} = \frac{1}{2} \cdot \frac{x_1^2 (y_2 - y_3) + x_2^2 (y_3 - y_1) + x_3^2 (y_{1} - y_2)}
{x_1 (y_2 - y_3) + x_2 (y_3 - y_1) + x_3 (y_1 - y_2)}
$$
This can also be factored so that it only involves differences of coordinates and function values.
$$
x_\text{min} =
x_{2} - \frac{1}{2} \cdot
\frac{(x_2 - x_3)^2 (y_2 - y_1) - (x_2 - x_1)^2 (y_2 - y_3)}{(x_2 - x_3) (y_2 - y_1) - (x_2 - x_1) (y_2 - y_3)}
$$
## Nelder-Mead
TODO: Demonstrate that the Numerical Recipes formulation is equivalent to the standard formulation.
......
Markdown is supported
0% or .
You are about to add 0 people to the discussion. Proceed with caution.
Finish editing this message first!
Please register or to comment