| Optimization problem | Algorithm |
|---|---|
| Line Search Methods | Exact Line Search |
| Root finding using Bisection | |
| Initial Bracketing | |
| Unconstrained Optimization | Exercise problems |
| Golden Section | |
| Fibonacci | |
| Newton | |
| Secant | |
| Bisection | |
| Gradient Methods | Gradient Descent |
| Momentum & Nesterov's Momentum | |
| Conjugate Gradient | |
| Adagrad_RMS | |
| Second Order Methods | DFP_BFGS |
| Direct Methods | cyclic coordinate |
| Hooke Jeeves |