relentless.optimize.SteepestDescent#

class relentless.optimize.SteepestDescent(stop, max_iter, step_size, scale=1.0, line_search=None)#

Steepest descent algorithm.

For an ObjectiveFunction \(f\left(\mathbf{x}\right)\), the steepest descent algorithm seeks to approach a minimum of the function.

The optimization is performed using scaled variables \(\mathbf{y}\). Define \(\mathbf{X}\) as the scaling parameters for each variable such that \(y_i=x_i/X_i\). (A variable can be left unscaled by setting \(X_i=1\)).

Define \(\alpha\) as the descent step size hyperparameter. A LineSearch can optionally be performed to optimize the value of \(\alpha\) between \(0\) and the input value. The function is iteratively minimized by taking successive steps down the gradient of the function. If the scaled variables are \(\mathbf{y}_n\) at iteration \(n\), the next value of the variables is:

\[\mathbf{y}_{n+1} = \mathbf{y}_n-\alpha\nabla f\left(\mathbf{y}_n\right)\]

The gradient of the function with respect to the scaled variables is:

\[\nabla f\left(\mathbf{y}\right) = \left[X_1 \frac{\partial f}{\partial x_1}, \cdots, X_n \frac{\partial f}{\partial x_n}\right]\]

Note that this optimization procedure is equivalent to:

\[\left(x_i\right)_{n+1} = \left(x_i\right)_n-\alpha{X_i}^2 \frac{\partial f}{\partial x_i}\]

for each unscaled design variable \(x_i\).

Parameters:

  • stop (ConvergenceTest) – The convergence test used as the stopping criterion for the optimizer. Note that the result being tested will have unscaled variables and gradient.

  • max_iter (int) – The maximum number of optimization iterations allowed.

  • step_size (float) – The step size hyperparameter (\(\alpha\)).

  • scale (float or dict) – A scalar scaling parameter or scaling parameters (\(\mathbf{X}\)) keyed on one or more ObjectiveFunction design variables (defaults to 1.0, so that the variables are unscaled).

  • line_search (LineSearch) – The line search object used to find the optimal step size, using the specified step size value as the “maximum” step size (defaults to None).

Methods

descent_amount(gradient)

Calculate the descent amount for the optimization.

optimize(objective, variables[, directory, ...])

Perform the steepest descent optimization for the given objective function.

Attributes

line_search

The line search used to optimize the step size.

max_iter

The maximum number of optimization iterations allowed.

scale

Scaling parameter.

step_size

The step size hyperparameter (\(\alpha\)).

stop

The convergence test used as the stopping criterion for the optimizer.
descent_amount(gradient)#

Calculate the descent amount for the optimization.

The amount that each update descends down the scaled gradient is a constant \(\alpha\).

Parameters:

gradient (KeyedArray) – The scaled gradient of the objective function.

Returns:

The descent amount, keyed on the objective function design variables.

Return type:

KeyedArray

The line search used to optimize the step size.

Type:

LineSearch

property max_iter#

The maximum number of optimization iterations allowed.

Type:

int

optimize(objective, variables, directory=None, overwrite=False)#

Perform the steepest descent optimization for the given objective function.

If specified, a LineSearch is performed to choose an optimal step size.

If directory is specified and overwrite is True, directory will be cleared before the optimization begins. The output will be saved into a directory created for each iteration of the optimization, e.g., directory/0. To advance to the next iteration of the optimization (e.g., from iteration 0 to iteration 1), a directory directory/0/.next is created at iteration 0 to hold the proposed result at iteration 1. If line_search is None, its contents are immediately moved to directory/1 (leaving directory/0/.next) empty. If line_search is not None, directory/0/.line will be created for LineSearch.find() to use; the final result of the line search will be moved to directory/1.

Parameters:

  • objective (ObjectiveFunction) – The objective function to be optimized.

  • variables (IndependentVariable or tuple) – Design variable(s) to optimize.

  • directory (str or Directory) – Directory for writing output during optimization. Default of None requests no output is written.

  • overwrite (bool) – If True, overwrite the directory before beginning optimization.

Returns:

True if converged, False if not converged, None if no design variables are specified for the objective function.

Return type:

bool or None

Raises:

OSError – If directory is not empty and overwrite is False.

property scale#

Scaling parameter.

A scalar scaling parameter or scaling parameters (\(\mathbf{X}\)) keyed on one or more ObjectiveFunction design variables. Must be positive.

Type:

float or dict

property step_size#

The step size hyperparameter (\(\alpha\)). Must be positive.

Type:

float

property stop#

The convergence test used as the stopping criterion for the optimizer.

Type:

ConvergenceTest