mlpack  2.2.5
 All Classes Namespaces Files Functions Variables Typedefs Enumerations Enumerator Friends Macros Pages
Public Member Functions | List of all members
GradientDescent< FunctionType > Class Template Reference

Gradient Descent is a technique to minimize a function. More...

Public Member Functions

 GradientDescent (FunctionType &function, const double stepSize=0.01, const size_t maxIterations=100000, const double tolerance=1e-5)
 Construct the Gradient Descent optimizer with the given function and parameters. More...
 
const FunctionType & Function () const
 Get the instantiated function to be optimized. More...
 
FunctionType & Function ()
 Modify the instantiated function. More...
 
size_t MaxIterations () const
 Get the maximum number of iterations (0 indicates no limit). More...
 
size_t & MaxIterations ()
 Modify the maximum number of iterations (0 indicates no limit). More...
 
double Optimize (arma::mat &iterate)
 Optimize the given function using gradient descent. More...
 
double StepSize () const
 Get the step size. More...
 
double & StepSize ()
 Modify the step size. More...
 
double Tolerance () const
 Get the tolerance for termination. More...
 
double & Tolerance ()
 Modify the tolerance for termination. More...
 

Detailed Description

template<typename FunctionType>
class mlpack::optimization::GradientDescent< FunctionType >

Gradient Descent is a technique to minimize a function.

To find a local minimum of a function using gradient descent, one takes steps proportional to the negative of the gradient of the function at the current point, producing the following update scheme:

\[ A_{j + 1} = A_j + \alpha \nabla F(A) \]

where $ \alpha $ is a parameter which specifies the step size. $ F $ is the function being optimized. The algorithm continues until $ j $ reaches the maximum number of iterations—or when an update produces an improvement within a certain tolerance $ \epsilon $. That is,

\[ | F(A_{j + 1}) - F(A_j) | < \epsilon. \]

The parameter $\epsilon$ is specified by the tolerance parameter to the constructor.

For Gradient Descent to work, a FunctionType template parameter is required. This class must implement the following function:

double Evaluate(const arma::mat& coordinates); void Gradient(const arma::mat& coordinates, arma::mat& gradient);

Template Parameters
FunctionTypeDecomposable objective function type to be minimized.

Definition at line 53 of file gradient_descent.hpp.

Constructor & Destructor Documentation

GradientDescent ( FunctionType &  function,
const double  stepSize = 0.01,
const size_t  maxIterations = 100000,
const double  tolerance = 1e-5 
)

Construct the Gradient Descent optimizer with the given function and parameters.

The defaults here are not necessarily good for the given problem, so it is suggested that the values used be tailored to the task at hand.

Parameters
functionFunction to be optimized (minimized).
stepSizeStep size for each iteration.
maxIterationsMaximum number of iterations allowed (0 means no limit).
toleranceMaximum absolute tolerance to terminate algorithm.

Member Function Documentation

const FunctionType& Function ( ) const
inline

Get the instantiated function to be optimized.

Definition at line 84 of file gradient_descent.hpp.

FunctionType& Function ( )
inline

Modify the instantiated function.

Definition at line 86 of file gradient_descent.hpp.

size_t MaxIterations ( ) const
inline

Get the maximum number of iterations (0 indicates no limit).

Definition at line 94 of file gradient_descent.hpp.

size_t& MaxIterations ( )
inline

Modify the maximum number of iterations (0 indicates no limit).

Definition at line 96 of file gradient_descent.hpp.

double Optimize ( arma::mat &  iterate)

Optimize the given function using gradient descent.

The given starting point will be modified to store the finishing point of the algorithm, and the final objective value is returned.

Parameters
iterateStarting point (will be modified).
Returns
Objective value of the final point.
double StepSize ( ) const
inline

Get the step size.

Definition at line 89 of file gradient_descent.hpp.

double& StepSize ( )
inline

Modify the step size.

Definition at line 91 of file gradient_descent.hpp.

double Tolerance ( ) const
inline

Get the tolerance for termination.

Definition at line 99 of file gradient_descent.hpp.

double& Tolerance ( )
inline

Modify the tolerance for termination.

Definition at line 101 of file gradient_descent.hpp.


The documentation for this class was generated from the following file: