Standard Gradient Descent with optional Line Search. More...

#include <gd.hpp>

Inheritance diagram for cpu_mlp::GradientDescent< V, M >:

Collaboration diagram for cpu_mlp::GradientDescent< V, M >:

Public Member Functions
void	setStepSize (double alpha) noexcept
	Sets the constant step size. More...

void	useLineSearch (bool enable) noexcept
	Enables or disables Line Search. More...

V	solve (V x, VecFun< V, double > &f, GradFun< V > &Gradient) override
	Run Gradient Descent. More...

Public Member Functions inherited from cpu_mlp::FullBatchMinimizer< V, M >
virtual	~FullBatchMinimizer ()=default

virtual void	setInitialHessian (const M &)
	Sets the initial Hessian approximation (if applicable). More...

virtual void	setHessian (const HessFun< V, M > &)
	Sets the Hessian function (for Second Order methods). More...

template<auto LossFn, typename DataType >
V	solve_with_enzyme (V x, DataType *data)
	Helper to solve directly using an Enzyme-compatible raw function. More...

void	setMaxIterations (int max_iters)
	Sets the maximum number of iterations. More...

void	setMaxLineIters (int max_line)
	Sets the maximum number of iterations for the line search. More...

void	setArmijoMaxIter (int max_armijo)
	Sets the maximum iterations for Armijo condition check (alias for setMaxLineIters). More...

void	setTolerance (double tol)
	Sets the tolerance for convergence. More...

unsigned int	iterations () const
	Returns the number of iterations performed. More...

double	tolerance () const
	Returns the tolerance used. More...

void	setRecorder (::IterationRecorder< CpuBackend > *recorder)
	Attach a recorder for loss/grad history. More...

Additional Inherited Members
Protected Member Functions inherited from cpu_mlp::FullBatchMinimizer< V, M >
double	line_search (V x, V p, VecFun< V, double > &f, GradFun< V > &Gradient)
	Backtracking Line Search satisfying Wolfe Conditions. More...

Protected Attributes inherited from cpu_mlp::FullBatchMinimizer< V, M >
unsigned int	_max_iters = 1000

unsigned int	_iters = 0

double	_tol = 1e-10

::IterationRecorder< CpuBackend > *	recorder_ = nullptr
	Optional recorder for diagnostics. More...

double	c1 = 1e-4

double	c2 = 0.9

double	rho = 0.5

double	max_line_iters = 50

Detailed Description

Standard Gradient Descent with optional Line Search.

Uses the full gradient of the objective. Can perform a line search or use a fixed step.

Member Function Documentation

template<typename V , typename M >

void cpu_mlp::GradientDescent< V, M >::setStepSize ( double alpha )

inlinenoexcept

Sets the constant step size.

Parameters

alpha Step size (effectively learning rate).

template<typename V , typename M >

V cpu_mlp::GradientDescent< V, M >::solve	(	V	x,
		VecFun< V, double > &	f,
		GradFun< V > &	Gradient
	)

inlineoverridevirtual

Run Gradient Descent.

Parameters

Here is the call graph for this function:

template<typename V , typename M >

void cpu_mlp::GradientDescent< V, M >::useLineSearch ( bool enable )

inlinenoexcept

Enables or disables Line Search.

Parameters

enable If true, uses Wolfe line search.

The documentation for this class was generated from the following file: