cpu_mlp::LBFGS< V, M > Class Template Reference

Limited-memory BFGS (L-BFGS) minimizer. More...

#include <lbfgs.hpp>

Inheritance diagram for cpu_mlp::LBFGS< V, M >:

Collaboration diagram for cpu_mlp::LBFGS< V, M >:

Public Member Functions
void	setHistorySize (size_t history_size)
	Set history size for L-BFGS. More...
V	solve (V x, VecFun< V, double > &f, GradFun< V > &Gradient) override
	Solves the optimization problem using L-BFGS. More...
V	compute_direction (const V &grad, const RingBuffer< V > &s_list, const RingBuffer< V > &y_list, const RingBuffer< double > &rho_list)
	Two-loop recursion to compute search direction. More...
Public Member Functions inherited from cpu_mlp::FullBatchMinimizer< V, M >
virtual	~FullBatchMinimizer ()=default
virtual void	setInitialHessian (const M &)
	Sets the initial Hessian approximation (if applicable). More...
virtual void	setHessian (const HessFun< V, M > &)
	Sets the Hessian function (for Second Order methods). More...
template<auto LossFn, typename DataType >
V	solve_with_enzyme (V x, DataType *data)
	Helper to solve directly using an Enzyme-compatible raw function. More...
void	setMaxIterations (int max_iters)
	Sets the maximum number of iterations. More...
void	setMaxLineIters (int max_line)
	Sets the maximum number of iterations for the line search. More...
void	setArmijoMaxIter (int max_armijo)
	Sets the maximum iterations for Armijo condition check (alias for setMaxLineIters). More...
void	setTolerance (double tol)
	Sets the tolerance for convergence. More...
unsigned int	iterations () const
	Returns the number of iterations performed. More...
double	tolerance () const
	Returns the tolerance used. More...
void	setRecorder (::IterationRecorder< CpuBackend > *recorder)
	Attach a recorder for loss/grad history. More...

Additional Inherited Members
Protected Member Functions inherited from cpu_mlp::FullBatchMinimizer< V, M >
double	line_search (V x, V p, VecFun< V, double > &f, GradFun< V > &Gradient)
	Backtracking Line Search satisfying Wolfe Conditions. More...
Protected Attributes inherited from cpu_mlp::FullBatchMinimizer< V, M >
unsigned int	_max_iters = 1000
unsigned int	_iters = 0
double	_tol = 1e-10
::IterationRecorder< CpuBackend > *	recorder_ = nullptr
	Optional recorder for diagnostics. More...
double	c1 = 1e-4
double	c2 = 0.9
double	rho = 0.5
double	max_line_iters = 50

Detailed Description

template<typename V, typename M>
class cpu_mlp::LBFGS< V, M >

Limited-memory BFGS (L-BFGS) minimizer.

Approximates the inverse Hessian using a history of size m.

Member Function Documentation

compute_direction()

template<typename V , typename M >

V cpu_mlp::LBFGS< V, M >::compute_direction ( const V &  grad, const RingBuffer< V > &  s_list, const RingBuffer< V > &  y_list, const RingBuffer< double > &  rho_list  )

inline

Two-loop recursion to compute search direction.

Here is the call graph for this function:

Here is the caller graph for this function:

setHistorySize()

template<typename V , typename M >

void cpu_mlp::LBFGS< V, M >::setHistorySize ( size_t  history_size )

inline

Set history size for L-BFGS.

Parameters

history_size

Number of curvature pairs to store (m).

solve()

template<typename V , typename M >

V cpu_mlp::LBFGS< V, M >::solve ( V  x, VecFun< V, double > &  f, GradFun< V > &  Gradient  )

inlineoverridevirtual

Solves the optimization problem using L-BFGS.

Parameters

x	Initial parameter vector.
f	Objective function.
Gradient	Gradient function.

Returns

Optimized parameter vector.

Implements cpu_mlp::FullBatchMinimizer< V, M >.

Here is the call graph for this function:

---

The documentation for this class was generated from the following file:

• src/minimizer/lbfgs.hpp