cpu_mlp::SLBFGS< V, M > Class Template Reference

Stochastic Limited-memory BFGS (S-LBFGS) minimizer. More...

#include <s_lbfgs.hpp>

Inheritance diagram for cpu_mlp::SLBFGS< V, M >:

Collaboration diagram for cpu_mlp::SLBFGS< V, M >:

Public Types
using	Base = StochasticMinimizer< V, M >
using	BatchGradFun = std::function< void(const V &, const std::vector< size_t > &, V &)>
using	BatchLossFun = std::function< double(const V &, const std::vector< size_t > &)>

Public Member Functions
V	stochastic_solve (V weights, const BatchLossFun &f, const BatchGradFun &batch_g, int m, int M_param, int L, int b, int b_H, double step_size, int N)
	Stochastic Solve using Batch Callbacks. More...
void	setData (const BatchLossFun &f, const BatchGradFun &g)
	Configure data and callbacks for the solver. More...
void	setMaxIterations (int max_iters)
	Sets the maximum number of iterations. More...
void	setStepSize (double s)
	Sets the step size (learning rate). More...
void	setTolerance (double tol)
	Sets the tolerance for convergence (full gradient norm). More...
Public Member Functions inherited from cpu_mlp::StochasticMinimizer< V, M >
virtual	~StochasticMinimizer ()=default
void	setMaxIterations (int max_iters)
	Sets the maximum number of iterations. More...
void	setStepSize (double s)
	Sets the step size (learning rate). More...
void	setTolerance (double tol)
	Sets the tolerance for convergence (full gradient norm). More...
void	setRecorder (::IterationRecorder< CpuBackend > *recorder)
	Attach a recorder for loss/grad history. More...

Static Public Member Functions
static std::vector< size_t >	sample_minibatch_indices (const size_t N, size_t batch_size, std::mt19937 &rng)
	Helper to sample minibatch indices. More...

Protected Attributes
unsigned int	_iters
unsigned int	_max_iters
double	_tol
double	step_size
Protected Attributes inherited from cpu_mlp::StochasticMinimizer< V, M >
unsigned int	_max_iters = 1000
unsigned int	_iters = 0
double	_tol = 1e-4
double	step_size = 0.01
::IterationRecorder< CpuBackend > *	recorder_ = nullptr
	Optional recorder for diagnostics. More...

Detailed Description

template<typename V, typename M>
class cpu_mlp::SLBFGS< V, M >

Stochastic Limited-memory BFGS (S-LBFGS) minimizer.

Implements a stochastic variance-reduced quasi-Newton method with batch callbacks.

Member Typedef Documentation

Base

template<typename V , typename M >

using cpu_mlp::SLBFGS< V, M >::Base = StochasticMinimizer<V, M>

BatchGradFun

template<typename V , typename M >

using cpu_mlp::SLBFGS< V, M >::BatchGradFun = std::function<void(const V &, const std::vector<size_t> &, V &)>

BatchLossFun

template<typename V , typename M >

using cpu_mlp::SLBFGS< V, M >::BatchLossFun = std::function<double(const V &, const std::vector<size_t> &)>

Member Function Documentation

sample_minibatch_indices()

template<typename V , typename M >

std::vector< size_t > cpu_mlp::SLBFGS< V, M >::sample_minibatch_indices ( const size_t  N, size_t  batch_size, std::mt19937 &  rng  )

static

Helper to sample minibatch indices.

setData()

template<typename V , typename M >

void cpu_mlp::SLBFGS< V, M >::setData ( const BatchLossFun &  f, const BatchGradFun &  g  )

inline

Configure data and callbacks for the solver.

Parameters

f	Batch loss function.
g	Batch gradient function.

setMaxIterations()

template<typename V , typename M >

void cpu_mlp::StochasticMinimizer< V, M >::setMaxIterations

inline

Sets the maximum number of iterations.

Parameters

max_iters

Limit on iterations.

setStepSize()

template<typename V , typename M >

void cpu_mlp::StochasticMinimizer< V, M >::setStepSize

inline

Sets the step size (learning rate).

Parameters

s	Step size.

setTolerance()

template<typename V , typename M >

void cpu_mlp::StochasticMinimizer< V, M >::setTolerance

inline

Sets the tolerance for convergence (full gradient norm).

Parameters

tol	Tolerance value.

stochastic_solve()

template<typename V , typename M >

V cpu_mlp::SLBFGS< V, M >::stochastic_solve	(	V	weights,
		const BatchLossFun &	f,
		const BatchGradFun &	batch_g,
		int	m,
		int	M_param,
		int	L,
		int	b,
		int	b_H,
		double	step_size,
		int	N
	)

Stochastic Solve using Batch Callbacks.

Parameters

weights	Initial parameter weights.
f	Loss function callback (evaluates loss on batch indices).
batch_g	Gradient function callback (computes gradient on batch indices).
m	Number of stochastic steps per epoch.
M_param	History size for L-BFGS pairs.
L	Hessian update interval.
b	Batch size for gradient steps.
b_H	Batch size for Hessian vector products.
step_size	Learning rate.
N	Total dataset size (used for indexing).

Returns

Optimized weights.

Here is the call graph for this function:

Member Data Documentation

_iters

template<typename V , typename M >

unsigned int cpu_mlp::StochasticMinimizer< V, M >::_iters

protected

_max_iters

template<typename V , typename M >

unsigned int cpu_mlp::StochasticMinimizer< V, M >::_max_iters

protected

_tol

template<typename V , typename M >

double cpu_mlp::StochasticMinimizer< V, M >::_tol

protected

step_size

template<typename V , typename M >

double cpu_mlp::StochasticMinimizer< V, M >::step_size

protected

---

The documentation for this class was generated from the following file:

• src/minimizer/s_lbfgs.hpp