cpu_mlp::StochasticGradientDescent< V, M > Class Template Reference

Stochastic Gradient Descent (SGD) minimizer. More...

#include <s_gd.hpp>

Inheritance diagram for cpu_mlp::StochasticGradientDescent< V, M >:

Collaboration diagram for cpu_mlp::StochasticGradientDescent< V, M >:

Public Types
using	S_VecFun = std::function< double(const V &, const V &, const V &)>
using	BatchGradFun = std::function< void(const V &, const std::vector< size_t > &, V &)>

Public Member Functions
	StochasticGradientDescent ()=default
	Default constructor. More...
void	setData (const M &inputs, const M &targets, const S_VecFun &f, const BatchGradFun &g)
	Configure data and callbacks for the solver. More...
V	stochastic_solve (const V &init_w, int m, int b, double step, bool verbose=false, int print_every=50)
	Execute Stochastic Solve. More...
void	setMaxIterations (int max_iters)
	Sets the maximum number of iterations. More...
void	setStepSize (double s)
	Sets the step size (learning rate). More...
void	setTolerance (double tol)
	Sets the tolerance for convergence (full gradient norm). More...
Public Member Functions inherited from cpu_mlp::StochasticMinimizer< V, M >
virtual	~StochasticMinimizer ()=default
void	setMaxIterations (int max_iters)
	Sets the maximum number of iterations. More...
void	setStepSize (double s)
	Sets the step size (learning rate). More...
void	setTolerance (double tol)
	Sets the tolerance for convergence (full gradient norm). More...
void	setRecorder (::IterationRecorder< CpuBackend > *recorder)
	Attach a recorder for loss/grad history. More...

Static Public Member Functions
static std::vector< size_t >	sample_minibatch_indices (const size_t N, size_t batch_size, std::mt19937 &rng)
	Helper to sample minibatch indices without replacement (Fisher-Yates shuffle subset). More...

Additional Inherited Members
Protected Attributes inherited from cpu_mlp::StochasticMinimizer< V, M >
unsigned int	_max_iters = 1000
unsigned int	_iters = 0
double	_tol = 1e-4
double	step_size = 0.01
::IterationRecorder< CpuBackend > *	recorder_ = nullptr
	Optional recorder for diagnostics. More...

Detailed Description

template<typename V, typename M>
class cpu_mlp::StochasticGradientDescent< V, M >

Stochastic Gradient Descent (SGD) minimizer.

Operates on minibatches provided by callbacks.

Member Typedef Documentation

BatchGradFun

template<typename V , typename M >

using cpu_mlp::StochasticGradientDescent< V, M >::BatchGradFun = std::function<void(const V &, const std::vector<size_t> &, V &)>

S_VecFun

template<typename V , typename M >

using cpu_mlp::StochasticGradientDescent< V, M >::S_VecFun = std::function<double(const V &, const V &, const V &)>

Constructor & Destructor Documentation

StochasticGradientDescent()

template<typename V , typename M >

cpu_mlp::StochasticGradientDescent< V, M >::StochasticGradientDescent ( )

default

Default constructor.

Member Function Documentation

sample_minibatch_indices()

template<typename V , typename M >

std::vector< size_t > cpu_mlp::StochasticGradientDescent< V, M >::sample_minibatch_indices ( const size_t  N, size_t  batch_size, std::mt19937 &  rng  )

static

Helper to sample minibatch indices without replacement (Fisher-Yates shuffle subset).

Here is the caller graph for this function:

setData()

template<typename V , typename M >

void cpu_mlp::StochasticGradientDescent< V, M >::setData ( const M &  inputs, const M &  targets, const S_VecFun &  f, const BatchGradFun &  g  )

inline

Configure data and callbacks for the solver.

setMaxIterations()

template<typename V , typename M >

void cpu_mlp::StochasticMinimizer< V, M >::setMaxIterations

inline

Sets the maximum number of iterations.

Parameters

max_iters

Limit on iterations.

setStepSize()

template<typename V , typename M >

void cpu_mlp::StochasticMinimizer< V, M >::setStepSize

inline

Sets the step size (learning rate).

Parameters

s	Step size.

setTolerance()

template<typename V , typename M >

void cpu_mlp::StochasticMinimizer< V, M >::setTolerance

inline

Sets the tolerance for convergence (full gradient norm).

Parameters

tol	Tolerance value.

stochastic_solve()

template<typename V , typename M >

V cpu_mlp::StochasticGradientDescent< V, M >::stochastic_solve ( const V &  init_w, int  m, int  b, double  step, bool  verbose = false, int  print_every = 50  )

inline

Execute Stochastic Solve.

Parameters

init_w	Initial weights.
m	Number of minibatches per epoch.
b	Batch size.
step	Step size/learning rate.
verbose	Print progress.
print_every	Print interval.

Returns

Optimized weights.

Here is the call graph for this function:

---

The documentation for this class was generated from the following file:

• src/minimizer/s_gd.hpp