Optimizer

class Optimizer(schedulers=None)

The base class for all optimizers. It allows us to implement an optimizer on a per-parameter basis and apply it to a parameter tree.

Attributes

Optimizer.state

The optimizer's state dictionary.

Methods

Optimizer.apply_gradients(gradients, parameters)	Apply the gradients to the parameters and return the updated parameters.
Optimizer.init(parameters)	Initialize the optimizer's state
Optimizer.update(model, gradients)	Apply the gradients to the parameters of the model and update the model with the new parameters.