Autograd

An AutoGrad engine (also known as reverse mode automatic differentiation), is a tool used in machine learning to automatically compute the gradients of functions by applying the chain rule in reverse order through the computation graph.

It works by recording the operations performed on tensors during the forward pass of a computation. When the backward pass is initiated, the engine uses the recorded operations to apply the chain rule of calculus, efficiently computing the derivatives of the output with respect to the inputs or parameters.

With AutoGrad, one only needs to define the loss function of the model, without having to manually define its gradient function.