Loss or Cost function

A loss function or cost function is a function that maps an event or values of one or more variables onto a real number intuitively representing some "cost" associated with the event.

An optimization problem seeks to minimize a loss function. The form of loss function is chosen based on the nature of the problem and mathematical needs.

The following are the different loss functions for different scenarios:

• Binary classification: binary cross entropy loss function.

• Multi-class classification: multi cross entropy loss function.

• Regression: squared error function.