Activation and Norms
Activation / Non-linear function A non-linear function is one in which cannot be expressed from y = mx + c lets suppose a function with y = x * W1 * W2 .. Wl Linearity with respect to x, this means rest of the variable are constant and only x is changing , so linearity with respect to x is linear Linearity with respect to Wi, this means rest of the variable (x here) is constant and only weights are changing , so linearity with respect to W is non-linear ...