Activation Functions
The activation ops provide different types of nonlinearities for use in neural networks. These include smooth nonlinearities (sigmoid, tanh, elu, selu, softplus, and softsign), continuous but not everywhere differentiable functions (relu, relu6, crelu and relu_x), and random regularization (dropout). All activation ops apply componentwise, and produce a tensor of the same shape as the input tensor.
Contents
tanh (hyperbolic tangent)
ReLU (Rectified Linear Unit)
Leaky ReLU (Rectified Linear Unit)
Softmax
Sigmoid
Threshold (binary step) ...not offered in TensorFlow library
Piecewise
Identity (linear)
