Understanding Softmax as an Activation Function in Deep Learning
In the case of multi-class classification problems, the softmax function may be used as the activation function. Based on the inputs, the softmax function returns a probability for each of the possible classes. Here are two implementations of the softmax function in Python (one very concise, the other verbose to clarify what is happening):
Sample code for Softmax as an Activation Function
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| #Concise implementation | |
| def softmax(inputs): | |
| return np.exp(inputs) / np.sum(np.exp(inputs)) | |
| #Verbose implementation (for clarification/understanding) | |
| def softmax_verbose(inputs): | |
| results = [np.exp(x) for x in inputs] | |
| summation = sum(results) | |
| probabilities = [(x / summation) for x in results] | |
| return probabilities |
Output values are between 0 and 1 and are normalized to sum a to a value of 1, which means that the result is basically a categorical probability distribution. For example, if the input to our softmax function is [2.0, 1.0, 0.1], the output will be [ 0.7, 0.2 , 0.1] (rounded) indicating a 70% chance of the first class, a 20% chance of the second class, and a 10% chance of the third class.
Implementations of the softmax function are available in a number deep learning libraries, including TensorFlow.
Keep up the great work, I read few blog posts on this site and I believe that your website is really interesting and has loads of good info. Lovely blog ..! I really enjoyed reading this article. keep it up!!
Softmax Activation Function Python
Best Artificial Intelligence & Data Science In Pune With Placement