I was wondering what was the difference between Activation Layer and Dense layer in Keras.
Since Activation Layer seems to be a fully connected layer, and Dense have a parameter to pass an activation function, what is the best practice ?
Let's imagine a fictionnal network like this : Input -> Dense -> Dropout -> Final Layer Final Layer should be : Dense(activation=softmax) or Activation(softmax) ? What is the cleanest and why ?
Using Dense(activation=softmax) is computationally equivalent to first add Dense and then add Activation(softmax). However there is one advantage of the second approach - you could retrieve the outputs of the last layer (before activation) out of such defined model. In the first approach - it's impossible.
References:
https://stackoverflow.com/questions/40866124/difference-between-dense-and-activation-layer-in-keras#:~:text=Using%20Dense(activation%3Dsoftmax),the%20first%20approach%20%2D%20it's%20impossible.
No comments:
Post a Comment