Stack Exchange network consists of 183 Q&A communities including Stack Overflow, the largest, most trusted online community for developers to learn, share their knowledge, and build their careers.
Visit Stack Exchange
Stack Internal
Knowledge at work
Bring the best of human thought and AI automation together at your work.
Explore Stack InternalMy question is that what will happen if iI arrange different activation functions in the same layer of a neural network and continue the same trend for the other hidden layers.
Suppose iI have 3 relurelu units in the starting and 3 tanhtanh after that and other activation function iI the same hidden layer and for the other hidden layers iI am scaling all the nodes with the same scale (decreasing/increasing) and the arrangement and order of the activation function is not changing.
My question is that what will happen if i arrange different activation functions in the same layer of a neural network and continue the same trend for the other hidden layers.
Suppose i have 3 relu units in the starting and 3 tanh after that and other activation function i the same hidden layer and for the other hidden layers i am scaling all the nodes with the same scale(decreasing/increasing) and the arrangement and order of the activation function is not changing.
My question is that what will happen if I arrange different activation functions in the same layer of a neural network and continue the same trend for the other hidden layers.
Suppose I have 3 relu units in the starting and 3 tanh after that and other activation function I the same hidden layer and for the other hidden layers I am scaling all the nodes with the same scale (decreasing/increasing) and the arrangement and order of the activation function is not changing.
My question is that what will happen if i arrange different activation functions in the same layer of a neural network and continue the same trend for the other hidden layers.
Suppose i have 3 relu units in the starting and 3 tanh after that and other activation function i the same hidden layer and for the other hidden layers i am scaling all the nodes with the same scale(decreasing/increasing) and the arrangement and order of the activation function is not changing.
