Question

Here is my code (for a neuron in a MLP network):

double summation = 0;
for (int i = 0; i < weights.length; i++) {
    summation += inputs[i] * weights[i];
}

double normalized = Math.tanh(summation);
if (normalized > 0.9 || normalized < -0.9) {
    activated = 1;
} else {
    activated = 0;
}

I think it is incorrect. Is the output supposed to be the normalized value, or is it always limited to 0 or 1?

Answer 1

You could simply use the sign of the output, but normally, the output of a neuron is required to be continuous and differentiable, so a real-value between -1 and 1 (since you’ve chosen the tanh function) would be more appropriate, especially if you are going to train the model using backpropagation.