Question

Silva and Almeida’s algorithm improves on the existing backpropagation algorithm by introducing individual, adaptive learning-rates for each weight. The value for the new learning rate is computed as follows:

I read that the constants u and d are set to be u > 1 and d < 1. Those constraints are rather broad, so are there any general guidelines for setting these values or do I have to figure it out by experimentation for my specific problem?

Answer 1

I have read that good “starting” values to fit most problems are to try u = 1.2 and d = 0.8 but i can’t find the source right now.

Edit: I found it, PDF page 10-11

Also note the comments about how to improve upon the algorithm by introducing a momentum term, if you don’t have that already…