Question

I want to use this article for neural network construction, but I meet some problems with the update algorithm of the weight vectors. Specifically, with formulas marked red.
Can anybody help me to understand, what is the hm(i) and the symbol “|” means?

Answer 1

This looks like the back-propagation computation for the gradient of the training error of a neural network. Bishop (on page 244) lists a key formula as:

δj = h'(aj) SUM(k, wkj δk)

The δ are the errors between the predicted and labeled values of the hidden or output nodes. The δ terms on the right side have been already calculated, and correspond to the next layer output-ward from the one being considered.

The h' term is the derivative of the non-linear activation function, which is typically the sigmoid function or tanh. The listed hm in your image looks like the derivative of tanh with a change of variables.

The vertical bar is a syntax for evaluation: f(t) = f(x) | t. I can’t quite tell what the expression in your image is; I could be wrong.