Question

I build this example using pybrain:

from pybrain.tools.shortcuts import buildNetwork
from pybrain.datasets import SupervisedDataSet
from pybrain.supervised.trainers import BackpropTrainer

net = buildNetwork(3, 3, 1)

dataSet = SupervisedDataSet(3, 1)
dataSet.addSample((0, 0, 0), (0))
dataSet.addSample((1, 1, 1), (0))
dataSet.addSample((1, 0, 0), (1))
dataSet.addSample((0, 1, 0), (1))
dataSet.addSample((0, 0, 1), (1))

trainer = BackpropTrainer(net, dataSet)

trainer.trainUntilConvergence()

result = net.activate([0, 0, 0])
print result

Output is: [ 0.10563189]

I don’t understand what is output of activate(). Network is trained, I test it for output with one of train samples, so I expect value exactly like in train samples. Input [0, 0, 0] should get output 0. What am I missing here? How do I get valid result?

Even more confusing is, that every time I run this code, I get different result. I’m obviously doing something wrong. What is it?

Answer 1

Training a network until convergence does not imply that the training set is remembered perfectly. There are many reasons: size of hidden layer, activation function, learning rate, etc. All these parameters need to be tuned.