Question

The context:

I’m experimenting with using a feed-forward artificial neural network to create AI for a video game, and I’ve run into the problem that some of my input features are dependent upon the existence or value of other input features.

The most basic, simplified example I can think of is this:

feature 1 is the number of players (range 2…5)
feature 2 to ? is the score of each player (range >=0)

The number of features needed to inform the ANN of the scores is dependent on the number of players.

The question: How can I represent this dynamic knowledge input to an ANN?

Things I’ve already considered:

1. Simply not using such features, or consolidating them into static input.
   I.E using the sum of the players scores instead. I seriously doubt this is applicable to my problem, it would result in the loss of too much information and the ANN would fail to perform well.

2. Passing in an error value (eg -1) or default value (eg 0) for non-existant input
   I’m not sure how well this would work, in theory the ANN could easily learn from this input and model the function appropriately. In practise I’m worried about the sheer number of non-existant input causing problems for the ANN. For example if the range of players was 2-10, if there were only 2 players, 80% of the input data would be non-existant and would introduce weird bias into the ANN resulting in a poor performance.

3. Passing in the mean value over the training set in place on non-existant input
   Again, the amount of non-existant input would be a problem, and I’m worried this would introduce weird problems for discrete-valued inputs.

So, I’m asking this, does anybody have any other solutions I could think about? And is there a standard or commonly used method for handling this problem?

I know it’s a rather niche and complicated question for SO, but I was getting bored of the “how do I fix this code?” and “how do I do this in PHP/Javascript?” questions :P, thanks guys.

Answer 1

Try thinking about some model like the following:

Say xi (e.g. x1) is one of the inputs that a variable number of can exist. You can have n of these (x1 to xn). Let y be the rest of the inputs.

On your first hidden layer, pass x1 and y to the first c nodes, x1,x2 and y to the next c nodes, x1,x2,x3 and y to the next c nodes, and so on. This assumes x1 and x3 can’t both be active without x2. The model will have to change appropriately if this needs to be possible.

The rest of the network is a standard feed-forward network with all nodes connected to all nodes of the next layer, or however you choose.

Whenever you have w active inputs, disable all but the wth set of c nodes (completely exclude them from training for that input set, don’t include them when calculating the value for the nodes they output to, don’t update the weights for their inputs or outputs). This will allow most of the network to train, but for the first hidden layer, only parts applicable to that number of inputs.

I suggest c is chosen such that c*n (the number of nodes in the first hidden layer) is greater than (or equal to) the number of nodes in the 2nd hidden layer (and have c be at the very least 10 for a moderately sized network (into the 100s is also fine)) and I also suggest the network have at least 2 other hidden layers (so 3 in total excluding input and output). This is not from experience, but just what my intuition tells me.

This working is dependent on a certain (possibly undefinable) similarity between the different numbers of inputs, and might not work well, if at all, if this similarity doesn’t exist. This also probably requires quite a bit of training data for each number of inputs.

If you try it, let me / us know if it works.

If you’re interested in Artificial Intelligence discussions, I suggest joining some Linked-In group dedicated to it, there are some that are quite active and have interesting discussions. There doesn’t seem to be much happening on stackoverflow when it comes to Artificial Intelligence, or maybe we should just work to change that, or both.

UPDATE:

Here is a list of the names of a few decent Artificial Intelligence LinkedIn groups (unless they changed their policies recently, it should be easy enough to join):

• ‘Artificial Intelligence Researchers, Faculty + Professionals’

• ‘Artificial Intelligence Applications’

• ‘Artificial Neural Networks’

• ‘AGI — Artificial General Intelligence’

• ‘Applied Artificial Intelligence’ (not too much going on at the moment, and still dealing with some spam, but it is getting better)

• ‘Text Analytics’ (if you’re interested in that)