Question

Would I be right in saying a neural network are good at finding ‘good enough’ solutions for a problem?

I’m thinking this because they don’t provide a binary output for an given input but a probability, for example 0.67 could be an output.

I’m also guessing because they’re often used for generalisation they’re good at find solutions that often solve the problem but in some rare cases won’t.

Thank you!

Answer 1

There is no easy answer to this question. The advantages/disadvantages of neural networks are a very complex topic. Here are some pointers:

• No free lunch theorem: Roughly stated, this theorem proves that there is no “perfect” machine learning method. For every problem, for which a certain method is good, there is another problem for which the same method will fail horribly. The methods at which it fails may be solved by other methods quite easily however. This should always be considered, when doing any machine learning.

• Neural networks are quite simple to implement (you do not need a good linear algebra solver as for examples for SVNs).

• The VC dimension of neural networks is unclear. This is very important when you want to consider how good a solution might be.

• Neural networks cannot be retrained. If you add data later, this is almost impossible to add to an existing network.

• Neural networks often exhibit patterns similar to those exhibited by humans. However this is more of interest in cognitive sciences than for practical examples.

• Handling of time series data in neural networks is a very complicated topic.

This is all I can think of for now. Maybe other can add more.