I can't get my feedforward net to work. it's a single layer one with a single neuron...
are any of these equations wrong?
output = sum of all (inputs x corresponding weight)
error = desired output - neuron output
weight change => new weight = old weight + learningrate*error*input for that synapse
if not maybe you can see whats wrong when i go through two patterns of 3 inputs each time:
Data | Weight | Output | Desired | Error | New Weight
334.6 | 0.00125126 | 257.953 | 342.2 | 84.247 | 281.892
340.1 | 0.563585 |
340.7 | 0.193304 |
340 | 281.892 | 293,693 ..... making weights rediculous and the program error
342 | 287.088 | hitting #INF fast because it cant handle it....
347 | 287.223 |
The problem is that your activation function (i. e. the formula for calculating the output) is a linear function. Because of that, no matter what weights you set, the entire neural network of yours is simply a linear function and can never approximate more than two target values at once.
Basically the best you can achieve is linear regression, but then there is no point for using NNs for linear regression since you can directly solve that a lot faster. (and anyway your correction formula is wrong). The main point of NNs is that they can solve nonlinear problems with the right type of activation function:
What you need is a nonlinear activation function, and a correction formula specifically suited to that function. Backpropagation uses a simple sigmoid function that allows for a simple error correction formula, but isn't so hard to use different functions with similar results.
There are various articles on CP about Backpropagation and other types of nueral networks, I suggest you hit search and see what helps you best.
You didn't mention the neural network algorithm. This actually looks like backpropagation network (since you calculate error).
In any case (either feed forward or backpropagation):
output = f( sum of all (inputs x corresponding weight) )
where, f() is the activation function.
In binary neural nets (e.g. Hopfield net), f() is a step function
In continuous nets, f() is a sigmoid function such as tanh(x)
This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)