12,751,690 members (33,647 online)
Rate this:
See more:
Hi guys,

I am trying to implement a neural network training algorithm in vb .net. I have found a open-source matlab code of this algorithm, so basically I am simply converting the matlab code to vb and adding some UIs.

Right now I have already finished the job (they are getting the same results if the same data was tested), but the program runs much much slower in vb than in matlab, like about 30 times! What I did is just to run a small function and to compare the running time.

Here are the codes of the function (VB .NET):
```Public Function CalculateError(ByVal input As Matrix, ByVal output As Matrix) As Double
Dim result As Double
result = 0
Dim Nm, Nn, Ni, No As Integer
Dim net As Double
Nm = input.mRow
Nn = CInt(parameters.value(2))
Ni = CInt(parameters.value(4))
No = CInt(parameters.value(5))

For p As Integer = 1 To Nm
Dim temp = New List(Of Double)
'compute the first Ni entries of temp
For i As Integer = 1 To Ni
Next

For n As Integer = 1 To Nn
net = weights.value(iw.value(n))
For i As Integer = iw.value(n) + 1 To iw.value(n + 1) - 1
'Make sure topology is a matrix with integer entries only
net += temp(topology.value(i) - 1) * weights.value(i)
Next

Next

For k As Integer = 1 To No
result += Math.Pow(output.value(p, 1) - temp(Nn + Ni - No + 1 - 1), 2)
Next

Next

' SSE
Return result
End Function```

and matlab:

```function [err] = calculate_error(input,output, topo, weight,param, iw, gain, act)
err = 0;
for p = 1:size(input, 1)     % number of patterns
temp(1:param(4)) = input(p,1:param(4));
for n = 1:param(2) % number of neurons
j = param(4) + n;
net = weight(iw(n));
for i = (iw(n)+1):(iw(n+1)-1)
net = net + temp(topo(i))*weight(i);
end;
tic
out=actFunc(n,net,act,gain);
toc
temp(j) = out;
end;
for k = 1:param(5)
err = err + (output(p,k)-temp(param(2)+param(4)-param(5)+k))^2;                % calculate total error
end;
end;```

The Matrix is a custom class I defined and it is implemented using Array; ActivationFunction is another simple function (I will attach the code also); and other variables like iw, weights are some internal variables, I don't think they really affect the runnning time much.

The code for ActivationFunction is:

```Public Function ActivationFunction(ByVal n As Integer, ByVal net As Double) As Double
'Dim result As Double
Select Case activations.value(n)
Case 0
Return gain.value(n) * net           'linear neuron
Case 1
Return 1.0 / (1.0 + System.Math.Exp(-1.0 * gain.value(n) * net))     'unipolar neuron
Case 2
Return 2.0 / (1.0 + System.Math.Exp(-1.0 * gain.value(n) * net)) - 1.0     'bipolar neuron
Case 3
Return (gain.value(n) * net) / (1.0 + gain.value(n) * System.Math.Abs(net))     'unipolar elliot neuron
Case 4
Return (2.0 * gain.value(n) * net) / (1.0 + gain.value(n) * System.Math.Abs(net)) - 1     'bipolar elliot neuron
End Select
End Function```

The problem I have now is that I tried to run these two CalculateError functions (one in vb. net and other in matlab), the vb code needs like 50 milliseconds, while the matlab code runs in 1.5 millisecond! BTW, even for the ActivationFunction (even simpler), matlab runs much faster.

Anyone can help me with this or any idea that why the VB code is that slow?

TIA.
Posted 24-Jul-12 16:47pm

The very first thing that I would question is how many iterations are you measuring? 1? 10? 1000? 1M? The other thing that I would question is how you are measuring the time?
Sergey Alexandrovich Kryukov 24-Jul-12 22:04pm

I just asked the same thing -- in my answer. I suspect OP measures just the JIT operation rather then the execution. Please see my answer.
--SA
Sergey Alexandrovich Kryukov 24-Jul-12 22:04pm

How exactly did you find it? Did you use System.Diagnostic.Stopwatch (you should)?
--SA
DonkeyBo 24-Jul-12 23:16pm

Thanks for the reply. This is my first time to post a question, I think I did not make the question clear enough. Sorry about that. I used, as SA said, System.Diagnosis.Stopwatch class to measure the time. The function I posted is only a very small portion of the whole algorithm, and the algorithm will call the function a lot when it is run. The elapsed time (1.5 and 50 ms) I mentioned in my question is just for one single run of the function (in vb and matlab). You can imagine how slow it will get when the whole algorithm is run. Hope I make myself clearer this time.
DonkeyBo 24-Jul-12 23:23pm

Thank you so much for your help. Please see my reply to Wes. What I actually did is I pulled out the function to another new project where I only call this function once and I measured the elapsed time using Stopwatch class. The reason I did not put the elapsed time for the whole algorithm in my question is because the algorithm will run like 10 minutes in VB vs. several seconds in Matlab when I used the exactly same input data, and the ratio of 10mins. / few seconds seems similar to the running time ratio of 50 ms / 1.5 ms (I did not calculate them though :)). So I only posted the single function to make my question look clearer. Did this reply answer your question?

Then I would say that you have measured the compile time from the JIT and you have not measured the actual execution time of the function.
DonkeyBo 25-Jul-12 10:38am

I ma not sure I understood what you said. Did you mean that if you only call some function once in my code and measure the elapsed time using a Stopwatch, the resulting time will be the JIT, not the exact execution time? I just tried to call the function twice and only measure the second call, the running time was almost the same as before. I just got another thought (maybe stupid to you), is that possible that since I declared this CalculateError function in a class as Public, every time you call this method from outside, the complier has to compile the function again? It is not likely to me neither, I am just trying to find some answers. Thanks
Sergey Alexandrovich Kryukov 25-Jul-12 12:07pm

No compilation happens again as soon as you execute a single process. As I tried to explain in my answer, your timing include JIT compilation, which happens only once per method, so you should time more accurately to exclude it. You should make sure that before you time some operations, all the method involved were already called at least once in the same process. Only then your timing could be correct.
--SA
Sergey Alexandrovich Kryukov 25-Jul-12 12:10pm

Yes, basically. Please see my answer and my previous comment above. It looks like they are applicable to your case. Please try to time accurately, the way JIT would not be included in measured time, as I explained above. It would be interesting to hear from you the results of corrected measurements.
--SA
DonkeyBo 25-Jul-12 18:42pm

Thans SA. The thing is that I tried call the function once, after which I time the second call of the function, but the elasped time was almost the same. Do you notice any other possibilities causing so long running time?
DonkeyBo 25-Jul-12 19:23pm

Hi SA, it just occured to me, do you think it is helpful if it pass the input arguments ByRef instead of ByVal (sorry I don't have access to my machine right now so I can only guess :))
Sergey Alexandrovich Kryukov 27-Jul-12 16:51pm

For performance, it's helpful to use by ref, but only for struct types. For primitive types, enumeration types it's only needed if you need to modify the value and return modified value, for reference types it's totally pointless in all cases (the parameters are already references). For any types except structures, using By Ref without purpose can actually make performance only worse.
--SA
Sergey Alexandrovich Kryukov 27-Jul-12 16:52pm

So, how about correct timing and the timing results?
--SA
DonkeyBo 27-Jul-12 17:01pm

I measured the time for the second call of the function, it made no much differences. However, I found some other issues about my implementation (maybe I did not put them here). All those variables, weights, topology and etc, I declared them as Matrix and internally they are two-dimensional Arrays (I used Matrix since there are some matrices operations in the algorithm somewhere else; but in fact, most of them are only vectors). I changed all of them to one-dimensional Arrays and used jagged Array instead of two-dimensional Array when needed. This simple function now runs faster than in Matlab, but the whole algorithm still runs slower, because I still have to implement some matrices operations (product, addition and etc) myself, which it will always be much faster in Matlab. Although slower than in Matlab, the algorithm runs 2-3 times faster than before, and thank you for your kindly helps. :) A lesson learned. :)
Sergey Alexandrovich Kryukov 27-Jul-12 17:10pm

Strange... You can also repeat calculations many times and calculate the average time...
--SA

Rate this:

## Solution 1

It's hard to say how fast is your implementation of the algorithm by just looking at it, but… I can give you just one idea.

Did you run all the algorithm methods twice? How did you time it?

Here is the thing: if you call some time-consuming method only once, it could be possible that you time not the method execution, but… its compilation. If so, there would not be anything wrong or bad about it: first time does not count, because — who would care much about some 50 milliseconds, it they are spent only for the first time? Run it all again in the same process and time the performance again. Please share with us your findings.

Such things can happen, because .NET is based in JIT (Just-In-Time) compilation, and usually it happens on per-method basis: the method is compiled just before it's called for the very first time. Please see:
http://en.wikipedia.org/wiki/Just-in-time_compilation[^].

—SA
Rate this:

## Solution 2

First, don't use a List(Of Double) when an array will do. You're creating a list of a known size that doesn't change size during the loop, so an array will be more appropriate and faster. If the list does change size, try to compute the maxiumum size of the array first and allocate the array ahead of time.

You're also populating the list NM times when it imput values never appear to change. This means you're doing the exact same thing over and over again NM times when you only need to do it once.
DonkeyBo 25-Jul-12 10:34am

Thanks. However, I did try the array implementation, the running time did not change. How this CalculateError function work is to iterate every row of the input and compute the sum of square error. Thanks.
Dave Kreskowiak 25-Jul-12 11:55am

OK, and that loop where you're copying the values of the input.values() array?? You're spending a lot of time in this loop copying individual values. Instead, look for a way to structure your loop so you're not doing this copy operation at all, or can just do it once into a temp array and then work on the temp array data. Avoid copying data whenever you can.
Sergey Alexandrovich Kryukov 25-Jul-12 12:10pm

Good point, a 5.
--SA
DonkeyBo 25-Jul-12 18:20pm

Yes, I noticed that also. I tried what you said (use the input value directly instead of copying them to a list) also and it took the exactly same time...
DonkeyBo 25-Jul-12 18:45pm

I don't want to get details of the algorithm. But to be more precise, since basically, the temp list will contain a fixed (known ahead of time) length, so I tried to store the computed values only in the temp and use the input values directly without copying them.
Rate this:

## Solution 3

Variables in my implementation use two-dimensional Arrays. Change them into one-dimensional Arrays and jagged Arrays (which microsoft recommends) makes the function run much faster.

Top Experts
Last 24hrsThis month
 Graeme_Grant 328 Karthik Bangalore 305 Maciej Los 145 Jochen Arndt 105 OriginalGriff 100
 OriginalGriff 3,933 Peter Leow 2,891 Karthik Bangalore 2,330 ppolymorphe 2,107 Richard MacCutchan 1,484