Click here to Skip to main content
Rate this: bad
good
Please Sign up or sign in to vote.
See more: VB.NET Matlab Runtime
Hi guys,
 
I am trying to implement a neural network training algorithm in vb .net. I have found a open-source matlab code of this algorithm, so basically I am simply converting the matlab code to vb and adding some UIs.
 
Right now I have already finished the job (they are getting the same results if the same data was tested), but the program runs much much slower in vb than in matlab, like about 30 times! What I did is just to run a small function and to compare the running time.
 
Here are the codes of the function (VB .NET):
Public Function CalculateError(ByVal input As Matrix, ByVal output As Matrix) As Double
        Dim result As Double
        result = 0
        Dim Nm, Nn, Ni, No As Integer
        Dim net As Double
        Nm = input.mRow
        Nn = CInt(parameters.value(2))
        Ni = CInt(parameters.value(4))
        No = CInt(parameters.value(5))
 
        For p As Integer = 1 To Nm
            Dim temp = New List(Of Double)
            'compute the first Ni entries of temp
            For i As Integer = 1 To Ni
                temp.Add(input.value(p, i))
            Next
 
            For n As Integer = 1 To Nn
                net = weights.value(iw.value(n))
                For i As Integer = iw.value(n) + 1 To iw.value(n + 1) - 1
                    'Make sure topology is a matrix with integer entries only
                    net += temp(topology.value(i) - 1) * weights.value(i)
                Next
 
                temp.Add(ActivationFunction(n, net))
                'temp.Add(0.0)
            Next
 

 
            For k As Integer = 1 To No
                result += Math.Pow(output.value(p, 1) - temp(Nn + Ni - No + 1 - 1), 2)
            Next
 
        Next
 

        ' SSE
        Return result
    End Function

 
and matlab:
 
function [err] = calculate_error(input,output, topo, weight,param, iw, gain, act)
err = 0;
for p = 1:size(input, 1)     % number of patterns
   temp(1:param(4)) = input(p,1:param(4));
   for n = 1:param(2) % number of neurons
      j = param(4) + n;
      net = weight(iw(n));
      for i = (iw(n)+1):(iw(n+1)-1)
         net = net + temp(topo(i))*weight(i);
      end;
      tic
      out=actFunc(n,net,act,gain);
      toc
      temp(j) = out;
   end;
   for k = 1:param(5)
      err = err + (output(p,k)-temp(param(2)+param(4)-param(5)+k))^2;                % calculate total error
   end;
end;
 
The Matrix is a custom class I defined and it is implemented using Array; ActivationFunction is another simple function (I will attach the code also); and other variables like iw, weights are some internal variables, I don't think they really affect the runnning time much.
 
The code for ActivationFunction is:
 
Public Function ActivationFunction(ByVal n As Integer, ByVal net As Double) As Double
        'Dim result As Double
        Select Case activations.value(n)
            Case 0
                Return gain.value(n) * net           'linear neuron
            Case 1
                Return 1.0 / (1.0 + System.Math.Exp(-1.0 * gain.value(n) * net))     'unipolar neuron
            Case 2
                Return 2.0 / (1.0 + System.Math.Exp(-1.0 * gain.value(n) * net)) - 1.0     'bipolar neuron
            Case 3
                Return (gain.value(n) * net) / (1.0 + gain.value(n) * System.Math.Abs(net))     'unipolar elliot neuron
            Case 4
                Return (2.0 * gain.value(n) * net) / (1.0 + gain.value(n) * System.Math.Abs(net)) - 1     'bipolar elliot neuron
        End Select
    End Function
 
The problem I have now is that I tried to run these two CalculateError functions (one in vb. net and other in matlab), the vb code needs like 50 milliseconds, while the matlab code runs in 1.5 millisecond! BTW, even for the ActivationFunction (even simpler), matlab runs much faster.
 
Anyone can help me with this or any idea that why the VB code is that slow?
 
TIA.
Posted 24-Jul-12 16:47pm
Comments
Wes Aday at 24-Jul-12 21:54pm
   
The very first thing that I would question is how many iterations are you measuring? 1? 10? 1000? 1M? The other thing that I would question is how you are measuring the time?
Sergey Alexandrovich Kryukov at 24-Jul-12 22:04pm
   
I just asked the same thing -- in my answer. I suspect OP measures just the JIT operation rather then the execution. Please see my answer.
--SA
DonkeyBo at 24-Jul-12 23:16pm
   
Thanks for the reply. This is my first time to post a question, I think I did not make the question clear enough. Sorry about that. I used, as SA said, System.Diagnosis.Stopwatch class to measure the time. The function I posted is only a very small portion of the whole algorithm, and the algorithm will call the function a lot when it is run. The elapsed time (1.5 and 50 ms) I mentioned in my question is just for one single run of the function (in vb and matlab). You can imagine how slow it will get when the whole algorithm is run. Hope I make myself clearer this time.
Wes Aday at 25-Jul-12 7:31am
   
Then I would say that you have measured the compile time from the JIT and you have not measured the actual execution time of the function.
DonkeyBo at 25-Jul-12 10:38am
   
I ma not sure I understood what you said. Did you mean that if you only call some function once in my code and measure the elapsed time using a Stopwatch, the resulting time will be the JIT, not the exact execution time? I just tried to call the function twice and only measure the second call, the running time was almost the same as before. I just got another thought (maybe stupid to you), is that possible that since I declared this CalculateError function in a class as Public, every time you call this method from outside, the complier has to compile the function again? It is not likely to me neither, I am just trying to find some answers. Thanks
Sergey Alexandrovich Kryukov at 25-Jul-12 12:07pm
   
No compilation happens again as soon as you execute a single process. As I tried to explain in my answer, your timing include JIT compilation, which happens only once per method, so you should time more accurately to exclude it. You should make sure that before you time some operations, all the method involved were already called at least once in the same process. Only then your timing could be correct.
--SA
DonkeyBo at 25-Jul-12 18:42pm
   
Thans SA. The thing is that I tried call the function once, after which I time the second call of the function, but the elasped time was almost the same. Do you notice any other possibilities causing so long running time?
Sergey Alexandrovich Kryukov at 24-Jul-12 22:04pm
   
How exactly did you find it? Did you use System.Diagnostic.Stopwatch (you should)?
--SA
DonkeyBo at 24-Jul-12 23:23pm
   
Thank you so much for your help. Please see my reply to Wes. What I actually did is I pulled out the function to another new project where I only call this function once and I measured the elapsed time using Stopwatch class. The reason I did not put the elapsed time for the whole algorithm in my question is because the algorithm will run like 10 minutes in VB vs. several seconds in Matlab when I used the exactly same input data, and the ratio of 10mins. / few seconds seems similar to the running time ratio of 50 ms / 1.5 ms (I did not calculate them though :)). So I only posted the single function to make my question look clearer. Did this reply answer your question?
Sergey Alexandrovich Kryukov at 25-Jul-12 12:10pm
   
Yes, basically. Please see my answer and my previous comment above. It looks like they are applicable to your case. Please try to time accurately, the way JIT would not be included in measured time, as I explained above. It would be interesting to hear from you the results of corrected measurements.
--SA
DonkeyBo at 25-Jul-12 19:23pm
   
Hi SA, it just occured to me, do you think it is helpful if it pass the input arguments ByRef instead of ByVal (sorry I don't have access to my machine right now so I can only guess :))
Sergey Alexandrovich Kryukov at 27-Jul-12 16:51pm
   
For performance, it's helpful to use by ref, but only for struct types. For primitive types, enumeration types it's only needed if you need to modify the value and return modified value, for reference types it's totally pointless in all cases (the parameters are already references). For any types except structures, using By Ref without purpose can actually make performance only worse.
--SA
Sergey Alexandrovich Kryukov at 27-Jul-12 16:52pm
   
So, how about correct timing and the timing results?
--SA
DonkeyBo at 27-Jul-12 17:01pm
   
I measured the time for the second call of the function, it made no much differences. However, I found some other issues about my implementation (maybe I did not put them here). All those variables, weights, topology and etc, I declared them as Matrix and internally they are two-dimensional Arrays (I used Matrix since there are some matrices operations in the algorithm somewhere else; but in fact, most of them are only vectors). I changed all of them to one-dimensional Arrays and used jagged Array instead of two-dimensional Array when needed. This simple function now runs faster than in Matlab, but the whole algorithm still runs slower, because I still have to implement some matrices operations (product, addition and etc) myself, which it will always be much faster in Matlab. Although slower than in Matlab, the algorithm runs 2-3 times faster than before, and thank you for your kindly helps. :) A lesson learned. :)
Sergey Alexandrovich Kryukov at 27-Jul-12 17:10pm
   
Strange... You can also repeat calculations many times and calculate the average time...
--SA
Rate this: bad
good
Please Sign up or sign in to vote.

Solution 1

It's hard to say how fast is your implementation of the algorithm by just looking at it, but… I can give you just one idea.
 
Did you run all the algorithm methods twice? How did you time it?
 
Here is the thing: if you call some time-consuming method only once, it could be possible that you time not the method execution, but… its compilation. If so, there would not be anything wrong or bad about it: first time does not count, because — who would care much about some 50 milliseconds, it they are spent only for the first time? Run it all again in the same process and time the performance again. Please share with us your findings.
 
Such things can happen, because .NET is based in JIT (Just-In-Time) compilation, and usually it happens on per-method basis: the method is compiled just before it's called for the very first time. Please see:
http://en.wikipedia.org/wiki/Just-in-time_compilation[^].
 
—SA
  Permalink  
Rate this: bad
good
Please Sign up or sign in to vote.

Solution 2

First, don't use a List(Of Double) when an array will do. You're creating a list of a known size that doesn't change size during the loop, so an array will be more appropriate and faster. If the list does change size, try to compute the maxiumum size of the array first and allocate the array ahead of time.
 
You're also populating the list NM times when it imput values never appear to change. This means you're doing the exact same thing over and over again NM times when you only need to do it once.
  Permalink  
Comments
DonkeyBo at 25-Jul-12 10:34am
   
Thanks. However, I did try the array implementation, the running time did not change. How this CalculateError function work is to iterate every row of the input and compute the sum of square error. Thanks.
Dave Kreskowiak at 25-Jul-12 11:55am
   
OK, and that loop where you're copying the values of the input.values() array?? You're spending a lot of time in this loop copying individual values. Instead, look for a way to structure your loop so you're not doing this copy operation at all, or can just do it once into a temp array and then work on the temp array data. Avoid copying data whenever you can.
DonkeyBo at 25-Jul-12 18:20pm
   
Yes, I noticed that also. I tried what you said (use the input value directly instead of copying them to a list) also and it took the exactly same time...
DonkeyBo at 25-Jul-12 18:45pm
   
I don't want to get details of the algorithm. But to be more precise, since basically, the temp list will contain a fixed (known ahead of time) length, so I tried to store the computed values only in the temp and use the input values directly without copying them.
Sergey Alexandrovich Kryukov at 25-Jul-12 12:10pm
   
Good point, a 5.
--SA
Rate this: bad
good
Please Sign up or sign in to vote.

Solution 3

Variables in my implementation use two-dimensional Arrays. Change them into one-dimensional Arrays and jagged Arrays (which microsoft recommends) makes the function run much faster.
  Permalink  

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

  Print Answers RSS
0 OriginalGriff 350
1 Jochen Arndt 190
2 Richard MacCutchan 135
3 Sergey Alexandrovich Kryukov 130
4 DamithSL 105
0 OriginalGriff 6,045
1 DamithSL 4,601
2 Maciej Los 4,087
3 Kornfeld Eliyahu Peter 3,480
4 Sergey Alexandrovich Kryukov 3,310


Advertise | Privacy | Mobile
Web01 | 2.8.141220.1 | Last Updated 27 Jul 2012
Copyright © CodeProject, 1999-2014
All Rights Reserved. Terms of Service
Layout: fixed | fluid

CodeProject, 503-250 Ferrand Drive Toronto Ontario, M3C 3G8 Canada +1 416-849-8900 x 100