Click here to Skip to main content
14,268,557 members
Rate this:
Please Sign up or sign in to vote.
See more:
Thanks to Richard I was able to make it one step further and incorporated his additional coding into my current project.

Public Sub Match_Merge(I As Long)


Is the method I am call for the parallel for.

inside this method I am comparing my temp table which is my import table and by current database table.

For I = 1 To temptable.Rows.Count - 1

                    countI = countI + 1
                    'this will be table 2

                    For I2 = I3 To dt.Rows.Count - 1
Next I
Next i2


if there is not a match then it the information goes to
removeduptable.ImportRow(temptable.Rows(I))


if there a match then the information is stored in a separate table.
duplicaterow = Duplicatetable.NewRow()
                            duplicaterow.Item("First_name") = dt.Rows(I2).Item("First_name")
                            duplicaterow.Item("Middle_name") = dt.Rows(I2).Item("Middle_name")
                            duplicaterow.Item("Last_name") = dt.Rows(I2).Item("Last_name")
                            duplicaterow.Item("Duplicatecount") = count

                            'add rows to table
                            Duplicatetable.Rows.Add(duplicaterow)


This is the method I am using for my parallel for:
Public Sub ParallelForEach(ByVal startOfIteration As Integer, ByVal endOfIteration As Integer, ByVal subFunctionName As String)

        processorcount.MaxDegreeOfParallelism = 4
        Const flags As BindingFlags = BindingFlags.Public Or BindingFlags.NonPublic Or BindingFlags.Static Or BindingFlags.Instance

        ' Find the method with the specified name which accepts a single Integer parameter:
        Dim method As MethodInfo = Me.GetType().GetMethod(subFunctionName, flags, Nothing, New Type() {GetType(Long)}, Nothing)

        If method Is Nothing Then
            Throw New ArgumentException(String.Format("Method '{0}' was not found.", subFunctionName), "subFunctionName")
        End If

        ' Create a delegate of the correct type pointing to the method:
        Dim del As System.Delegate
        If method.IsStatic Then
            del = System.Delegate.CreateDelegate(GetType(Action(Of Long)), method)
        Else
            del = System.Delegate.CreateDelegate(GetType(Action(Of Long)), Me, method)
        End If

        ' Cast the delegate to the correct type:
        Dim action As Action(Of Long) = DirectCast(del, Action(Of Long))

        ' NB: Second parameter to Range is the number of items to return, not the last number to return:
        '      Dim numbers As IEnumerable(Of Long) = Enumerable.Range(startOfIteration, 1 + endOfIteration - startOfIteration)


        System.Windows.Forms.Control.CheckForIllegalCrossThreadCalls = False
        System.Threading.Tasks.Parallel.For(1, 2, processorcount, action)
        
    End Sub


If I leave the parallel for like this:
System.Threading.Tasks.Parallel.For(1, 2, processorcount, action)
it runs as fast as a sequential.

if I change it to
System.Threading.Tasks.Parallel.For(1, 10, processorcount, action)
it runs quicker, but the parallel for adds in duplicate rows.

So if the record count is 150,000 it will show up as 300,000 I can not figure out a way to stop this from happening.

What I have tried:

I tried to cut off the row add when the count made it to the amount of temptable rows. I also tried to synclock both tables and that did not help either.
Posted
Updated 10-Aug-19 20:35pm
v2

1 solution

Rate this:
Please Sign up or sign in to vote.

Solution 1

First of all, the DataTable class is only thread safe for reads. To write to a DataTable object from multiple threads, they must be synchronized. That means only a single thread must be allowed to write to the table at any one time.
   
Comments
Member 11856456 11-Aug-19 11:29am
   
I have been reading about synchronization, however, I can't find any good examples to she how to do this with datatables. From what I am gathering the reading and the comparing is fine. What is the best approach in these types of situations?
Dave Kreskowiak 11-Aug-19 11:34am
   
Synchronization in this case is very simple. All you need is a lock object.
Dim lock As Object
.
.
.
SyncLock lock
    ' Code to create a new row in the DataTable
    ' and write the new row data.
End SyncLock
Member 11856456 11-Aug-19 11:47am
   
Dave, I tried this but its saying "value cannot be null" there is a green line under the lock object, what should be tied to this object?
Dave Kreskowiak 11-Aug-19 12:02pm
   
Whoops. That should be
Dim lock As New Object
Member 11856456 11-Aug-19 15:09pm
   
Hey Dave, I tried to use this the way you have suggested and I tried to place this in other portions of my code and I still could not get this to work. I think I am missing something. But I wanted to try a couple things before I posted back.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)




CodeProject, 503-250 Ferrand Drive Toronto Ontario, M3C 3G8 Canada +1 416-849-8900 x 100