Task action - seems I have a syntax problem

Question

0.00/5 (No votes)

See more:

Hello,

I need to loop through a large number of files (usually around 150.000 files of approx. 48 MB each) very fast and sum up file sizes as well as file count whilst showing a progress bar for the user.

From a MS sample I learned that Parallel.ForEach seem to be the fastest way (pls have a look at 'what have you tried').

The byteCount calculates fine, while the fileCount results are false. You will surely recognize that the 'fileCount' variable isn't interlocked. Unfortunately I have no idea how to get that done properly, I run into syntax errors with whatever I try.

Could someone of you help me with the proper syntax, please?
Also I'm not sure if that's really the fastest way or if there's probably something much simpler that I could do.

Thank you,
Mick

What I have tried:

VB.NET

Parallel.ForEach(files, Function() 0, Function(f As FileInfo, loopState As ParallelLoopState, local_bytes As Long, local_files As Long)
    If (f.Attributes And FileAttributes.Directory) = 0 Then
        local_bytes += f.Length
    End If
    Return local_bytes
End Function, Sub(c)
    Interlocked.Add(total_bytes, c)   ' byteCount
    total_files += 1                  ' fileCount
    BeginInvoke(CType(Sub()
        pbVal = progBar.Value + increm
        If pbVal > 100 Then pbVal = 100
        progBar.Value = pbVal
   End Sub, Action))
End Sub)

Posted 3-Sep-20 3:03am

Sonhospa

Updated 3-Sep-20 3:22am

Richard Deeming

v3

Add a Solution

Comments

[no name] 3-Sep-20 9:14am

The whole concept of passing interlocked "value" variables as function parameters makes no sense. You create a common class / method that every one uses to access the "interlock", or use a lock, the variable in question; you don't pass it around.

1 solution

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Richard Deeming · Accepted Answer · 2020-09-03T03:21:00

Solution 1

Try using a Tuple for the local state:

VB.NET

Parallel.ForEach(files, Function() (Length := 0L, Count := 0L), 
    Function(f As FileInfo, loopState As ParallelLoopState, index As Long, accumulator As (Length As Long, Count As Long))
        If (f.Attributes And FileAttributes.Directory) = 0 Then
            accumulator = (accumulator.Length + f.Length, accumulator.Count + 1)
        End If
        Return accumulator
    End Function,
    Sub(total As (Length As Long, Count As Long))
        Interlocked.Add(total_bytes, total.Length)
        Interlocked.Add(total_files, total.Count)
        BeginInvoke(CType(Sub()
            pbVal = progBar.Value + increm
            If pbVal > 100 Then pbVal = 100
            progBar.Value = pbVal
        End Sub, Action))
    End Sub)

Tuples - Visual Basic | Microsoft Docs[^]

Posted 3-Sep-20 3:21am

Richard Deeming

Comments

Sonhospa 3-Sep-20 9:53am

That's perfect - thank you, Richard!

Sonhospa 3-Sep-20 10:39am

Hi again, sorry but I have a follow-up question... using the code now shows that the progressbar isn't working. First the result shows (for the test I display it in a message box) and AFTER this the progress bar increments. Is it at all possible to show the progress in a parallel.foreach loop?

Richard Deeming 3-Sep-20 10:41am

It sounds like you're blocking the UI thread waiting for the Parallel.ForEach call to complete. You probably need to use a BackgroundWorker[^] to perform the work on a background thread, so that your UI has a chance to update.

Sonhospa 3-Sep-20 11:04am

That's possible... I'll check and hope to resolve that. Thank you again!

Sonhospa 3-Sep-20 14:42pm

Sorry that I have to ask you again... I still didn't manage to make it work. The method is called in a click-Handler like this, which should put it on a background thread:
Dim t As Task = Task.Run(Sub()
msg = MethodName(Path, "*.*", System.IO.SearchOption.AllDirectories)
End Sub)
t.Wait()

I've set a breakpoint at the "progbar.value = " line and observed that it's not reached for every file but in somewhat arbitrary intervals, which makes the calculation of the value guesswork. I'm sooooo stuck :-( that I'd really be glad about another good idea from you...

Richard Deeming 4-Sep-20 3:58am

Task.Run(...).Wait() pushes the work onto a background thread, and then blocks the current thread until the work has completed. You're still blocking the UI thread, but now you're using a thread-pool thread at the same time.

Try removing the t.Wait() call. Or use an async method:

Private Sub ButtonClick(ByVal sender As Object, ByVal e As EventArgs)
    CountFilesAsync();
End Sub

Private Async Function CountFilesAsync() As Task
    Button.Enabled = False
    Try
        Await Task.Run(...)
    Finally
        Button.Enabled = True
    End Try
End Sub

Sonhospa 4-Sep-20 17:31pm

Thank you for your help and the idea, and have a nice weekend!