How to search Filesystem with TPL?

Question

0.00/5 (No votes)

See more:

I have tried to use Task Parallel Library within my for each statement to search the filesystem with no luck. I know that I must be doing something wrong or do not understand how to implement the multiple process threading.

along that being said I have seen some limited solutions that use the get files method but I have moved away from that because I wish to search for more than one file extension.

can someone please give a link or quick example of how to iterate a filesystem using this method?

thank you very much!

the for each statement is as follows:

VB

Try
               Using fse As New FindFiles.FileSystemEnumerator(pathToSearchCombo.Text, fileSpecsCombo.Text, includeSubdirectoriesCheck.Checked)
                   Dim ien As IEnumerator(Of FileInfo) = fse.Matches().GetEnumerator()
                   ien.Dispose()
                   For Each fi As FileInfo In fse.Matches

Posted 3-Jun-12 11:33am

Dale 2012

Add a Solution

2 solutions

Solution 1

I doubt you are going to get the TPL to give you much of a performance gain here. Most of the processing time is involved in generating your list. If you just do something simple with each file you won't get any benefits from the TPL and you will just add complexity. However, if you were to do something complicated or processor-intensive with each file then the TPL would really help you out.

The other option here would be to use the TPL to search each directory in a set of directories. Say, for example, you wanted to do the search in the following directories for the files you wanted:

C:\Files
C:\Temp
C:\Users\Tim\Documents
F:\
G:\Data
S:\SavedDocs

You could set these folder names in an array and then do a FindFiles method that took in the name of the folder path. Then you could do a Parallel.ForEach on the array and run the method inside each task. That would give you some performance gains, although I'm not sure you would see a lot just because the file system only has so much performance to give.

Usually the TPL does best when you are doing large calculations or are dealing with tasks that have waits in them (async for example). Then the TPL will go to the next task when one is waiting and thus you don't waste your processor. Disk access won't be sped up with the TPL.

Posted 3-Jun-12 14:03pm

Tim Corey

Comments

Sergey Alexandrovich Kryukov 3-Jun-12 23:18pm

Good points, a 5. Overall, using TPL should not be very effective for such task.
--SA

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Dave Kreskowiak · Accepted Answer · 2012-06-03T14:56:00

Since the disk is pretty much a serial device, it can only give you the file/directory information, or a single disk sector, for a single thread at a time, file system/disk scanning doesn't really lend itself to parrellism.

I know caching will make that statement false to a limited degree. But, it will not in the case of scanning an entire file system, as caching will only go so far before you're reading the disk again.