Click here to Skip to main content
15,888,527 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
Hi everyone,

mostly due to readability I am using the following code to a) find a specific csv file on a drive , b) read that csv file using a streamReader and c) parse it into a List to be able to use LINQ and/or PLINQ later on.

However, the process takes about 4-5 seconds, which is simply put too long. Any suggestions on how to improve the following (or maybe even replace it?) :

C#
var query = (from x in Directory.GetFiles(_path, "*.csv").AsParallel()
             where x.Contains(dateTime)
             select x).First();


            #region Read CSV File
            List _tempList = new List();

            FileStream stream = new FileStream(query, FileMode.Open, FileAccess.Read, FileShare.ReadWrite);
            using (StreamReader reader = new StreamReader(stream, Encoding.Default, true, 1024))
            {
                string currentLine;
                while ((currentLine = reader.ReadLine()) != null)
                {
                    string[] temp = currentLine.Split(new char[] { ',' }, StringSplitOptions.None);
                    _tempList.Add(temp);
                }
            }
            #endregion


I am thankfull for help. ;) The order inside the csv file is important - the file will contain between (2000-7000) x 25 entries.

Follow up question. The csv file has several fields which I am not interested in - and I don't necessarily need them in my List (or string[][]) .
I have tried to filter them with a LINQ statement similar to the following pseudo code:
C#
var query = from x in MyListOfStringArray
            select new {Col1 = x[1] , Col2 = c[4]}.ToArray();


This approach results in a strange type for query - and for that matter was pretty slow. (I would have an array with about 9-11 properties).
Any ideas on that second issue ?
Posted

1 solution

I have used A Fast CSV Reader[^] before, works well and fast, and is configurable (headers, no headers, delimiter etc.).

Here's another one that claims to be fast: Simple and fast CSV library in C#[^], I haven't used it but source code might be a bit easier to get through compared to the order one.
 
Share this answer
 
v2

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900