Click here to Skip to main content
15,889,335 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
C#
for (int i = 0; i < ds.Tables[s].Rows.Count; i++)
{
    for (int j = 0; j < ds.Tables[s].Columns.Count; j++)
    {
        if (ds.Tables[s].Rows[i][j].ToString().ToUpper().Trim().Contains("NO OF UNITS"))
        {

            int t = j + 1;
            if (ds.Tables[s].Rows[i][t].ToString() == "")
            {
                dt.Rows.Add();
                dt.Rows[mn][0] = "Parts Per uint Should be mandatory";
                mn = mn + 1;
            }

        }

    }
}


In the above code i will use "Contains" function to validate the data table.the problem is in this method execution process is very slow.how to fix it.
Note:
In this table have multiple rows and columns.and the header (Heading or description like "item number")
The headers rows and columns position is not fixed it is changed based on sheet so, only i have used "Contains" function. more then 10 for loops for that validation.so the process is very slow.

How to Fix it.

What I have tried:

C#
for (int i = 0; i < ds.Tables[s].Rows.Count; i++)
        {
            for (int j = 0; j < ds.Tables[s].Columns.Count; j++)
            {
                if (ds.Tables[s].Rows[i][j].ToString().ToUpper().Trim().Contains("NO OF UNITS"))
                {

                    int t = j + 1;
                    if (ds.Tables[s].Rows[i][t].ToString() == "")
                    {
                        dt.Rows.Add();
                        dt.Rows[mn][0] = "Parts Per uint Should be mandatory";
                        mn = mn + 1;
                    }

                }

            }
        }

I have used this code.
Posted
Updated 2-Sep-16 5:55am
v2
Comments
Andy Lanng 2-Sep-16 8:06am    
Have you heard of linq? It could at least simplify the code, at best improve execution time
Raja Ganapathy 2-Sep-16 8:11am    
How To do sir.
F-ES Sitecore 2-Sep-16 8:49am    
Linq does not improve performance, if anything it will slow it more. Linq is about convenience, not speed.
F-ES Sitecore 2-Sep-16 8:53am    
First thing I would try is to get rid of the ToUpper, ToString and Trim and use a case-insensitive IndexOf rather than Contains

http://stackoverflow.com/questions/444798/case-insensitive-containsstring

culture.CompareInfo.IndexOf((string)ds.Tables[s].Rows[i][j], "No of units", CompareOptions.IgnoreCase) != -1

Untested, but something like that
Raja Ganapathy 3-Sep-16 1:52am    
Thank you to all!

Start by getting it right: that code will throw an index out of range error if the last column contains your string, since the inner loop uses j from 0 to (columns count - 1) and the code uses j + 1 to access the the column.
Which implies that the final column doesn't need to be checked, so that saves some iterations of the inner loop.
Check what other cells do need to be checked - the chances are that only one or two will, and that will reduce the number of iterations considerably. Looking at your data and working out what is actually necessary will have the biggest effect on execution time.

And be aware that Linq is still a loop - it's just hidden so the execution is delayed until you "collapse the waveform" by getting the results.
 
Share this answer
 
I think the best is to put some brain in your code.
I guess ds is the result of a query in database and that NO OF UNITS is not scattered in every columns.
Detecting which columns can hold this string would greatly improve the runtime.

You can safely remove Trim() from your code.
 
Share this answer
 
Step 1 - Parallelize the processing

            var bag = new ConcurrentBag<tuple><int,>>();

            Parallel.For(0, ds.Tables[s].Rows.Count, (i) =>
            {
                for (int j = 0; j < ds.Tables[s].Columns.Count; j++)
                {
                    if (ds.Tables[s].Rows[i][j].ToString().ToUpper().Trim().Contains("NO OF UNITS"))
                    {

                        int t = j + 1;
                        if (ds.Tables[s].Rows[i][t].ToString() == "")
                        {
                            bag.Add(new Tuple<int,>(i, j));
                        }
                    }
                }
            });
</tuple>


Step 2 - Perform a bulk operation

var indexes = bag.ToList();

// Do a bulk operation


How you do the bulk operation depends significantly on what DB server you're using and how you want to hook into the data locally. Inserting is really expensive one at a time... If your DataTable is purely disconnected you can just sequentially add them.

Step 3 - Design the database better

I put this as the last step because, realistically, changing schemas in a deployed or legacy application can be challenging and risky. The first two steps are also easier to implement and you want to do those anyway... Searching values for hard coded values in databases that are meaningful is really bad. You want to have metadata that can be enforced with referential integrity so the results this process can come from an optimized query.
 
Share this answer
 
Ok - I'll give you the very quick version.

You should already have linq available if your using any sort of recent version of visual studio. You may not even have to add the using reference.

Linq is a query syntax used by .net to simplify loops like the one you show above.

I use linq extension syntax, so that's what I'll show you here. Others prefer the query syntax so you could always get help translating from one to the other.


No this query it pretty complex, so bare with me

C#
ds.Tables[s].Rows 
    .Cast<datarow>() 
    
var columns = ds.Tables[s].Columns.Cast<datacolumn>().Count();
ds.Tables[s].Rows//these are not currently queriable so we have to cast them out
   .Cast<datarow>() //this is only needed with collections 
   .Where(   //we want to bring back only rows that match the conditions
      dr =>  //this represents a single row
             //The rest of this is lambda.  thing of 'dr' as a parameter and the rest as a function (not entirely true) 
      {
         int i = //get the index of the "NO OF UNITS" item
           dr.ItemArray  //the columns in the row
           .Select((item, index) => new {item, index})  //we need the index so select an anon object
           .Where(  //select only the cell we want
               n => // n represents a class{item = object, index = int} that we just created
               n.item.ToString().ToUpper().Contains("NO OF UNITS") && //check the value
               (n.index + 1) < columns)  //check it's not the last column
               .Select(n => n.index + 1)  //select the next index
               .FirstOrDefault(); will return zero if conditions are not met 
           return i != 0 && string.IsNullOrEmpty(dr.ItemArray[i].ToString());
           // return false if index is zero or next item in not empty.
           //We must return a boolean to the "Where" clause.
     })
   //Now we have our rows.  You can do what you like.  Select any detail you want from the row or just create new rows in dt
   .Select(r=>);

</datarow></datacolumn></datarow>



This is pretty advanced linq & lambda.
I know you'll have loads of questions about this. ^_^
 
Share this answer
 
Comments
Andy Lanng 2-Sep-16 9:09am    
TBH - I'd go with what the other guys are saying - they know a lot more about performance that I ever have
Philippe Mori 2-Sep-16 11:17am    
Using Linq like that will most certainly not help to optimize code. Linq can sometime help for SQL queries as it might generate query that are optimized. But here, it won't help much since OP does not have data in predefined columns. Still if the database is remote and only a few rows need to be retrieve it might help.
Andy Lanng 2-Sep-16 11:22am    
That's pretty much exclusively how I use it so understandably my experience is bias. Thanks for clarifying ^_^
You should update the database so that it uses predefined columns of appropriate type. Don't expect to get good performance from a poorly designed database.

Thus, the best solution would be to convert the database once and then use it in the new format both for queries and adding new stuff.

If the database is remote and only a few rows need to be returned, then doing filtering mainly on server would help a lot.

If you really have to process a lot of unstructured data, it would probably be best to use a SQL reader for that kind of code.

If you need performance, then learn to properly design a database and write efficient queries.
 
Share this answer
 
Comments
Raja Ganapathy 3-Sep-16 1:52am    
Thanks to all!

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900