How can I add straight to a list without looping throughout each list object using foreach?

Question

0.00/5 (No votes)

See more:

I am making few API calls to get data and loop through to add objects to a list which is working fine. However, because the API is returning more than 12000 records, my function takes more than 30 minutes to execute which is slowing the system performance.

How can I add straight to a list without looping throughout each list object?

This is my current code

What I have tried:

<pre>List<Destination> destinations = GetDestinationByCountryCode(rateSheetSummaries.FirstOrDefault().CountryCode).Where(d => d.Used == true).ToList();
                    foreach (var rateSheetSummary in rateSheetSummaries)
                    {
					    
                        List<Destination> insertDestinations = destinations.Where(d => d.Description.ToLower().Trim().Contains(rateSheetSummary.DestinationLookup.ToLower().Trim())).ToList();

                        foreach (var destination in insertDestinations)
                        {
                            Rate rate = new Rate()
                            {
                                DestinationGroup = rateSheetSummary.Description,
                                Description = rateSheetSummary.Description,
                                DestinationId = destination.DestinationId,
                                Country = rateSheetSummary.CountryCode,
                                TariffId = tariffId,
                                NextPrice = rateSheetSummary.Peak,
                                Price = rateSheetSummary.Peak,
                                Interval = 1,
                                Discontinued = "N",
                                Forbidden = "N"
                            };
                            rates.Add(rate);
                        }
                    }

Posted 24-Jun-21 23:00pm

Nkhanedzeni

Updated 25-Jun-21 6:52am

Add a Solution

2 solutions

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Chris Copeland · Answer 1 · 2021-06-24T23:20:00

I would think processing this sort of filtering on 12000 records is unfortunately going to be slow. The bloat in this process is dependent on how many entries in your rateSheetSummaries collection. If you have one entry in there, then you're looking at 1x12000 searches; otherwise, if you have 100 then this expands greatly up to 1,200,000 searches. Even worse is that every one of these searches performs the following:

C#

d.Description
  .ToLower()
  .Trim()
  .Contains(rateSheetSummary
    .DestinationLookup
    .ToLower()
    .Trim())

That's a lot of operations to be doing on every single item. Instead, think about ways to optimise this particular operation. First, the Contains() method has an overload for whether to be case insensitive. Second, consider whether using Trim() on every d.Description is needed. Third, you could cache the DestinationLookup prior to the search. Fourth, the call to ToList() on the results of the search seems unnecessary. Something like:

C#

foreach (var rateSheetSummary in rateSheetSummaries)
{
  string search = rateSheetSummary.DestinationLookup.Trim();					    
  IEnumerable<Destination> insertDestinations = destinations
    .Where(d => d.Description
      .Contains(search, StringComparison.InvariantCultureIgnoreCase));

  foreach (var destination in insertDestinations)
  {
    Rate rate = new Rate()
    {
      DestinationGroup = rateSheetSummary.Description,
      Description = rateSheetSummary.Description,
      DestinationId = destination.DestinationId,
      Country = rateSheetSummary.CountryCode,
      TariffId = tariffId,
      NextPrice = rateSheetSummary.Peak,
      Price = rateSheetSummary.Peak,
      Interval = 1,
      Discontinued = "N",
      Forbidden = "N"
    };
    
    rates.Add(rate);
  }
}

At the end of the day though, this might only provide a minor improvement to performance. The main issue here is iterating over so many records.

RickZeeland · Answer 2 · 2021-06-25T06:52:00

Solution 2

You might speed up things using Parallel.ForEach, see this article:
Process your list in parallel to make it faster in .NET[^]

Posted 25-Jun-21 6:52am

RickZeeland

Comments

Nkhanedzeni 25-Jun-21 15:07pm

It seems Parallel.ForEach will make things easier. I did a bit of digging and i come up with the below. I want to add to this list
List<rate> rates = new List<rate>();

Parallel.ForEach(insertDestinations, destination => {

//Now i am not sure how i can continue base on my Foreach statement. Can you please assist?
});