Click here to Skip to main content
15,888,233 members
Please Sign up or sign in to vote.
0.00/5 (No votes)
See more:
Hi,In my program I receive some values every second from a particular ip. I have to do some operation on those values which takes more than a second. If I spend a second on this operation, I would not be able to read the values generated during that second and I would miss out those values. So I am planning to create a Task in C# and invoke it whenever I get the values and pass the values to it so that it would run the operation and I would not miss out any values.
I was doing the following
C#
if(array_size)<500
{       
Task.Run(() => SendMessage(testarray));
}

But I think this is creating new task every time I invoke it and after few hours I got out of memory exception.
Could you guys please suggest me some way to solve this problem in an efficient way so that I don't encounter out of memory error. Also, if possible please share a snippet of the solution.
Thanks

What I have tried:

I tried running it and after few hours ,it would crash with out of memory exception
Posted
Updated 1-May-18 3:41am
v2
Comments
BillWoodruff 1-May-18 12:35pm    
Is there any possibility you can pause, or delay, the incoming data stream ?
RickZeeland 1-May-18 14:20pm    
What hardware are you running your application on, what CPU, amount of memory etc ?

That won't fix your problem in the long run: even with a threaded system, if your processing takes 1.1 seconds and you receive new data every 1.0 seconds, you will last 0.1 times the number of cores you have in your processor seconds at best before you are losing data again.

Threading can't solve this, in fact it may make the problem worse because each thread adds extra overhead in terms of memory and processing time that complicates your task.

You need to look very carefully at the processing job you are doing and either improve its efficiency or replace it with a more efficient algorithm so that your processing takes 0.9 seconds, not try to "offload" the problem onto threads.

Sorry, but we can't do that for you.
 
Share this answer
 
Comments
Member 13245297 1-May-18 13:11pm    
Hi. Thanks for pointing that mistake out. What if I maintain a queue and enqueues it with the message read and have a task which will periodically dequeue the contents and process it? It will make sure that I wont miss any data, right? Thanks
OriginalGriff 1-May-18 14:00pm    
No, you'll still have the same problem.
If you can't take stuff off the queue faster than it arrives, then you will run out of memory eventually - you will always have a finite number of cores to run threads on (and each thread needs a free core to execute).
That's what your current code is showing: your queue is filling faster than you take stuff off.

Either you need to speed up the processing of each packet, or reduce the frequency at which they arrive.
You could use taskname.IsCompleted, and then dispose the task, see: How to: Wait on One or More Tasks to Complete[^]
Example:
C#
Task checkProcesses = Task.Factory.StartNew(this.CheckProcesses);
//... e.g. a loop ...
While (True)
{
  if (checkProcesses.IsCompleted)
  {
	checkProcesses.Dispose();
	checkProcesses = Task.Factory.StartNew(this.CheckProcesses);
  }
}

private void CheckProcesses()
{
}
 
Share this answer
 
Comments
BillWoodruff 1-May-18 12:34pm    
This sounds reasonable only if the OP can pause/delay the incoming data stream ... imho the OP's description suggests that not possible.
RickZeeland 1-May-18 13:39pm    
He could use multiple tasks or an array of Tasks in the loop.
But the key problem seems to me that his Tasks are not properly disposed, normally dispose should not be needed for Tasks but in his case it seems necessary ...

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)



CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900