|
Phil Martin wrote: The simplest is to read in multiple files at the same time
Wow, where do you get such multi-tasking harddisks from? 
No, seriously: that step is likely to be the slowest of all, hence hardly anything can be gained from parallelizing. Only when processing the file contents takes a relatively high timespan, parallelizing might help, but then I'd uncouple reading and processing (i.e. read the file into memory and then send it to the processing function, with the next file read being started at that moment - note: may require large memory). The locks on the index may cause some extra overhead, i.e. slow down again.
|
|
|
|
|
I'm not sure Phil meant to read the files in parallel then continue processing them on a single thread. By parallelizing at the top level, the thread reading the file goes on to processing it so I'd say his assertion was correct but probably better stated as 'read in and process multiple files at the same time'.
It's an odd thing to do anyway. If I were to try and improve the performance of this code the first thing I'd do is get rid of the ReadAllLines and switch to streams before even thinking of going parallel.
Regards,
Rob Philpott.
|
|
|
|
|
Thanks for clarifying that for me Rob. And I agree with you, there are many other things I'd do first and differently. I was aiming for minimum effort on the developer's part with minimum side effects.
I'll edit the post and try to be a little less ambiguous about my intent.
|
|
|
|
|
Ha! I wish I had multitasking hard disks!
My choice of wording was poor, and I hope the code clarified the intent, i.e. choosing the simplest point in the original code to parallelize. Will the changes be optimal? No. Will they be parallel with significant and measurable difference? Yes.
And yes, there are many other options, but I think this was one of the simpler approaches to understand. If they already understood ways to set up a dataflow to achieve the task, I'm pretty sure they would have already done it that way.
|
|
|
|
|
Minimum changes would have been to just use Parallel.For on the outer loop and switch to a ConcurrentDictionary or use locks around the regular Dictionary. Your overall solution is overly complicated (no offense). Any time you save by parallelizing the main loop will be destroyed (and then some) by the merging loop.
|
|
|
|
|
Thanks for taking the time to reply. And I agree, parallel.for would have been the least invasive. And no offense taken!
|
|
|
|
|
Hi! First, I am totally new in programming, so excuse me if I ask dumb question. 
I am trying to make a new project in Visual Studio. I have Northwind database installed (probably came with VS installation). It is in Sample folder.
In my project I need to use stored procedures from Northwind database and also to make one new stored procedure. I managed how it's working, but I can't do it. I connected to the database, but it's not showing me that I have any stored procedures in it, in Database explorer. 
Any ideas?
|
|
|
|
|
Well, if you're using Sql Management Studio, make sure you refresh the Object Explorer. This isn't clever enough to update by itself if you are expecting it to do so.
Regards,
Rob Philpott.
|
|
|
|
|
what is your code can you show here
Sankarsan Parida
|
|
|
|
|
I have a question about plotting data from database to a line grpah. In my line graph there is 3 data that is tempin, tempout and also humdity that I must plot in a line graph..I made this coding but it seem doesn't work.I have a sql statement to selesct data but I do know how to plotting the line in a graph. Can anyone help me?
string sql = "SELECT TempIn, tempOut, humidity FROM Environmnets WHERE DateT > DateADD(mi, -30, Current_TimeStamp) INNER JOIN TDevice ON Environmnets.Deviceid = TDevice.id INNER JOIN TMushroomHouse ON TDevice.MushroomHouseId = TMushroomHouse.id WHERE TMushroomHouse.id = @id";
using (SqlConnection con = new SqlConnection(strConnString))
using (SqlCommand cmd = new SqlCommand(sql, con))
{
cmd.Parameters.AddWithValue("@id", DropDownList1.Text);
con.Open();
using (SqlDataReader dr = cmd.ExecuteReader())
{
while (dr.Read())
{
TempInTextBox.Text = dr.GetValue(0).ToString();
TempOutTextBox.Text = dr.GetValue(1).ToString();
HumidTextBox.Text = dr.GetValue(2).ToString();
Chart1.Series["TempIn"].Points.AddXY(DateTime.Now.ToShortTimeString(), Double.Parse(TempInTextBox.Text));
Chart1.Series["TempOut"].Points.AddXY(DateTime.Now.ToShortTimeString(), Double.Parse(TempOutTextBox.Text));
Chart1.Series["Humidity"].Points.AddXY(DateTime.Now.ToShortTimeString(), Double.Parse(HumidTextBox.Text));
Chart1.ChartAreas["ChartArea1"].Area3DStyle.Enable3D = true;
}
con.Close();
|
|
|
|
|
There's not much to go on here.
Is your chart emtpy or do you get an error? What is the error? Did you debug and step through to find where it goes wrong?
Is the SQLDataReader filled after the statement is executed?
This Chart1 object, what is it? A library? Where there samples where you found the library. Some documentation? Do those samples work?
Also make sure to split business logic from fetching data...
|
|
|
|
|
<asp:Chart ID="Chart1" runat="server" Height="339px" Width="896px"
Chart1 is a Chart ID. Actually there is no error, but the graph is not display line graph as what I want. It will display all data in database but I just want to display data from 30 minutes only. Do you know in this forum there is function to attach picture? so u can see clearly what I want to explain.Tq
|
|
|
|
|
You cannot attach pictures, but you can upload one to an online repository (like flickr or something) and provide the link.
If you execute the SQL to the database directly, is the resultset correct?
If that is correct, debug and check if the resultset in your application is correct. If the graph is showing and drawing, the culprit is most probably getting the data.
|
|
|
|
|
The SQL is wrong. When I execute the SQL to the database, no result will show. I will try to attach the picture for you
|
|
|
|
|
Syafiqah Zahirah wrote: TempInTextBox.Text = dr.GetValue(0).ToString();
TempOutTextBox.Text = dr.GetValue(1).ToString();
HumidTextBox.Text = dr.GetValue(2).ToString();
Chart1.Series["TempIn"].Points.AddXY(DateTime.Now.ToShortTimeString(), Double.Parse(TempInTextBox.Text));
Chart1.Series["TempOut"].Points.AddXY(DateTime.Now.ToShortTimeString(), Double.Parse(TempOutTextBox.Text));
Chart1.Series["Humidity"].Points.AddXY(DateTime.Now.ToShortTimeString(), Double.Parse(HumidTextBox.Text));
Why do you send the values from the database to textboxes, and then parse them from the textboxes to numeric values again?
What about
double tempIn = dr.GetDouble(0);
Chart1.Series["TempIn"].Points.AddXY(DateTime.Now.ToShortTimeString(), tempIn);
And since you always use DateTime.Now on the x axis, I think you are missing something else. What about querying the DateTime value also from the database, and using it here instead of DateTime.Now?
|
|
|
|
|
Hye Bernhard! Tq for replying my message. I will try to change my coding referring in what u say. And one more thing, I actuallt didn't understand on what u try to explain to me with this sentence '. What about querying the DateTime value also from the database, and using it here instead of DateTime.Now?'. can you give some example? tq
Student that not familiar with C# code and try very hard to finish her final year project with flying colours. Hope that everyone out there can help me 
|
|
|
|
|
I tried to retrieve the DateTime value from database,and plot it to the graph but it doesn't seems to work. This is my code:
while (dr.Read())
{
double tempIn = dr.GetDouble(0);
double tempOut = dr.GetDouble(1);
double humidity = dr.GetDouble(2);
double date = dr.GetDouble(3);
DateTime? value = (DateTime?)dr["DateT"];
Chart1.Series["TempIn"].Points.AddXY(value, tempIn);
Chart1.Series["TempOut"].Points.AddXY(value tempOut);
Chart1.Series["Humidity"].Points.AddXY(value, humidity);
Chart1.ChartAreas["ChartArea1"].Area3DStyle.Enable3D = true;
|
|
|
|
|
using System;
using System.IO;
using System.Collections;
namespace Applica
{
class Program
{
static void Main(string[] args)
{
DirectoryInfo da = new DirectoryInfo("C:\\Folder");
FileInfo[] Arr = da.GetFiles();
if (Arr.Length == 0)
{
throw new InvalidOperationException("No files found.");
}
FileInfo ap = Arr[Arr.Length - 1];
long Totbyte = ap.Length;
string filePath = ap.FullName;
string temPath = Path.GetTempFileName();
byte[] data = File.ReadAllBytes(filePath);
}
}
}
|
|
|
|
|
The ReadAllBytes method does exactly that: it reads the entire file content into the memory and returns it as an array of bytes. If you are getting "Out of memory" errors, then there are two main possibilities:
1) The size of any one file exceeds 2GB. There is an absolute limit on any .NET object of 2GB, no item (and an array of bytes is one item) can exceed this.
2) These are large files, and as such they will go on the Large Object Heap - anything larger than 85KB is a "large object" and as such goes on the LOH - which is not compacted, it's objects are "connected" when they are garbage collected, and it's filling up...but that should trigger a GC which should empty it so it shouldn't cause a problem.
The bad news is...System.Array does not implement IDisposable so you can't free it yourself. You might be able to force it, by setting data to null and manually calling the GC, but that's a nasty solution.
I would suggest that your best approach might be to "block read" your files: 64K lumps for example so that any allocation problem is reduced.
Those who fail to learn history are doomed to repeat it. --- George Santayana (December 16, 1863 – September 26, 1952)
Those who fail to clear history are doomed to explain it. --- OriginalGriff (February 24, 1959 – ∞)
|
|
|
|
|
<pre>
using System;
using System.IO;
using System.Collections;
namespace Applica
{
class Program
{
static void Main(string[] args)
{
DirectoryInfo da = new DirectoryInfo("C:\\Folder");
FileInfo[] Arr = da.GetFiles();
if (Arr.Length == 0)
{
throw new InvalidOperationException("No files found.");
}
FileInfo ap = Arr[Arr.Length - 1];
long Totbyte = ap.Length;
string filePath = ap.FullName;
string temPath = Path.GetTempFileName();
byte[] data = File.ReadAllBytes(filePath);
File.WriteAllBytes(temPath, data);
decimal[] arry = new decimal[Totbyte];
for (int count = 0; count < data.Length; count++)
{
arry[count] = data[count];
}
byte[] data2 = new byte[Totbyte];
for (int count = 0; count < arry.Length; count++)
{
data2[count] = (byte)arry[count];
}
string filePath2 = Path.Combine("C:\\check", Path.GetFileName(filePath));
File.WriteAllBytes(filePath2, data2);
data = File.ReadAllBytes(temPath);
data2 = File.ReadAllBytes(filePath);
}
}
}
|
|
|
|
|
¡Ay, caramba!
You do realize what that code does, don't you?
Not only do you allocate two arrays of bytes the same size as each file in the folder, you also allocate a third array 16 times larger for each file as well!
The decimal datatype is 128 bits wide - or 16 bytes!
So is any file in your folder is bigger than 134MB, you will exceed the 2GB maximum-object limit...and get "out of memory".
And frankly, that code doesn't make a whole lot of sense.
You read the file as bytes, copy each byte value to a decimal (which does nothing in practice), and then convert each decimal back to a byte (which will give you exactly what you started with!)
And then you write the data to a new place...
You could do the same much more easily with File.Copy... 
Those who fail to learn history are doomed to repeat it. --- George Santayana (December 16, 1863 – September 26, 1952)
Those who fail to clear history are doomed to explain it. --- OriginalGriff (February 24, 1959 – ∞)
|
|
|
|
|
See my comment below; I thought I recognised this code.
|
|
|
|
|
|
hello guys,
below is my sql select statement :
SELECT product.P_ID, Product.P_Name,Product.Leadtime, Product.SafetyStockamount,
Monthlysales.Month, Monthlysales.totalsalesamount, (totalsalesamount/30) as Averagedailysales, ((totalsalesamount/30) * Leadtime + SafetyStockamount) as reorderpoint
FROM Product, Monthlysales
where Product.P_ID = Monthlysales.P_ID
in product table there are product id, name, lead time, safety stock column, meanwhile in monthly sales table, there are productid, month (in int), and totalsalespermonth.
the problem is :
i want to retrieve the reorder point result from the select statement above in visual studio c#
the interface will include product id , month options, button and label to show the result.
the reorder point result will be based on the month, for example
if the user choose
pid 1 : 1
month options: 2 (february)
then label result will show the reorder point amount for pid 1 in february.
i also wondering whether i can directly calculate the reorder point result based on product id and its month in sql server and directly call it in c#.
how could i do it?? please help me
thank you
|
|
|
|
|
And?
What help do you need?
You know how to use SQL from C# I assume, or you wouldn't be writing complicated SQL...so what do you need help with?
Those who fail to learn history are doomed to repeat it. --- George Santayana (December 16, 1863 – September 26, 1952)
Those who fail to clear history are doomed to explain it. --- OriginalGriff (February 24, 1959 – ∞)
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
