|
Ronenb wrote: should i use StreamFile, StreamRead or MemoryStream service?
A FileStream[^] would be the most appropriate. Next, you use a StreamReader, which has a <a href="http://msdn.microsoft.com/en-us/library/system.io.streamreader.readline.aspx">ReadLine</a>[<a href="http://msdn.microsoft.com/en-us/library/system.io.streamreader.readline.aspx" target="_blank" title="New Window">^</a>] method.
Ronenb wrote: can you send me example how to stream a file for each line?
See the links in this post, I don't provide copy/paste answers.
Bastard Programmer from Hell 
if you can't read my code, try converting it here[^]
|
|
|
|
|
Are you sure you don't deliberately sometimes supply copy/paste answers that are for the wrong question?
|
|
|
|
|
Failed to parse your question; can you slice it into simpler pieces for me?
To try and answer that; yes, I do sometimes post code that can be copied/pasted. Yes, I also supply the wrong answer sometimes.
Did I miss anything?
Bastard Programmer from Hell
if you can't read my code, try converting it here[^]
|
|
|
|
|
Eddy Vluggen wrote: Did I miss anything?
Yes.
I had suggested that depending on how stroppy the OP had been, you could deliberately supply the wrong piece of code. Not that I'm advocating this of course. Oh no. Definitely not. Noway. Nosirree.
|
|
|
|
|
|
|
this is a console app that may do what you are after, replace the console.writeline with a call to your parse method
class Program
{
static void Main(string[] args)
{
string filePath = @"D:\ReadMe.txt";
using (var streamReader = new System.IO.StreamReader(filePath))
{
while (!streamReader.EndOfStream)
{
var toParse = streamReader.ReadLine();
System.Console.WriteLine(toParse);
}
}
System.Console.ReadLine();
}
}
|
|
|
|
|
ok
that this is Basically what i did
srLog is type of StreamReader
so my problem was that i didint need to read all file content to memory and then process it 
I was thinking it is more efficient to read all file content (with one stroke) and work on it then to access file system for each line(ReadLine) and process it
Isn’t more expensive?
while (!srLog.EndOfStream)
{
line = srLog.ReadLine();
fileLines.Add(line);
}
fileLinesContent = fileLines.ToArray();
|
|
|
|
|
Being honest i don't know which way would perform better, your origional post involved calling ReadLine many times in a loop any way and you were getting memory issues. This way should not be less efficient than the origional proposed solution.
I'd suggest you try it with something many times larger than your expected log file size, if the performance is good then don't stress too much about how you could make it better. If the file IO part of this App is not the bottle neck in the process then don't lose too much time optimising it.
|
|
|
|
|
No, it's not. The way you were reading the file (line by line) and storing it in memory is no less expensive than processing that giant log one line at a time.
You're trying to read 700MB of data into memory and running into OutOfMemory problems. How efficient do you think that is?? By reading everything into memory all at once your solution only works on limited log sizes, dependant on system memory.
If you process every line, one at a time, without reading the entire file into memory, you can process log files of ANY size, up to the file size limit of the operating system and do it without requiring the machine to have terabytes of memory.
|
|
|
|
|
Got it
thanks all
Ronen
|
|
|
|
|
700MB is not that large, you shouldn't run out of memory if you process it efficiently, even if you store the whole thing in memory. Calling List.ToArray is going to double the memory usage, though; you should either store it as a List all the time, or read it into an array to begin with (almost certainly the former).
However, I suspect you are doing some streaming task and you don't actually need the whole thing. Parse each line as it comes, and don't store it; instead store whatever information about a line you need to know, if anything.
|
|
|
|
|
An array is a contiguous block of memory. Other than the fact that for large files all processing should be done in your while loop without retaining any of the file in memory your ToArray call could be killing you.
If you need Random Access to the File, look into Memory Mapped Files.
|
|
|
|
|
I'll repeat that you probably shouldn't be keeping a bunch of rows in memory.
However, if you really need to, I suggest defining a class to hold an entry once it has been parsed. And don't use an array; use a List or a Queue, or if you're passing it to another class, consider using an event.
|
|
|
|
|
You already know how many lines are in the file using the pageSize argument. Interesting as it suggests the file has already been read at least once. There may be memory issues there.
Can't tell what object types fileLinesContent or fileLines are but Arrays, ArrayLists and List<t> enable you to set the collections capacity. Setting the capacity, when initializing will potentially save memory.
Why not make fileLines the same type as fileLinesContent and then you can drop the ToArray() method.
As already mentioned, 700M isn't a big deal, there must be another part to this problem unless you are working on an old PC.
"You get that on the big jobs."
|
|
|
|
|
Am new to programming in general and am asked to design an app in C# that will distribute names in a database to different tables but randomly.
I need help seriously cos am on a deadline.
Any help will be duly appreciated.
Thanks
|
|
|
|
|
ayk439 wrote: I need help seriously cos am on a deadline
Then I suggest you seriously start learning. Here are some links that might help :-
ADO.Net[^] - to communicate with databases from c#.
Random Class[^] - to generate random numbers, so you can randomly pick the names from a List.
No-one is going to just do your homework for you.
When I was a coder, we worked on algorithms. Today, we memorize APIs for countless libraries — those libraries have the algorithms - Eric Allman
|
|
|
|
|
A few questions that may help people point you in the right direction to get the help you need.
1. what database, are you using SQL Server, MySQL, Access etc?
2. Where are the names comming from?
2. Do all the destination tables have the same structure?
4. Are there any restrictions on what you can use or must use, do you need a UI or is this a console application.
|
|
|
|
|
Hi,
this code sometimes works!and sometime doesnt open any page and even doesnt give any error.
i am using this code in event DoubleClick of GridView.
void AdvanceAutoGridViewList_DoubleCilickEvent(DataRow e)
{
try
{
Response.Write("<script>");
Response.Write("window.open('http://localhost:5162/WorkFlows/Forms/PersonalWork/?md=Edit&jb=11&stp=1')");
Response.Write("</script>");
}
catch(Exception ex)
{
Master.SetMessage(ex);
}
}
Thanks in advanced!
modified 31-Jul-12 2:17am.
|
|
|
|
|
Check on the browser side that the response was received (read the source of the page received).
Also note that JavaScript code is to be executed on the brwoser side, not on the server side - if you test on the computer where the web server is installed, a call to "localhost" works, but from a different computer it will fail.
|
|
|
|
|
Thanks for ans.
I knew that its not on web now(its now on localHost).
how to check that response was received (read the source of the page received)
Thanks in advanced!
|
|
|
|
|
In the browser, View menu -> Source.
You really need to pick up some beginner books on the stuff you're working with.
|
|
|
|
|
Dear respected sir, I needed asp.net project with database. database connection in Sql server. then how will connect the database some about idea based information I needed . I hope you will help me
|
|
|
|
|
For the right connection string, you can always use http://www.connectionstrings.com/[^].
If you are looking for some startup tutorials for ASP.Net / database just search the internet - you will get some good examples.
|
|
|
|
|
Assume a table called tblCompanies and another called tblUsers. Each Company can have one or more Users.
So I have this AddUser method. See the comments in-line
private int AddUser(UserEntity User)
{
int retVal = 0;
using (var dc = getDataContext())
{
var user = (from u in dc.tblUsers
where u.CompanyId == User.CompanyId &&
u.UserName.Trim().ToLower() == User.UserName.Trim().ToLower()
select u).FirstOrDefault();
if (user == null)
{
tblUser newUser = new tblUser
{
CompanyId = User.CompanyId,
RoleId = User.RoleId,
FirstName = User.FirstName,
LastName = User.LastName,
UserName = User.UserName,
Password = User.Password,
CanLogIn = User.CanLogIn
};
dc.tblUsers.InsertOnSubmit(newUser);
retVal = newUser.UserId;
}
else
{
}
}
return retVal;
}
Aside from try/catch & exceptions, what about the 2 potential FK violations on CompanyId and RoleId?
What's the right way to handle all this. I'v heard people say "Handle these issues in the BLL", while other folks seem to think exceptions related to data should be handled in the DAL?
What's your thoughts?
If it's not broken, fix it until it is
|
|
|
|
General 
News 
Suggestion 
Question 
Bug 
Answer 
Joke 
Praise 
Rant 
Admin 
Submit an article or tip
