Click here to Skip to main content
15,077,156 members
Please Sign up or sign in to vote.
2.00/5 (2 votes)
See more:
Hi,
I need to read .txt file where I have id number like 1320905829 is the key and it has related records to this key. Like that I have a big txt file which is 26MB and
almost 226000 records in it. To avoid any memory leaks what is the best way to handle reading a large .txt file in VB.net. Does any one has any sample code to deal
this or any suggestions.
As the file grows what is the best way to handle this situation. some times the file can be more than 26MB too. It depends. The main reason is to compare the values of this file to a different .xls file later on.
Posted

See this link

File.ReadLines Method (String)[^]

The important note there:
Quote:
The ReadLines and ReadAllLines methods differ as follows: When you use ReadLines, you can start enumerating the collection of strings before the whole collection is returned; when you use ReadAllLines, you must wait for the whole array of strings be returned before you can access the array. Therefore, when you are working with very large files, ReadLines can be more efficient.

Also refer this link

How to: Read Text from a File[^]

Regards..:)
   
v2
Comments
PIEBALDconsult 16-Sep-13 1:00am
   
Sure, but I don't think he wants to read the entire file into memory at once; I wouldn't unless there was no better way. Additionally, he hasn't yet said that there is a one-to-one relationship of lines to records.
Sounds like a bad situation. You should consider loading the file into a database and then working on it there.
If you can't do that, then more information would help.

0) Is each record one line of the file?
1) Are the IDs in order (sorted)?
2) How often does the file change?


"To avoid any memory leaks..."

When using VB.net, you probably needn't worry about memory leaks, but memory usage could be a concern with a large file.


"compare the values of this file to a different .xls file later on"

Ah, that's a different matter entirely; you should consider reading the text file and then seeking the IDs in the Excel file instead of the other way around.
   
v5

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)




CodeProject, 20 Bay Street, 11th Floor Toronto, Ontario, Canada M5J 2N8 +1 (416) 849-8900