how to read a large text file in VB.NET

Question

5.00/5 (1 vote)

See more:

Hi .. I'm working on a project that reads a text file. I open the file using READBLOCK. I get an error when opening big files. No issues when opening a 10MB file but got an error when opening a 1GB file.

Here's the part of the code that opens/reads the file:

VB

Const MAX_BYTES As Integer = 1048576 * 5 '=10 MB 
Dim bytesRead(MAX_BYTES) As Char
Dim numBytesRead As Integer
Dim currentPos As Integer = 0
Dim strFileName As String
Dim strm As System.IO.Stream
Dim TextLine As String
Dim FileDetail As IO.FileInfo

OpenFileDialogMain.ShowDialog()
strm = OpenFileDialogMain.OpenFile()
strFileName = OpenFileDialogMain.FileName.ToString()
  
Using reader As System.IO.TextReader = System.IO.File.OpenText(strFileName)
  numBytesRead = reader.ReadBlock(bytesRead, currentPos, MAX_BYTES)
  TextLine = reader.ReadLine()
End Using

txtEditor.Text = New String(bytesRead)
txtEditor.Text += TextLine
txtEditor.Text = txtEditor.Text & vbCrLf & Now

Does anyone have any suggestion on how to open a large file?

Thanks!

Posted 19-Jun-12 18:48pm

sdcruise

Updated 19-Jun-12 18:49pm

JF2015

v2

Add a Solution

Comments

abb_saleh 22-Feb-13 15:10pm

very good tanks

2 solutions

Add a Solution

Add your solution here

Treat my content as plain text, not as HTML

Preview 0

…

Existing Members

Sign in to your account

...or Join us

Download, Vote, Comment, Publish.

Your Email
Password
Forgot your password?

Your Email
This email is in use. Do you need your password?
Optional Password

I have read and agree to the Terms of Service and Privacy Policy
Please subscribe me to the CodeProject newsletters

When answering a question please:

Read the question carefully.
Understand that English isn't everyone's first language so be lenient of bad spelling and grammar.
If a question is poorly phrased then either ask for clarification, ignore it, or edit the question and fix the problem. Insults are not welcome.
Don't tell someone to read the manual. Chances are they have and don't get it. Provide an answer or move on to the next question.

Let's work to help developers, not make them feel stupid.

This content, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Ganesan Senthilvel · Answer 1 · 2012-06-19T18:55:00

If you want to read it all into memory, a simple File.ReadAllText() will do just fine. If your file is indeed very large, then you can use the StreamReader class, see the below approach. It is sometimes inevitable but should mostly be avoided for style reasons.

VB

im file As New FileInfo("path\to\file")

Using reader As StreamReader = file.OpenText()
    While Not reader.EndOfStream
        Dim nextLine As String = reader.ReadLine()
        ProcessLine(nextLine)
    End While
End Using

The challenges on large read, is discussed in detail at http://kirillosenkov.blogspot.com/2007/11/please-use-filereadalltext-and-like.html[^]

Jared Nathan Drake · Answer 2 · 2013-01-16T19:14:00

Solution 2

Got a problem with ProcessLine, it says 'ProcessLine' is not declared. It may be inaccessible due to its protection level?

Posted 16-Jan-13 19:14pm

Jared Nathan Drake

Comments

OneInNineMillion 17-Jan-13 3:03am

In response to your question about ProcessLine:
ProcessLine seems to be a custom Sub here.
Crappy example to point out the obvious:

Dim TotalChars As Integer = 0
Private Sub ProcessLine(ByVal LineOfText As String)
TotalChars = TotalChars + LineOfText.Count
End Sub

In this case ProcessLine would help you count the total amount of chars in your file.
Hope that was helpful.

Jared Nathan Drake 21-Nov-14 15:15pm

Hey there, thanks! Will do that :-)