Click here to Skip to main content
Click here to Skip to main content
Search within:



 
Comments & Discussions (23)

CrcStream stream checksum calculator

By , 8 Oct 2005
 

Introduction

CRC (Cyclic Redundancy Check) is commonly used as a way to confirm that a file had not corrupted during download. While convenient, it takes some time to read the data off of the disk after downloading for the check. It would be convenient if applications checked the CRC on-the-fly during download, so as not to waste idle CPU time and disk read time.

Downloading is done at a relatively leisurely pace (typically anywhere between 5-300kb/s) and over a long period of time, so it makes for a great opportunity to process data without impeding performance. Although ugly and impractical for most applications (it'd be safe to assume that most users think they've "broken the intarweb" when they see a hex number), displaying the CRC to the user immediately as a download finishes can often be a well-appreciated bonus.

This class passively calculates CRCs as data passes through it, ready to be used at any time.

Using the code

To calculate the CRC of a file as it is read to the end, create a new CrcStream passing the FileStream as an argument, and use the ReadCrc property to retrieve the CRC. Be sure to use the new CrcStream instead of the file stream to read from the file; otherwise the checksum will not be calculated.

//Open a file stream, encapsulate it in CrcStream
FileStream file = new FileStream(filename, FileMode.Open);
CrcStream stream = new CrcStream(file);

//Use the file somehow -- in this case, read it as a string
StreamReader reader = new StreamReader(stream);
string text = reader.ReadToEnd();

//Print the checksum
Console.WriteLine("CRC: " + stream.ReadCrc.ToString("X8"));

There are four public members in addition to the abstract Stream overrides:

  • ReadCrc - gets the checksum of the data that was read through the stream.
  • WriteCrc - gets the checksum of the data that was written to the stream.
  • ResetChecksum - resets the CRC values.
  • Stream - gets the encapsulated stream.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

 Rei Miyasaka
Canada Canada
Member
The cows are here to take me home now...

Article Top
 
Sign Up to vote   Poor Excellent
Add a reason or comment to your vote: x
Votes of 3 or less require a comment

Comments and Discussions

 
You must Sign In to use this message board.
Search this forum  
    Spacing  Noise  Layout  Per page   
First PrevNext
GeneralExcellent!memberJonKristian31 Mar '11 - 22:56 
Thanks for this gem! This helped me a LOT and saved my weekend! Smile | :)
 
Keep on truckin'!
Sign In·View Thread·Permalink
GeneralMy vote of 5memberdrago697 Mar '11 - 20:59 
This is exactly what I was looking for. Thank you!!
Sign In·View Thread·Permalink
GeneralMy vote of 5memberChewsHumans6 Sep '10 - 12:00 
I haven't tested this yet, but if it works, it's just what I'm looking for. Cheers!
Sign In·View Thread·Permalink
GeneralRe: My vote of 5memberChewsHumans6 Sep '10 - 13:11 
Yep, it works fine! Thanks very much!
Sign In·View Thread·Permalink
NewsCrcStream in VB.netmemberMike19537 Nov '08 - 11:16 
Great job! I don't know if this is the right place for this but I needed it in VB.net so I rewrote.
Code is below.
 

Mike
 

 
Imports System
Imports System.Collections.Generic
Imports System.Text
Imports System.IO
 

Namespace crcCheckSum
 
''' <summary>
''' Encapsulates a <see cref="System.IO.Stream" /> to calculate the CRC32 checksum on-the-fly as data passes through.
''' </summary>
Public Class CrcStream
Inherits Stream
 
Private Shared table As UInteger() = GenerateTable()
Private m_stream As Stream
Private m_readCrc As UInteger = 4294967295
Private m_writeCrc As UInteger = 4294967295
 

''' <summary>
''' Encapsulate a <see cref="System.IO.Stream" />.
''' </summary>
''' <param name="stream">The stream to calculate the checksum for.</param>
Public Sub New(ByVal stream As Stream)
Me.m_stream = stream
End Sub
 

''' <summary>
''' Gets the underlying stream.
''' </summary>
Public ReadOnly Property Stream() As Stream
Get
Return m_stream
End Get
End Property
 
Public Overloads Overrides ReadOnly Property CanRead() As Boolean
Get
Return m_stream.CanRead
End Get
End Property
 
Public Overloads Overrides ReadOnly Property CanSeek() As Boolean
Get
Return m_stream.CanSeek
End Get
End Property
 
Public Overloads Overrides ReadOnly Property CanWrite() As Boolean
Get
Return m_stream.CanWrite
End Get
End Property
 
Public Overloads Overrides Sub Flush()
m_stream.Flush()
End Sub
 
Public Overloads Overrides ReadOnly Property Length() As Long
Get
Exit Property
End Get
End Property
 
Public Overloads Overrides Property Position() As Long
Get
Exit Property
End Get
Set(ByVal value As Long)
m_stream.Position = value
End Set
End Property
 
Public Overloads Overrides Function Seek(ByVal offset As Long, ByVal origin As SeekOrigin) As Long
Return m_stream.Seek(offset, origin)
End Function
 
Public Overloads Overrides Sub SetLength(ByVal value As Long)
m_stream.SetLength(value)
End Sub
 
Public Overloads Overrides Function Read(ByVal buffer As Byte(), ByVal offset As Integer, ByVal count As Integer) As Integer
count = m_stream.Read(buffer, offset, count)
m_readCrc = CalculateCrc(m_readCrc, buffer, offset, count)
Return count
End Function
 
Public Overloads Overrides Sub Write(ByVal buffer As Byte(), ByVal offset As Integer, ByVal count As Integer)
m_stream.Write(buffer, offset, count)
m_writeCrc = CalculateCrc(m_writeCrc, buffer, offset, count)
End Sub
 
Private Function CalculateCrc(ByVal crc As UInteger, ByVal buffer As Byte(), ByVal offset As Integer, ByVal count As Integer) As UInteger
Dim i As Integer = offset, [end] As Integer = offset + count
While i < [end]
crc = (crc >> 8) Xor table((crc Xor buffer(i)) And &HFF)
i += 1
End While
Return crc
End Function
 
Private Shared Function GenerateTable() As UInteger()
Dim table As UInteger() = New UInteger(255) {}
Dim crc As UInteger
Const poly As UInteger = 3988292384
For i As UInteger = 0 To table.Length - 1
crc = i
For j As Integer = 8 To 1 Step -1
If (crc And 1) = 1 Then
crc = (crc >> 1)
crc = crc Xor poly
Else
crc >>= 1
End If
Next
table(i) = crc
Next
Return table
End Function
 
''' <summary>
''' Gets the CRC checksum of the data that was read by the stream thus far.
''' </summary>
Public ReadOnly Property ReadCrc() As UInteger
Get
Return m_readCrc 'Xor &HFFFFFFFF
End Get
End Property
 

''' <summary>
''' Gets the CRC checksum of the data that was written to the stream thus far.
''' </summary>
Public ReadOnly Property WriteCrc() As UInteger
Get
Return m_writeCrc ' Xor &HFFFFFFFF
End Get
End Property
 
''' <summary>
''' Resets the read and write checksums.
''' </summary>
Public Sub ResetChecksum()
m_readCrc = 4294967295
m_writeCrc = 4294967295
End Sub
End Class
End Namespace
Sign In·View Thread·Permalink
QuestionHow does it handle big files?memberDrDtieltirjt5 Mar '08 - 2:27 
How does it handle big files?
Sign In·View Thread·Permalink
AnswerRe: How does it handle big files?memberreinux5 Mar '08 - 2:35 
It works really well with big files, especially if you're already reading or writing them for other purposes. The main idea of this class is that everything is done on-the-fly, thus getting rid of any significant overhead and wait times.
Sign In·View Thread·Permalink
GeneralRe: How does it handle big files?memberDrDtieltirjt13 Mar '08 - 1:31 
Hello again,
since ReadToEnd produces an out of memory exception, one must use the code in another way, but how?
 
Regards
Sign In·View Thread·Permalink
GeneralRe: How does it handle big files?memberreinux13 Mar '08 - 10:05 
You're getting an OutOfMemoryException because you're reading in a huge stream all at once.
 
You have to read it in a little bit at a time, like this:
 
byte[] buffer = new byte[4096];
int length;
 
while((length = buffer.Read(buffer, 0, buffer.Length)) != 0)
{
   // Do whatever you need to do in here
}
 
Console.WriteLine("CRC: " + stream.ReadCrc.ToString("X8"));
 
The CRC is calculated each time you call Read, but it won't be the CRC of the complete file until you've read the entire file.
Sign In·View Thread·Permalink
GeneralRe: How does it handle big files?memberChristian Loft16 Mar '08 - 21:55 
you are right. thx
Sign In·View Thread·Permalink
Last Visit: 31 Dec '99 - 18:00     Last Update: 23 May '13 - 2:07Refresh123 Next »

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

About Article
Make better use of time by calculating CRCs on-the-fly.
Type Article
Licence CPOL
First Posted 8 Oct 2005
Views 67,807
Bookmarked 40 times

, +
Top News

The Next Version of Android - Some of What's Coming

Get the Insider News free each morning.
Related Videos
C++ Programming - 0709 Simple Calculator
Beginners Java Programming - 0808 Exercise 2 - A Calculator Program
Calculating MD5 Checksum
Writing Your Own GPS Applications: Part I
Adler-32 Checksum Calculation
C# CCITT-8 CRC Algorithm
Managed C++ wrapper for ZLib
32-bit UDP Checksum
Identification numbers and check digit algorithms
Ping and Traceroute
SimpleZip
CFileInfoArray: A class for gathering file information recursively through directories
CRC32: Generating a checksum for a file
Writing Your Own GPS Applications: Part 2
How to craft UDP packets and send them with WinPCap
A File Checksum Shell Menu Extension DLL
An Analysis of the Windows PE Checksum Algorithm
CRC Encoding
Checksum for Database
Python Implementation of IP Checksum
Verifying .MD5 file verification databases
Simple Modbus Protocol in C# / .NET 2.0
Permalink | Advertise | Privacy | Mobile
Web04 | 2.6.130523.1 | Last Updated 8 Oct 2005
Article Copyright 2005 by Rei Miyasaka
Everything else Copyright © CodeProject, 1999-2013
Terms of Use
Layout: fixed | fluid