Click here to Skip to main content
Click here to Skip to main content

A C# component to encode and decode yEnc data

, 12 Apr 2004
Rate this:
Please Sign up or sign in to vote.
A fully tested component capable of encoding and decoding yEnc data.

Sample Image - yenc.gif

Introduction

This is an implementation of the yEnc algorithm, as described at http://www.yenc.org/ . yEnc is not an official standard, but it is nonetheless a very popular encoding method on binary newsgroups. As algorithms go, yEnc is very simple. It uses 8-bit characters to encode binary data. Since binary data is usually stored as 8-bit bytes, it does not have to accomplish too much Smile | :)

yEnc's popularity is due to the fact that it uses a full byte to encode the data, whereas other methods use only 7-bits. This makes messages encoded with yEnc smaller by a factor of 33-40%, according to the website. Smaller means quicker to upload and download, which is important when dealing with large binary files. It has additional benefits as well, in the form of an optional CRC32 check.

My implementation is interesting from 2 points of view:

  1. It is the only open-source one written in C#
  2. It is implemented as a cryptographic transform - more on that later

Some Info on yEnc

There are some peculiarities of newsgroup messages:

  • messages must be broken into lines, max around 1000 characters
  • some characters have meaning, and as such need to be escaped out

The current yEnc algorithm escapes out CR, LF and the NULL character by default. However, individual encoders are free to escape other characters as they wish. Lines are broken at 128 characters, or 256 characters, by convention. Other line lengths are supported.

yEnc data begins with a =ybegin tag at the start of a line. The tag has additional attributes that specify the number of bytes to expect, as well as the name of the file and the length of the lines. Multipart messages are supported. The data ends with a line starting with =yend. The reason for the "=y" is that, due to the nature of the algorithm, it could never occur naturally as part of the data.

Using the code

My implementation of the algorithm deals purely with encoding and decoding the data, not parsing of messages, or even parsing of the yEnc headers and footers. To me, that is a separate challenge, which I'll leave to someone else.

Initially, I started coding the encoder as an implementation of System.Text.Encoder. However, I soon realized that, although I could read the data as text, I was really dealing with bytes. Probably, that should have been obvious to me from the beginning, but sometimes it takes a while Frown | :( Eventually, I decided it would work best as an implementation of ICryptoTransform. This is not to imply that it is a cryptographic algorithm, just that it transforms data in similar ways - the size of the input data does not necessarily match the size of the output data. Microsoft chose to implement the Base64 transformation objects in a similar way.

The benefit is that you can use the objects together with a CryptoStream object, which is a fairly easy interface to use, and automatically adds support for streams. I'll stress again though, that this is not an encryption technique - I am just making use of existing Framework objects and interfaces to add power to my objects.

To encode some yEnc data, your code might look like this:

MemoryStream ms = new MemoryStream();  <BR>  //this could be any stream we want to write to

YEncEncoder encoder = new YEncEncoder();
CryptoStream cs = new CryptoStream(ms, encoder, CryptoStreamMode.Write);
StreamWriter w = new StreamWriter(cs);
w.Write("Test string");

//make sure everything is written out
w.Flush();
cs.Flush();

To decode it again, the code might continue:

//reading back from the memorystream
ms.Position = 0;        
YEncDecoder decoder = new YEncDecoder();
CryptoStream cs2 = new CryptoStream(ms, decoder, CryptoStreamMode.Read);
StreamReader r = new StreamReader(cs2);
string finalText = r.ReadToEnd();
        

This is pretty standard code that you might write if you were encrypting your data, the only difference being that we are using the yEncEncoder and yEncDecoder instead of a system-supplied encryption algorithm.

Points of Interest

I have made use of Phil Bolduc's implementation of the CRC32 algorithm, found at http://www.codeproject.com/csharp/crc32_dotnet.asp . Unfortunately, there were some bugs in that that consumed a significant amount of my time. I had to make some modifications to make it work 100%. Other than that, the code is an original work of my own, not based off of any other implementations. You are free to use it for whatever purpose you may desire, as long as you attribute it to me in the code comments.

The dowloadable code includes a lot of NUnit tests, which test things to a point where I am comfortable that everything works. They should make the code easy to expand on for anyone who wants to add functionality.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Steven Campbell
Web Developer
United States United States
Steve is a software developer working in Minneapolis, MN.

Comments and Discussions

 
GeneralC# unchecked keyword on Line 173 of Decoder.cs Pinmemberdavejuk22-Oct-09 6:32 
GeneralRe: C# unchecked keyword on Line 173 of Decoder.cs PinmemberSteven Campbell24-Oct-09 18:40 
GeneralRe: C# unchecked keyword on Line 173 of Decoder.cs Pinmemberdavejuk25-Oct-09 3:40 
GeneralRe: C# unchecked keyword on Line 173 of Decoder.cs Pinmemberdavejuk25-Oct-09 9:42 
GeneralRe: C# unchecked keyword on Line 173 of Decoder.cs Pinmemberdavejuk26-Oct-09 7:37 
GeneralRe: C# unchecked keyword on Line 173 of Decoder.cs PinmemberHarmen29-Nov-09 8:42 
GeneralRe: C# unchecked keyword on Line 173 of Decoder.cs PinmemberMember 259847821-Dec-09 10:54 
GeneralAwesome try... Pinmemberalexn29-Nov-06 17:52 
GeneralRe: Awesome try... Pinmemberalexn6-Dec-06 12:20 
GeneralRe: Awesome try... PinmemberBinindex21-Jan-07 11:39 
what encoding did you use for nfo files? I am trying to write a simple console app that will grab only nfo files from certain groups and decode them and put them in a db. For some reason so far they keep coming out just a little bit off. Only thing I can think of is that I have the encoding wrong on something.
GeneralRe: Awesome try... Pinmemberfdg sdfg8-Jul-09 4:02 
GeneralRe: Awesome try... PinmemberBinindex8-Jul-09 5:34 
GeneralDecoding giving me a hard time PinmemberCarl Mercier22-Oct-04 9:54 
GeneralRe: Decoding giving me a hard time PinmemberSteven Campbell22-Oct-04 17:09 
GeneralThanks... PinmemberJaseNet9-Apr-04 18:36 
GeneralRe: Thanks... PinmemberSteven Campbell10-Apr-04 17:35 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Mobile
Web03 | 2.8.140709.1 | Last Updated 13 Apr 2004
Article Copyright 2004 by Steven Campbell
Everything else Copyright © CodeProject, 1999-2014
Terms of Service
Layout: fixed | fluid