Click here to Skip to main content
11,631,884 members (70,829 online)
Click here to Skip to main content

FileHelpers v2.0 - Delimited (CSV) or Fixed Data Import/Export Framework

, 10 Jul 2007 BSD 1.4M 1.8K 864
Rate this:
Please Sign up or sign in to vote.
An easy to use .NET library to read/write strong typed data from files with fixed length or delimited records (CSV). Also has support to import/export data from different data storages (Excel, Acces, SqlServer, MySql)

FileHelpers Home Page (Project Summary)

FileHelpers v 2.0

Browse the Forums Development Blog Support the project


The FileHelpers are an easy to use library to import/export data from fixed length or delimited flat files (like the CSV). FileHelpers also has support to import/export data from different data storages (Excel, Acces, SqlServer).

The library has a set of converters for the basic types and can be easily extended to provide custom converters.

The main idea is pretty simple:

You can strong type your flat file (fixed or delimited) simply describing a class that maps to each record of the file and work with your file like a strong typed .NET array.

Basic Uses

Directly between the files and the .NET source code:

Use 1

It also can be used as an intermediate between files and MS Access or MS SqlServer tables:

Use 2

Quick Start Guide (easy steps)

To start using the FileHelpers Library you only need to add a reference in your project to the file: FileHelpers.dll. You can find it in the Release directory of the distribution. Tip: remember to leave the FileHelpers.xml file to get Intellisense support.

Next you need to define a class that maps to the record in the source/detination file. For this example we use a file with this format delimited by a |:


The class that we refer can be like this:

public class Orders
   public int OrderID;

   public string CustomerID;

   [FieldConverter(ConverterKind.Date, "ddMMyyyy")]
   public DateTime OrderDate;

   public decimal Freight;

Later you must instantiate a FileHelperEngine and Read/Write files:

FileHelperEngine engine = new FileHelperEngine(typeof(Orders));

/// to Read use:
Orders[] res = (Orders[]) engine.ReadFile("input.txt");

/// to Write use:
engine.WriteFile("output.txt", res);

Finally you can use the res array to access each item in the file, for example:

 foreach (Orders order in res)
    Console.WriteLine("Order Info:");
    Console.WriteLine(order.CustomerID + " - " +

Working with FixedLength files is exactly the same but you need to use the [FixedLengthRecord] and [FieldFixedLength] attributes.

Record Class Wizard

Since version 1.3.5 the library has had a Record Class Wizard to generate the record class. The Record Class Wizard allows you to generate your record mapping class, save the definition, generate code based on snippets (Read File, Read with Generics, ReadAsync and so on).

The wizard uses the excellent FireBall CodeEditor

Click to see how it works

Who needs the File Helpers Library?

In almost every project there is a need to read/write data from/to a flat file of a specified format. Writing the code to process these files is not hard, you can use String.Split or String.SubString, but if you do that the code sometimes becomes hard to read and change. If you want to add support later for quoted string, multiline, convert types, etc. these things tend to get complicated.

So with the FileHelpers you don't need to worry about changes in the file structure because you can change the Type, Converters, Triming, Length in seconds.

Another place where you can find the FileHelpers useful is for log parsing, data warehouse and OLAP applications, communication between systems, file format transformations (for example from a fixed length to an CSV file), load test data into your NUnit tests and a lot more!!!

This library aims to provide an easy and reliable way to accomplish all these tasks.

Main Features

Easy to use: The FileHelpers Lib is straight forward to learn and use. (see EasyExample)

Auto Converters: The library has a set of converters for the basic types and can be easy extended to provide custom converters. (see ConverterBase and Converter Example)

RunTime Classes since version 1.6.0 you have been able to create your record class at run time, and load them from files with source code or an XML description (check the example)

Master-Detail: You can read and write records with a master/detail pattern. (see Example1 and Example2)

Multiple record format support: With the MultirecordEngine you can read files with different record layout, you can also read files with some delimited and some fixed length records. (see Example)

Event Support: The engines of the library contain some events to allow you to easily extend the behavior of the library (see Example)

MS Excel Storage: Are a way to extract / insert records between any source and an excel file. (see ExcelDataStorage and ExcelDataLinkExample)

DataLinks: Are a way to extract / insert records between a database and a file. (see DataStorage and FileDataLinkExample)

GenericDataLink: Now you can to copy records between two Data Storages (see GenericDataLink and DataStorage)

Asynchronous Mode: You can use the library to read line by line and not the whole file. (see Async Methods)

.NET Compact Framework Support: From the version 1.1 you can use the FileHelpers library for you PocketPC and WindowsCE developments. (thanks Pierre)

File Transform Engine: To convert files in one format to another (for example a file with CSV to a FixedLength record format) (check the example)

Progress Notification: To get notified of the progress in each operation in the library (check the example)

.NET 2.0 Generics: the cast less and strong typed version of the engines (check the example)

FileDiffEngine to allow compare files with the same record layout (check the example)

Others Features

Files, Stream and String Support: You can use the library to read/write any stream or string, not only files. (see FileHelperEngine)

Different Error Behaviors: You can set the behavior of the library when an error is found (throw and exception, ignore, save and continue, etc) (see the examples)

Quoted String Support: It allows to indicates that the field must be read and written as Quoted String, like Excel CSV. (see FieldQuotedAttribute )

CommonEngine a easy way to access to common operations (check the example)

Encoding Aware: You can define the encoding used to read and write files or streams. (see BaseEngine.Encoding)

NullValues: The library identifies the null values in the files and assigns an adequate value. (see FieldNullValueAttribute)

Good Documentation: The library is fully documented (at least that's the intention) with a lot of Examples of Use.

Align and Trimming: You can set the processing mode of the in/out strings with a lot of attributes like the FieldAlignAttribute and the FieldTrimAttribute.

Progress Notification

Here is a screenshot of the progress notification of the demo app.

Since versión 1.4.0 you can get notified of the progress of each operation of the library to show feedback to the user or whatever you want.

Class Diagram

Detailed Diagram

Detailed Diagram


The library contains more than 250 NUnit tests to ensure correctness:

Sample image


Version 2.0 (April-2007)

The library has passed his 1st birthday!!! Thank you all for your work and contributions (partial list of them)

A lot of things were changed (mostly internally), now we have a performance gain of more than 60% for .NET 2.0

I keep releasing the .NET 1.1 version because I know that a lot of people use it in some corportaions, but it could be that new features in the versions to come will be .NET 2.0 only. It is monumental work to maintain and optimize both versions.

There are also a lot of major changes, a lot of refactoring in the code, and a big core rewrite that allows the engines to use less memory and temporary strings (we are using more buffers to aviod this)

I keep working hard to keep the library updated and give support to the users. You can check for examples in the Forums which shows a lot of problems that have been solved. I also ask for a lot of mail with problems from the users.

We have our own domain thanks to (Antoine) and a Development Blog

Breaking Changes

  • The constructor of the MultiRecordEngine was changed to allow params args, and to disallow users from passing the RecordSelector (in write operations were not needed).
  • EndsRead and EndsWrite deleted, now you have a Close() operation to simplify the API and to make it similar to the System.Data namespace. (The async engines also implement IDisposable)
  • Renamed FileHelperException --> FileHelpersException (a better name and an easy solution to a problem)
  • By default the numeric fields in the FixedLengthRecords that don't have a [FieldAlign] will be aligned to the right and the rest the left by default, with this we avoid the problem of generating files with different meaning when we read and write.

Performance Related

  • Internal use of Reflection.Emit, Dynamic Methods and char buffers to get more than a 50% enhace in performance and for .NET 2.0 more than 65%
  • The Read..AsDT methods now create the DataTable record by record, and not at the end so you can handle large files without any memory overload
  • Not more reflection in the operations (only in the constructors).
  • FieldSorter faster too, Removed REFLECTION for EMIT

Shining new features

  • The very cool DelimitedFileEngine and FixedFileEngine that allow to change options at RunTime (also with generic versions) Check an example here
  • ConditionalRecords you can easily include or exclude certain records based on a RecordCondition (like BeginsWith, EndsWith, Contains or RegEx) Check the example
  • A new GenericDatabaseStorage which is excellent for allowing others Db with ASO.NET support to work with the FileHelpers (Thanks Rodolfo Finochietti)
  • Experimental Mono Project Support (some users are already using it, we hope that in the next release we have a completely working version for this framework)
  • New Version Checker in the demos and Wizard (both with renew look and feel) Check the movies
  • .NET 2.0 Nullable Types support. (Thanks Vijayan) Check the example
  • Notification Interfaces: you can now get notified of the events simply implement INotifyRead or INotifyWrite
  • Enhaced Debugging in .NET 2.0 with DebuggerDisplay and DebuggerBrowsable, DebugVisualizars to come.
  • The Wizard has a Record Class Test that allows you to introduce sample data and your class and check for errors or results.
  • You can check some screencast about the demos, wizard and other features of the library.
  • Now the code for VS2003 and VS2005 are both on SVN, sharing the same files and the build script autodetect your Visual Studio installation. So no more excuses: time to get involved.
  • The engines with async ops are now IDisposable and IEnumerable so you can declare it with using(..) and use them in a foreach loop

API Changes and Extensions

  • The write methods are less strict. You can pass now an IEnumerable instead of an array, so you can direct pass a List or ArrayLIst to the method without needing to do a ToArray()
  • RunTime records can now get a DataTable in the constructor and so use DataColumns to create one field for each column.
  • Generic versions of near all the engines of the library (it's really hard to mantain the two copies so everyone, it's time to port!!)
  • Async Operations in others engines like the MultiRecordEngine
  • Event support to the MultiRecordEngine thanks to the contribution of Francis de Fouchier
  • The library handle now has infinite levels of inheritance adding the high level fields first in the result record class.
  • Allow to ignore spaces and tabs in the IgnoreEmptyLinesAttribute
  • IgnoreCommentedLines thanks to MCampbell
  • The Read methods now have a maxRecords args to tell to the engines how much they must read.
  • A new constructor overload to the engines that allows you to pass the Encoding, which is useful for making the users aware of the Encoding feature.
  • FileTransformEngine with two new methods ReadAndTransformRecords and TransformRecords
  • Flush Method to AsyncEngines to allow users to ensure data write, for example, if they are using the lib for logging.
  • The converters are now smarter. They validate to which types can be assigned and throw exceptions if it is wrong (for example if you use ConverterKind.Decimal in an int field)
  • The ConvertException has a lot of context information: LineNumber, ColumnNumber, FieldName. Also has a better Exception Message.
  • The integer converters now recieve a decimal separator to build an InvariantCulture to format and parse the values and strings

Small Changes

  • Fixed the problem with ASP.NET: "Line 0: Metadata file 'filehelpers.dll' could not be found"
  • Fixed some problems with the CsvClassBuilder (it's a good practice to not release features added last-minute)
  • The Assemblies are now Signed
  • The new ConverterBase.DefaultDateTimeFormat to avoid setting the converter format field by field
  • Some error messages from the library were rewritten and now provide more context information (line and column numbers, field name, etc).
  • ExcelStorage support for more than 26 columns (thxs to Mark Izendooren)
  • Encoding Support to the CsvEngine
  • The wizard remembers all the paths now
  • SkipThisRecord in the AfterReadRecordEventArgs, this allows the skip records from the results (thanks Crestline)
  • More Methods for the CommonEngine
  • More than 420+ NUnit tests!!! (I can't belive it, I´m a lazy developer but I know how much worth there is in testing the lib)
  • Now you have two versions for the demos, one for each framework version.
  • The FixedLengthClassBuilder has more constructor to set the lengths of the fields in you instruction
  • Improved docs. (Thanks Antoine and Matt)
  • A lot of new examples check it out !!

Browse the complete history at SourceForge.

RoadMap (the future of the lib)

Check it out at SourceForge FileHelpers Roadmap.

Contact and Ideas

The main site of contact of the library are the FileHelpers Forums

Quick Feedback FileHelpers Forums

If you find that there is a feature that I must include, or you have a new idea (for the API, Source Code or Examples), only let me know, entering the forums or sending an e-mail to marcos[at]

Licence (LGPL)

FileHelpers Library is @Copyright 2005-2006 to Marcos Meli but it's source code and the binaries are free for commercial and non commercial use.

Vote for this article if you like the library.


This article, along with any associated source code and files, is licensed under The BSD License


About the Author

Marcos Meli
Software Developer (Senior)
Argentina Argentina
Marcos Meli v1.0 was released at august of 1980 by Francisco and Mabel Inc.
He´s from Bahia Blanca, Argentina.

He enjoy developing from the 12 years, has a degree in Computer Science, and of course, love this site and .NET in general.

He is the lead developer of the FileHelpers Library.

Marcos is also the co-funder of Devoo.Net.
A company that provides Object Oriented Components & Libraries for .NET developers (mostly open source)

You may also be interested in...

Comments and Discussions

QuestionLeer archivos de mas de 5 gb Pin
Member 26943073-Nov-14 10:07
memberMember 26943073-Nov-14 10:07 
QuestionCSV to SQL with c# Pin
Hind Ahmamad4-Aug-14 2:10
memberHind Ahmamad4-Aug-14 2:10 
AnswerRe: CSV to SQL with c# Pin
Mehulio Shah21-Oct-14 4:20
memberMehulio Shah21-Oct-14 4:20 
QuestionSupport for mysql? Pin
invisible.symbol5-Sep-13 3:04
memberinvisible.symbol5-Sep-13 3:04 
AnswerRe: Support for mysql? Pin
Rob Warthen10-Mar-14 5:34
memberRob Warthen10-Mar-14 5:34 
QuestionVisual Studio 2003 and Compact Framework 1.1? Pin
Jaspar Lamar Crabb8-Aug-13 23:43
memberJaspar Lamar Crabb8-Aug-13 23:43 
QuestionHow to check a file is open or not in FileHelper Pin
Praveen Jeganathan7-May-13 20:55
memberPraveen Jeganathan7-May-13 20:55 
QuestionProblem with errro handling simple files Pin
Brian D Hutchison6-Feb-13 6:31
memberBrian D Hutchison6-Feb-13 6:31 
GeneralMy vote of 5 Pin
DevMostafa27-Dec-12 0:51
memberDevMostafa27-Dec-12 0:51 
QuestionSort text file by chars 5 to 100, big files (4GB) Pin
washa7-Dec-12 7:55
memberwasha7-Dec-12 7:55 
AnswerRe: Sort text file by chars 5 to 100, big files (4GB) Pin
Marcos Meli7-Dec-12 8:54
memberMarcos Meli7-Dec-12 8:54 
GeneralRe: Sort text file by chars 5 to 100, big files (4GB) Pin
washa16-Dec-12 11:17
memberwasha16-Dec-12 11:17 
GeneralRe: Sort text file by chars 5 to 100, big files (4GB) Pin
Marcos Meli16-Dec-12 15:47
memberMarcos Meli16-Dec-12 15:47 
GeneralMy vote of 5 Pin
mertg1-Apr-12 4:00
membermertg1-Apr-12 4:00 
QuestionThe directory is not empty. Pin
leeg12121-Feb-12 7:25
memberleeg12121-Feb-12 7:25 
AnswerRe: The directory is not empty. Pin
Jeff Firestone5-Mar-12 9:46
memberJeff Firestone5-Mar-12 9:46 
GeneralLatest version Pin
Jeff Firestone21-Dec-11 9:22
memberJeff Firestone21-Dec-11 9:22 
GeneralRe: Latest version Pin
Gary Noble6-Feb-12 1:52
memberGary Noble6-Feb-12 1:52 
QuestionSeparador costumizado Pin
migc200413-Sep-11 4:33
membermigc200413-Sep-11 4:33 
QuestionMy Vote of 5 Pin
RaviRanjankr4-Sep-11 9:23
memberRaviRanjankr4-Sep-11 9:23 
QuestionProblem with Excel Pin
rainerdam8-Aug-11 12:09
memberrainerdam8-Aug-11 12:09 
GeneralMy vote of 5 Pin
lRUSHl21-Jun-11 0:26
memberlRUSHl21-Jun-11 0:26 
GeneralAdvise Pin
Member 218608623-Mar-11 21:56
memberMember 218608623-Mar-11 21:56 
GeneralRe: Advise Pin
Marcos Meli31-Mar-11 6:42
memberMarcos Meli31-Mar-11 6:42 
GeneralSome missing functionality Pin
bdaniel721-Mar-11 23:44
memberbdaniel721-Mar-11 23:44 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web02 | 2.8.150728.1 | Last Updated 10 Jul 2007
Article Copyright 2005 by Marcos Meli
Everything else Copyright © CodeProject, 1999-2015
Layout: fixed | fluid