Click here to Skip to main content
Click here to Skip to main content

Paradox database native .NET reader

By , 17 Mar 2011
 

Introduction

If you are working on software which accesses a Paradox database, you are probably using BDE (Borland Database Engine). This article shows how to use Paradox database files and read some data from them directly. You can also use the primary index to find specific records you need. This could be useful if BDE doesn't cooperate with you or there is another reason why you cannot or don't want to use BDE.

Background

Every developer sometimes has to deal with some kind of black-box - piece of technology which is not opened, and can be accessed only from some external interface, but there is a problem - the black-box is quite mysterious to us - one time it works, another time it doesn't, and whatever you do, it seems to be having its own moods. And then I always wish it would be opened so I could debug it, trace into it, and look at its teeth. Are you also a developer who has Reflector between quick links? Then you will probably understand why I started to search for some .NET native Paradox database reader. I couldn't find any, but I have found some Paradox format specifications, so I decided to write one myself and make it public for others who could use it. Many thanks to Randy Beck and Kevin Mitchell for sharing the Paradox internal structure, and because their materials are online, I will focus my efforts to describing my own code.

Paradox splits data to files using database objects - every table has its own file, and every index too. Data files and indexes have very similar structure, so we can handle them both in one class - ParadoxFile. BLOB values are also stored in their own files, but I haven't implemented the structure.

  • ParadoxFile - base class working with common structures from Paradox data files and indexes
  • ParadoxFile.DataBlock - represents one block of data
  • ParadoxFile.FieldInfo - data type of the field
  • ParadoxFile.V4Hdr - structure which is present only in certain Paradox files/versions
  • ParadoxTable - represents table data file
  • ParadoxPrimaryKey - represents table index
  • ParadoxFileType - enum with all file types
  • ParadoxFieldType - enum with all data types
  • ParadoxRecord - represents a data record
  • ParadoxDataReader - standard IDataReader implementation

The index file is, in fact, simply a table with indexes of datablocks and related indexed values. You can read it directly, but I also created a mechanism which browses through the index tree for you and picks up the desired record. You will have nothing more to do but specify a condition which you would like to apply on your data.

  • ParadoxCondition - base class for conditions which can be used for searching in index data
  • ParadoxCondition nested classes - various condition implementations
  • ParadoxCompareOperator - enum with supported compare operators (==, !=, <, <=, >, >=)

Class diagram

Using the code

One way to go is just open a table and start enumerating. This is done by the following piece of code. We will create a ParadoxTable object and use the Enumerate method to traverse through records. Data is retrieved synchronously as needed, so if we stop reading after a few records, no redundant read operations are taken.

var table = new ParadoxTable(dbPath, "zakazky");
var recIndex = 1;
foreach (var rec in table.Enumerate())
{
    Console.WriteLine("Record #{0}", recIndex++);
    for (int i=0; i<table.FieldCount; i++)
    {
        Console.WriteLine("    {0} = {1}", table.FieldNames[i], rec.DataValues[i]);
    }
    if (recIndex > 10) break;
}

There are, of course, scenarios where this simple method will not suffice. Sometimes we also have to read some data in the middle of a large database, so we need to use an index to minimize disk operations. In the next sample, we will use a primary key index to find records with key values in the range between 1750 and 1760. At first, the index file has to be opened, then we will create a condition composed from two compare operations. We can browse data by calling the Enumerate method on the index. In this case, we will use the ParadoxDataReader class with the the IDatareader implementation so we don't need to rewrite a lot of code if we have used BDE before.

var index = new ParadoxPrimaryKey(table, Path.Combine(dbPath, "zakazky.PX"));
var condition =
    new ParadoxCondition.LogicalAnd(
        new ParadoxCondition.Compare(
            ParadoxCompareOperator.GreaterOrEqual, 1750, 0, 0),
        new ParadoxCondition.Compare(
            ParadoxCompareOperator.LessOrEqual, 1760, 0, 0));
var qry = index.Enumerate(condition);
var rdr = new ParadoxDataReader(table, qry);
recIndex = 1;
while (rdr.Read())
{
    Console.WriteLine("Record #{0}", recIndex++);
    for (int i = 0; i < rdr.FieldCount; i++)
    {
        Console.WriteLine("    {0} = {1}", rdr.GetName(i), rdr[i]);
    }
}

Limitations

  • read-only solution
  • not all data types are supported
  • no SQL, LINQ, or other complex queries
  • you should not work with live database during writing

History

  • 1.0 - Initial version
  • 1.1 - Updated Number data type support (thanks, Tonki)
  • 1.2 - Additional data type support (thanks to Mark Kuin, Christopher Erker, and Tonki again)

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

About the Author

Petr Bříza
Software Developer (Senior)
Czech Republic Czech Republic
Freelance .NET developer from Ostrava, Czech republic. Mission of my life is to find a question for the answer 42, but in the meantime I try to use .NET and surrounding technologies in behalf of mankind. I hope I bring more good than damage Smile | :)

Sign Up to vote   Poor Excellent
Add a reason or comment to your vote: x
Votes of 3 or less require a comment

Comments and Discussions

 
You must Sign In to use this message board.
Search this forum  
    Spacing  Noise  Layout  Per page   
GeneralRe: I think I do it for suballocated data type (type 3)memberkaiserssosse9-Feb-11 21:34 
I received an email from Christopher and I'll post my code when it's done because now I have to concentrate in other things.
QuestionAny advance with BLOB?memberkaiserssosse16-Jan-11 22:23 
I am trying to solve the problem with the BLOB Binary fields but I can't solve it.
 
I will appreciate any help with this issue.
 
Thanks a lot in advance.
AnswerRe: Any advance with BLOB?memberkaiserssosse19-Jan-11 1:08 
I have this data in .DB about BLOB field: 63 16 0 0 74 4 0 0 1 0 in hex is 3F 10 0 0 4A 4 0 0 1 0 if I reverse:
0 0 10 3F 0 0 4 4A 0 1
44A=1098 length of the BLOB data (type 03. suballocated ) and the index is 103F??? that is my cuestion because if I have this index it doesn't fix with the address of the fisrts data in .MB, where it is in 1150hex address.
 
I need any help to see the light.
 
Thanks a lot in advance.
AnswerRe: Any advance with BLOB?memberChristopher Erker24-Jan-11 2:25 
I remember that when I followed that link for the paradox4.txt file, I googled a bit and found a pretty good C library to read paradox files. In there was code that helped mevwitg memo fields, and should have blob ad well.
 
Here we go, it has a function called PX_GET_DATA_BLOB.
 
Here is the link: http://pxlib.sourceforge.net/index.php
GeneralRe: Any advance with BLOB?memberkaiserssosse24-Jan-11 2:46 
I saw that lib, I solved. The lib was realy usefull.
GeneralRe: Any advance with BLOB?memberChristopher Erker24-Jan-11 4:26 
Cool! Maybe you can post your code in the comments here as well. Perhaps over time we can all develop this into a full .net library.
GeneralRe: Any advance with BLOB?memberkaiserssosse24-Jan-11 21:03 
I will do when I have fully tested, but my solution is only for type 3, suballocated blob type.
GeneralRe: Any advance with BLOB?memberGoran _23-Nov-12 16:33 
Hi Cristopher, I am trying for two dsys to understand how to read memo field. The problem is that I do not understand how is data stored.
 
For example, I have a table which contains a memo field.
 
Record 1: memo field = "Example number 1"
Record 2: memo field = "Example number 10"
Record 3: memo field = "Example number 100"
 
fieldSize for each field is 11 bytes (no matter if memo field contains 1 or 100 characters).
BitConverter returns correct size of the memo string, however the byte array that is passed to GetString function always has only 1 byte (11-10).
 
Can anyone help me with where is the memo string actually stored? Or help me with reading memo fields, I would be very grateful.
AnswerRe: Any advance with BLOB?memberviliam16-Aug-11 10:55 
First of all, this is cool library. Thank you guys.
I fought with blob too and solution for Suballocated block is below.
It is not fully error resistant but hope helps.
 
public byte[] ReadBlob(byte[] blobInfo)
{
if (blobInfo.Length < 10)
return null;
uint OffsetAndIndex = BitConverter.ToUInt32(blobInfo, blobInfo.Length-10); //was 0
uint index = OffsetAndIndex & 0x000000ff;
uint Offset = OffsetAndIndex & 0xffffff00;
 
int size = BitConverter.ToInt32(blobInfo, blobInfo.Length - 6); //was 4
int hsize = 9;
 
int mod_nr = BitConverter.ToInt16(blobInfo, blobInfo.Length - 2); //was 8
 
if (size > 0)
{
this.stream.Position = Offset;
 
byte[] head;
head = new byte[6];
this.reader.Read(head, 0, 3);
 
//first byte B
//00 - Header block
//02 - Single blob block
//03 - Suballocated block
//04 - Free block
//following short is number of 4K blocks
 
//based on _px_get_data_blob in paradox.c library http://pxlib.sourceforge.net/index.php
if (head[0] == 3) //Suballocated block
{
int checks = BitConverter.ToUInt16(head, 1);
byte[] rest;
rest = new byte[9];
this.reader.Read(rest, 0, 9); //read remaining 9 bytes of 12 bytes header
this.stream.Position = Offset + 12 + index * 5; //jump to blob data (header first)
this.reader.Read(head, 0, 5); //read header
if (size != ((int)head[1] - 1) * 16 + head[4])
return null; //Blob does not have expected size
byte[] blobdata;
blobdata = new byte[size];
this.stream.Position = Offset + head[0] * 16;
this.reader.Read(blobdata, 0, size);
return blobdata;
}
viliam

GeneralOverflow bug + solution [modified]memberChristopher Erker1-Jan-11 6:26 
Hi Petr,
 
This is a great little class library. Only thing still missing from it is Memo field support but I will see if I can add it myself.
Years ago, I wrote something very similar for xBase. It was a reader/writer with memo field support. You have inspired me to put up an article about it.
 
I wanted to point out a bug I just found while testing against a database I have. The ParadoxFile class, GetString() method does not respect the maximum string length set by the field. Here is a fix I put in for this.
public string GetString(byte[] data, int from, int maxLength)
{
    int stringLength = Array.FindIndex(data, from, b => b == 0) - from;
    if (stringLength > maxLength)
        stringLength = maxLength;
    return Encoding.Default.GetString(data, from, stringLength);
}
 
UPDATE:
 
I have figured out how to Parse Logical types, and have a very crude Memo field read, I am embarrassed to even include it, but maybe it helps someone... Unfortunately my database only has a 1 Memofield of size 250 bytes so I can't test large memo reads.
 
Inside of ParadoxRecord.DateValues {get;}
    case ParadoxFieldTypes.MemoBLOb:
        val = _block.File.GetStringFromMemo(_block._data, (int) buff.Position, dataSize);
        buff.Position += dataSize;
        break;
    
    case ParadoxFieldTypes.Logical:
        // False is stored as 128, and True looks like 129.
        val = (_block._data[(int)buff.Position] - 128) > 0;
        buff.Position += dataSize;
        break;
 
and inside of ParadoxFile
        public string GetStringFromMemo(byte[] data, int from, int size)
        {
            var memoBufferSize = size - 10;
            var memoDataBuffer = new byte[memoBufferSize];
            var memoMetaData = new byte[10];
            Array.Copy(data, from, memoDataBuffer, 0, memoBufferSize);
            Array.Copy(data, from + memoBufferSize, memoMetaData, 0, 10);
 
            //var offsetIntoMemoFile = (long)BitConverter.ToInt32(memoMetaData, 0); 
            //offsetIntoMemoFile &= 0xffffff00;
            //var memoModNumber = BitConverter.ToInt16(memoMetaData,8); 
            //var index = memoMetaData[0]; 

            var memoSize = BitConverter.ToInt32(memoMetaData, 4);
            return GetString(memoDataBuffer, 0, memoSize);
        }
modified on Sunday, January 2, 2011 5:59 AM

GeneralRe: Overflow bug + solutionmemberkaiserssosse17-Jan-11 0:52 
Any help with BLOB fields?
GeneralRe: Overflow bug + solutionmemberZodraz27-Jan-11 10:29 
Cool!!!!
 
Your updates are so much appreciated!!
 
Thx
GeneralRe: Overflow bug + solutionmemberPetr Bříza16-Mar-11 21:01 
Thank you for the sources, I cannot test it but I believe in you Wink | ;-) This will be included in the article.
GeneralRe: Overflow bug + solutionmemberSDMX211-Jul-11 22:15 
If from are equal to data.Length-1 then Array.FindIndex(data, from, b => b == 0) returns -1 and ArgumentOutOfRangeException will thrown in Encoding.Default.GetString().
 
So I use this code instead:
public string GetString(byte[] data, int from, int maxLength)
{
	int stringLength;
	for (stringLength = from; stringLength < data.Length 
		&& data[stringLength] != 0; stringLength++) ;
	stringLength -= from;
	if (stringLength > maxLength)
		stringLength = maxLength;
	return Encoding.Default.GetString(data, from, stringLength);
}

GeneralRe: Overflow bug + solution [modified]memberGoran _26-Aug-12 12:31 
I have tried all possible propositions in this messages for BlobMemo and Logical, and both of them do not work as expectesd. I am curious through, I have just tried to connect using MS OleDb and Odbc drivers, and both work. Is there some gotcha that I am not aware of, so I need to use this library?
Generalfile formatmemberPeter1537423-Dec-10 4:56 
Hi Petr,
 
I have got an internal question. Is it right that last record of data block is set again in next data block?
 
Thx Peter
GeneralRe: file formatmemberkaiserssosse19-Jan-11 2:17 
I don't think so, but confirm it please and if you know how to know the address of BLOB data in .MB please let me know.
Generalnumeric valuesmemberTonki14-Sep-10 5:37 
very good job, thank you!
 
greetings from vienna
 
2 suggestions:
 

// Numeric values should be decoded this way:
private void ConvertBytesNum(int start, int length) /* amk */
{
    if ( ((byte)(this.block.data[start])&0x80)!=0)
        this.block.data[start] = (byte)(this.block.data[start] &0x7F);
    else if (this.block.data[start+0]==0 &&
                this.block.data[start+1]==0 &&
                this.block.data[start+2]==0 &&
                this.block.data[start+3]==0 &&
                this.block.data[start+4]==0 &&
                this.block.data[start+5]==0 &&
                this.block.data[start+6]==0 &&
                this.block.data[start+7]==0 ) /* sorry, did not check lenght */
        ;
    else for(int i=0;i<8;i++)
            this.block.data[start + i] = (byte)(~(this.block.data[start + i]));
 
    Array.Reverse(this.block.data, start, length);
}
 

// since ParadoxFieldTypes.Number usually is integer:
case ParadoxFieldTypes.Number:
    ConvertBytesNum((int)buff.Position, dataSize);
    var dbl = r.ReadDouble(); 
    if (double.IsNaN(dbl))
        val = (object)DBNull.Value;
    else if (dbl == Convert.ToInt32(dbl)) /* sorry, min/max-check missing */
        val = Convert.ToInt32(dbl);
    else
        val = dbl;

GeneralRe: numeric valuesmemberPetr Bříza17-Sep-10 1:17 
Hi, I have updated my article, thank you for you suggestions, but I disagree with retyping double type to integer just because of the current value. The code using this library could count on the specific data type and returning various data types for each row and the same column could be very confusing.
 
Petr
GeneralRe: numeric valuesmemberTonki18-Oct-10 2:26 
two more suggestions:
 
ParadoxReader.ParadoxRecord.Datavalues.get():
 
case ParadoxFieldTypes.Date:
    ConvertBytes((int)buff.Position, dataSize);
    var days = r.ReadInt32();
    val = new DateTime(1, 1, 1).AddDays(days).AddDays(-1); // Tonki: AddDays(-1)
    break;
case ParadoxFieldTypes.Timestamp:
    ConvertBytes((int)buff.Position, dataSize);
    var ms = r.ReadDouble();
    val = new DateTime(1, 1, 1).AddMilliseconds(ms).AddDays(-1);
    break;
case ParadoxFieldTypes.Time:
    ConvertBytes((int)buff.Position, dataSize);
    val = r.ReadInt32(); // Tonki: read time as milliseconds
    break;
 
 
greetings
tonki
GeneralRe: numeric valuesmemberPetr Bříza16-Mar-11 21:08 
Thank you again for your suggestions Smile | :)
QuestionDo you plan on implementing the BLOB structure?memberelemeht1-Sep-10 12:01 
I see where you say you haven't implemented its structure, but do you plan to do so?
 
I have been trying to figure out how to pull data from a paradox database, and your project works great.
 
If you do not plan on doing so, can you point me in the right direction? I'd need to get blob data out too.
AnswerRe: Do you plan on implementing the BLOB structure?memberPetr Bříza17-Sep-10 0:56 
I am sorry, but I do not plan to implement BLOB support in the near future. You can find some information about BLOB data storage here: http://www.randybeck.com/files/pxformat.zip, file PARADOX4.TXT. It is not difficult, a data record just contain a position and length of the binary data in the .MB file.
GeneralRe: Do you plan on implementing the BLOB structure?memberkaiserssosse14-Jan-11 0:03 
Any advance in the BLOB structure? I'm very interested.
 
Thank you.
GeneralRe: Do you plan on implementing the BLOB structure?memberMark Kuin31-Jan-11 2:53 
I have some code which reads BLOB structures, which works in my case. I hope it helps you too. I'm not a C# developer, so it may contain some bugs...
 
Add this type
    public delegate byte[] ReadBlobDelegate(byte[] blobInfo);
 
Add this field to ParadoxFile
        public ReadBlobDelegate readBlob;
 
Add this to the DataValues property of ParadoxRecord
case ParadoxFieldTypes.BLOb:
                                    if (this.block.file.readBlob != null)                                    {
                                        byte[] blobInfo;
                                        blobInfo = new byte[dataSize];
                                        r.Read(blobInfo, 0, dataSize);
 
                                        val = this.block.file.readBlob(blobInfo);
                                    }
                                    else
                                    {
                                        val = null; // not supported
                                        buff.Position += dataSize;
                                    }
                                    break;
 
Add this field to the ParadoxTable class
        private readonly ParadoxBlobFile BlobFile; //Added mjk
 
Add this code to the ParadoxTable constructor
                //Added mjk
                if (Path.GetFileNameWithoutExtension(file).EndsWith(".MB", StringComparison.InvariantCultureIgnoreCase) ||
                    Path.GetExtension(file).Equals(".MB", StringComparison.InvariantCultureIgnoreCase))
                {
                    this.BlobFile = new ParadoxBlobFile(file);
                    this.readBlob = this.BlobFile.ReadBlob;
                }
 
Add this code the the ParadoxTable dispose procedure
            if (this.BlobFile != null)
            {
                this.BlobFile.Dispose();
            }
 
Add this class to the ParadoxTable class
        //Added mjk
        internal class ParadoxBlobFile : IDisposable
        {
            private readonly Stream stream;
            private readonly BinaryReader reader;
 
            public ParadoxBlobFile(string fileName)
                : this(new FileStream(fileName, FileMode.Open, FileAccess.Read, FileShare.ReadWrite))
            {
            }
 
            public ParadoxBlobFile(Stream stream)
            {
                this.stream = stream;
                this.reader = new BinaryReader(stream);
            }
 
            public virtual void Dispose()
            {
                this.stream.Dispose();
            }
 
            public byte[] ReadBlob(byte[] blobInfo)
            {
                uint OffsetAndIndex = (uint)System.Runtime.InteropServices.Marshal.ReadInt32(blobInfo, 0);
                uint index = OffsetAndIndex & 0x000000ff;
                uint Offset = OffsetAndIndex & 0xffffff00;
 
                int size = System.Runtime.InteropServices.Marshal.ReadInt32(blobInfo, 4);
                int hsize = 9;
 
                int mod_nr = System.Runtime.InteropServices.Marshal.ReadInt16(blobInfo, 8);
 
                if (size > 0)
                {
                    //Console.WriteLine("Graphic index={0}; blobsize={1}; mod_nr={2}", index, blobsize, mod_nr);

                    this.stream.Position = Offset;
 
                    byte[] head;
                    head = new byte[6];
                    this.reader.Read(head, 0, 3);
 
                    //TODO check for type 2 and index=255

                    this.reader.Read(head, 0, hsize - 3); //Read remaining 6 bytes of header
                    int checkSize = System.Runtime.InteropServices.Marshal.ReadInt32(head, 0);
                    if (checkSize == size)
                    {
                        byte[] buffer;
                        buffer = new byte[size];
 
                        this.reader.Read(buffer, 0, size);
                        return buffer;
                    }
                }
                return null;
            }
        }

GeneralRe: Do you plan on implementing the BLOB structure?memberPetr Bříza16-Mar-11 21:39 
That is great, thank you very much for your code, I will include to the article for others who could use it. I cannot test it but I believe you spent enough time on this. If there will be some bugs, please let me know here.
 
Petr
GeneralRe: Do you plan on implementing the BLOB structure?memberMark Kuin8-Nov-11 0:12 
Minor addition to the source code: the ParadoxFieldTypes.Graphic field type also uses the ReadBlob function (at least it does in my case).
 
                                case ParadoxFieldTypes.Graphic:
                                case ParadoxFieldTypes.BLOb:
                                    var blobInfo = new byte[dataSize];
                                    r.Read(blobInfo, 0, dataSize);
                                    val = this.block.file.ReadBlob(blobInfo);
                                    break;

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Permalink | Advertise | Privacy | Mobile
Web02 | 2.6.130619.1 | Last Updated 17 Mar 2011
Article Copyright 2010 by Petr Bříza
Everything else Copyright © CodeProject, 1999-2013
Terms of Use
Layout: fixed | fluid