Click here to Skip to main content
12,896,826 members (56,722 online)
Click here to Skip to main content
Add your own
alternative version

Stats

31.2K views
23 bookmarked
Posted 22 Jul 2009

Creation and memory mapping existing DBF files as alternative of data serialization during work with modified CListCtrl classes in virtual mode on dialogs of MDI application

, 22 Jul 2009 CPOL
Rate this:
Please Sign up or sign in to vote.
The demonstration of reading, writing and creation of standard DBF files with random access in memory instead of serialization of typical MFC application for descendants of CListCtrl classes in Virtual Mode.

Introduction

At last we can set ourselves the task of conservation and read-out of necessary information for our application from files of data. In typical MFC program standard serialization of data is usually supported with CDocument classes. In principle nothing interferes to us to use this method for our aims. However, at once does a question get up in what format to keep information? Certainly we can invent an own data format, appropriate it our extension, for example, *.eee and to work with it as this will be comfortably us. Perhaps, for educational aims or there where principle of serialization of information has a substantial value, this method is very quite good. But, as we design working application in which it is necessary to have an random access to file memory, the use of serialization does not seem an optimum enough decision for us already. In addition, for development of own format of data are needed very weighty grounds. I think that satisfaction of own ego is not such cause.

So, at first, we must be determined with the existent format of data, and secondly, to define a method of random access to file information. At such wording, decision of these tasks are fully obviously. The simplest and well known format of data is structure of dbf-file which gives a good account of itself. And random access to content of files of data easily to carry out taking advantages of technology of memory mapped files – MMF. Main desert of this technology is in that we work with files as with ordinary memory. For databases files it is an excellent decision. There is not a necessity preliminary to read information in ordinary memory for their further use, although we can do it for small but often using information, with the purpose of optimization, for example for meta data, describing structure of a database. In addition, an obvious necessity to be engaged in spooling of data falls off, that is very comfortably for filling of CListCtrl classes and its descendants in the virtual mode. Except for it, we can directly use static structure of database for random access to its elements of data. Plus lightness of the use of technology of MMF. Advantages are so much, that has no sense to talk about serialization of information, while practically at those efforts we get all of advantages of random access to memory mapped files.

On this stage we define organization of work thus. At opening a certain table form, the program searches proper by it a dbf-file, described in our meta data. If finds, tries to open its and use its information for filling of a proper list, otherwise creates this dbf-file, reading information from our static variables. If to change such file by some external editor program of dbf-files, at repeated his read-out we will see new information already. Later we will learn to edit dbf-files directly in our program. This project is already supplied with the prepared files, located in the Dbf folder. Its names are «First.dbf», «Second.dbf» and «Third.dbf». They have an alike structure (in the third file the fields «Name» and «Title» moved between itself) but its data are sorted variously. In all from them there are 360 records. If for example we delete the file «Third .dbf», instead of its our program will be to create the same file and will fill its data with our static variables. As the result we will get an example indicated on a Fig. 1.

Tables04.jpg

Fig. 1. Filling of lists in the Virtual Mode by information from dbf-files, including created.

1. Structure of DBF file

In the Internet is not very much hardness to find necessary information about dbf-files, as this format is opened and a long ago known. However succeeded finding an official specification was not. Therefore we will expound short results about their structure, see Fig. 2.

Dbf04.jpg

Fig. 2. Schema of dbf-file and range of definition of DBF_FILE1 and DBF_FILE2 structures.

If we know beforehand an amount of fields FLDCOUNT = N in a database, their lengths FLDLEN1 = M1, FLDLEN2 = M2, . . ., FLDLENn = Mn and number of records RECCOUNT = K, the proper structure of dbf-file can be defined as:

//*** The dbf file structure
typedef struct {
    DBF_HEADER DbfHdr;
    DBF_FIELD aDbfField[FLDCOUNT];  // FLDCOUNT = (DbfHdr.wFirstRec-296)/32 for VFP

    BYTE cHdrEnd;  // Header record terminator = 0x0D (13)

    BYTE acDbcFile[263];  // A data range for name of associated *.dbc file or contains 0x00

    DBF_RECORD aDbfRec[RECCOUNT];  // RECCOUNT = DbfHdr.nRecCount

    BYTE cFileEnd;  // File record terminator = 0x1A (26)

} DBF_FILE;

where the proper structures are determined as:

//*** The dbf header structure
typedef struct {
    /*00-00*/ BYTE cType;  // File type = 0x30 (48) for Visual FoxPro (VFP)

    /*01-01*/ BYTE cYear;  // Year of last update (for VFP nFullYear = 2000 + cYear)

    /*02-02*/ BYTE cMonth;  // Month of last update

    /*03-03*/ BYTE cDay;  // Day of last update

    /*04-07*/ ULONG nRecCount;    // Number of records in file

    /*08-09*/ WORD wFirstRec;  // Position of first data record

    /*10-11*/ WORD wRecSize;  // Length of one data record, including delete flag

    /*12-27*/ BYTE cReserv1227[16];  // Reserved, contains 0x00

    /*28-28*/ BYTE cFlag;  // Table Flags (Only Visual FoxPro)

    /*29-29*/ BYTE cCodePage;  // Code page mark = 0x03 (3) for code page 1252 (Windows ANSI)

    /*30-31*/ WORD wReserv3031;  // Reserved, contains 0x00

} DBF_HEADER;
//*** The dbf field structure
typedef struct {
    /*00-10*/ BYTE cName[11];  // Field name with null terminator (0x00)

    /*11-11*/ BYTE cType;  // Field type (C – Character; N – Numeric; D – Date and so on)

    /*12-15*/ ULONG nOffset;  // Displacement of field in record

    /*16-16*/ BYTE cLen;  // Length of field (in bytes)

    /*17-17*/ BYTE cDec;  // Number of decimal places

    /*18-18*/ BYTE cFlag;  // Field flag

    /*19-22*/ LONG nNext;  // Value of auto increment Next value 

    /*23-23*/ BYTE cStep;  // Value of auto increment Step value 

    /*24-31*/ BYTE cReserv2431[8];  // Reserved, contains 0x00

} DBF_FIELD;
//*** The dbf record structure
typedef struct {
    BYTE cDelete;  // Delete flag

    BYTE cData1[FLDLEN1];  // Where FLDLEN1 = aDbfField[0].cLen

    BYTE cData2[FLDLEN2],  // Where FLDLEN2 = aDbfField[1].cLen

    . . .
    BYTE cDataN[FLDLENN];  // Where FLDLENN = aDbfField[N-1].cLen

} DBF_RECORD;

Here is showed the static structure of dbf-file actually (we are oriented on the VFP format with the file type 0x30 (48)). For dynamic structures in ordinary memory, organization of data must be other – through pointers (size of which is known) to complete structures of data, sizes of which are unknown on the stage of compiling of program (or not desirable for the use, not to limit to ourselves with a concrete structure of one file). And as the compiler of VS6 C++ can not create dynamic types of data, size of which turns out only on the stage of executing of program (runtime), we can not use the structure of data obviously described higher for direct manipulation file information. But we also do not wish preliminary to read information of records in ordinary memory, to analyze and convert its directly to use in future. The same we lose advantages of the MMF. Thus, gets up the task of maximal description of dynamic structures of data with static ones, with the purpose of effective selection of elements of data from databases (in this case dbf-files) on the stage of executing of the program.

2. Determination of dynamic structures (unknown sizes) via static ones

This theme is important enough in itself, to spare it a little bit of attention. Ordinary compilers can not create dynamic types of data, when sizes of structures are ascertained in runtime. Just the «data type» bears in mind but not a dynamical storage area is reserved under it. We will illustrate said the known example. We will assume want to create an array of bytes for length which we determine dynamically, for example:

UINT nX = 100;
BYTE acData[nX];

But on this very simple construction a compiler will «swear». instead of nX it wants to see constant expression, as though

BYTE acData[100];

Thus, constant expression must be indicated obviously. For example, expression:

UINT nCount  = sizeof(aKnownStructure)/ sizeof(aKnownStructure[0]);
acData[nCount];

will be erroneous, while the direct use:

acData[sizeof(aKnownStructure)/ sizeof(aKnownStructure[0])];

will be faithful.

Nevertheless, building the indicated array of the dynamically calculated size is possible if only to give up obvious static determination of array and substitute it by non-obvious static determination. I.e. instead of

BYTE acData[nX];  // Correctly, BYTE acData[100];

we write

BYTE *acData = new BYTE[nX];  // UINT nX = 100;

A difference is here in that in first case of sizeof(acData) = 100, and in second sizeof(acData) = 4. I.e. in second case, actually we has not dynamic type, measuring 100 bytes, and static one of length 4 bytes, because it is a size of pointer to a dynamically selected storage area measuring 100 bytes. From viewpoint of the use of these elements of arrays are differences no, and from viewpoint of sizes of in-use structures difference is substantial. For an example will show yet how dynamically to build a bi-dimensional array.

//*** Dynamic array of field names of dbf file

BYTE **aaсFldName = new BYTE *[m_nFldCount];

//*** Initializes arrays of names of dbf fields

for(int i = 0; i < m_nFldCount; i++) {
    //*** Field name with a maximum of 10 characters with null terminator

    m_aacFldName[i] = new BYTE[11];

    //*** Copies field name

    for(int j = 0; j < 11; j++)
            m_aacFldName[i][j] = aDbfField[i].acName[j];
}

It is the real code from our application. Apparently, this technique is good for copying of dynamic structures of unknown beforehand (on the stage of compiling) size. But that well for often in-use and not largeness of meta data, is not very much well for enormous arrays of elements of data. It would be desirable directly to handle to them at their use (without a preliminary copying and analysis of structure of data). However for this purpose it is necessary beforehand to know key parameters of file of database, what on their basis to build the static types of complete structures of data. As it applies to the structure of dbf-file described higher, obvious knowledge of the parameters indicated there is needed. But as mentioned already, even if these parameters for this file know us, it is not interestingly us to specify them statically in our program, because it limits to us consideration only exactly this file. Certainly, we can beforehand describe the whole great number of dbf-files, in-use in our program, as it is ordinary and done, but however, if we wish to have an access to random dbf-file, this technique does not arrange us.

So, we walked up to the contradiction. From one side, static description of structure of database (as it is done at the beginning of the first section) allows us to take advantage memory mapped files, but limits to us the fixed set of these structures (i.e. by the certain amount of concrete types of files). From other side dynamic description of structures of data through pointers to structures, unknown beforehand sizes, results in the necessity of the use of flow of entrance data of unknown structure, that deprives us obvious advantages of MMF technology, namely random access to elements of a database.

For an example will specify that second a way was went author of «The alxBase classes for work with DBF files» by Alexey, therefore, MMF technician was not used him. He applies the classic method of retrieval of data from a dbf-file by the obvious positioning of file pointer and then reading of data elements. However one thing when we directly write a sort of (in right part it is file memory elements):

BYTE *acDataElement = acRecord[j].acField[i];  // ~ BYTE acDataElement[aDbfField[i].cLen];

and another, that at first we must calculate file pointer of current element of data, positioned on it and only after have read information. But even it is not main (for the dbf-file it is not difficult to do it by ourselves), but circumstance that it is needed to work at organization of share (multi-user , multi-thread) access to common data yet. As far as I understood Alexey limited to monopolistic access to database, that limits the use of his library substantially. It is visible already because he withdrew support his project from 2005 (at least, in public). In MMF technology considerably anymore possibilities for organization of share access to data, so that, I think, it is not needed to ignore this possibility.

But will go back to our «contradiction». Clear, that we are not arranged by neither first nor second way. If someone knows a third way, it would be interestingly to know about it. We will endeavour to unite these methods, so, as far as it is possible. As we do not can beforehand fully statically to describe a dbf-file structure, limited to then partial static description, thus by not one similar structure, but two. If we once again attentively will look at the structure of dbf-file resulted higher (which in such kind will not be «understood» by a compiler), easily to see that in it it’s possible to select two static parts, which already will be «clear» a compiler. Namely:

//*** The dbf file structure (Part No. 1)
typedef struct {
    DBF_HEADER DbfHdr;  // Dbf header structure

    DBF_FIELD aDbfField[1];  // Really FLDCOUNT = (DbfHdr.wFirstRec-296)/32 fields

} DBF_FILE1;
//*** The dbf file structure (Part No. 2)
typedef struct {
    BYTE cHdrEnd;  // Header record terminator = 0x0D (13)

    BYTE acDbcFile[263];  // A data range for associated *.dbc file or contains 0x00

    BYTE aDbfRec[1];  // Really DbfHdr.wRecSize * DbfHdr.nRecCount records

    //BYTE cFileEnd;  // File record terminator = 0x1A (26)

} DBF_FILE2;

Further, we will take advantage of the known «hacker» technique – conscious output outside the statically declared array of data. «Paying» for such combined approach will be a transition from «flat», «bi-dimensional» indexation (j, i) to «one-dimensional», linear indexation of ji = j*nColCount + i (for a count of linear number of cell of information) and necessity of calculation of general linear displacement nLineInd = j*nRowSize + m_anOff[i] (in bytes). But, I think, that it is not too large «price», for such approach.

Exactly these structures we will really attach to a dbf-file, applying after a conscious output outside indexes of the static arrays of structures aDbfField[1] and aDbfRec[1], by the mentioned linear indexes. As practice showed, it is fully good approach and it can be used for other data files, for example, for memo-fields (fpt-files), index cdx-files, database containers (dbc-files) etc.

3. MMF and reading of existent DBF files

Reading of having dbf-file takes a place by MMF technology, the use of which is presented in the function CMainDoc::OnOpenDocument:

/////////////////////////////////////////////////////////////////////////////
// OpenDocumentFile

/////////////////////////////////////////////////////////////////////////////
BOOL CMainDoc::OnOpenDocument(LPCTSTR szFileName) {  // szFileName isn't using

    TCHAR *szDbfName = m_MetaTable.szDbfName;

    CFileStatus FileStatus;

    //*** If file szDbfName does not exist creates its

    if(!CFile::GetStatus(szDbfName, FileStatus)) {
        //*** Creates current document (dbf) file on physical disk

        if(!CreateDocumentFile(szDbfName)) {
            //_M("CMainDoc: Failed to create a document file!");

            return FALSE;
        }
    }

    //*** Gets handle of dbf file

    m_hDbfFile = ::CreateFile(
            szDbfName,  // Name of dbf file

            GENERIC_READ | GENERIC_WRITE,  // Access (read-write) mode

            FILE_SHARE_READ | FILE_SHARE_WRITE,  // Share mode

            NULL,  // Pointer to security attributes

            OPEN_EXISTING,  // How to create

            FILE_ATTRIBUTE_NORMAL,  // File attributes

            NULL  // HANDLE hTemplateFile - Handle to file with attributes to copy

    );

    if(m_hDbfFile == INVALID_HANDLE_VALUE) {
        _M("CMainDoc: Failed to call ::CreateFile function!");
        return FALSE;
    }

    //*** The message buffer

    TCHAR szStr[MAXITEMTEXT];

    //*** Gets file size in bytes

    ULONG nDbfSize = ::GetFileSize(
        m_hDbfFile,  // Handle of file to get size of

        NULL  // Pointer to high-order word for file size

    );

    //*** Checks file size

    if(nDbfSize == 0) {
        swprintf(
                szStr,
                _T("CMainDoc: File '%s' is empty!"),
                szDbfName
        );

        _M(szStr);
        ::CloseHandle(m_hDbfFile);
        return FALSE;
    }

    //*** Creates file mapping

    m_hDbfMap = ::CreateFileMapping(
            m_hDbfFile,  // Handle to file to map

            NULL,  // Optional security attributes

            PAGE_READWRITE,  // Protection for mapping object

            0,  // High-order 32 bits of object size

            0,  // Low-order 32 bits of object size

            NULL  // Name of file-mapping object

    );

    if(!m_hDbfMap) {
        _M("CMainDoc: Failed to call ::CreateFileMapping function!");
        return FALSE;
    }

    //*** Maps view of dbf file for its first part (where are header)

    m_pDbfView1 = reinterpret_cast<DBF_FILE1 *>(::MapViewOfFile(
            m_hDbfMap,  // File-mapping object to map into address space

            FILE_MAP_WRITE,  // Access mode

            0,  // High-order 32 bits of file offset

            0,  // Low-order 32 bits of file offset

            0  // Number of bytes to map (if it is zero, the entire file is mapped)

    ));

    if(!m_pDbfView1) {
        _M("CMainDoc: Failed to call ::MapViewOfFile function!");
        return FALSE;
    }

    //*** Dbf header structure

    m_pDbfHdr = &m_pDbfView1->DbfHdr;

    //*** Checks dbf file type

    if(m_pDbfHdr->cType != VFPTYPE) {  // = 0x30 (48)

        swprintf(
                szStr,
                _T("CMainDoc: Dbf type: %d doesn't equal to VFP type: %d!"),
                m_pDbfHdr->cType,
                VFPTYPE
        );

        _M(szStr);
        return FALSE;
    }
    /*
    //*** Shows date of last update
    
    swprintf(
            szStr, 
            _T("Date of last update is %0.2d.%0.2d.%d"), 
            m_pDbfHdr->cDay, 
            m_pDbfHdr->cMonth, 
            BASEYEAR + m_pDbfHdr->cYear
    );

    _M(szStr);
    */
    //*** Number of records in file

    m_nRecCount = m_pDbfHdr->nRecCount;

    //*** Length of one data record (including delete flag)

    m_nRecSize = m_pDbfHdr->wRecSize;

    //*** Checks record size

    if((m_nRecSize == 0 && m_nRecCount != 0) ||
         (m_nRecSize != 0 && m_nRecCount == 0)) {
        swprintf(
                szStr,
                _T("CMainDoc: Not matches record size (%d) and record count (%d)!"),
                m_nRecSize,
                m_nRecCount
        );

        _M(szStr);
        return FALSE;
    }

    //*** Calculates size of all records

    ULONG nDataSize = nDbfSize - 1 - m_pDbfHdr->wFirstRec;

    //*** Checks record parameters

    if(nDataSize != m_nRecCount*m_nRecSize) {
        swprintf(
                szStr,
                _T("CMainDoc: Data size (%d) doesn't equal record count (%d) * record size (%d)!"),
                nDataSize,
                m_nRecCount,
                m_nRecSize
        );

        _M(szStr);
        return FALSE;
    }

    //*** Number of fields in file (for Visual FoxPro only)

    m_nFldCount = (m_pDbfHdr->wFirstRec - 296)/32;

    //*** Checks field count

    if(m_nFldCount > m_nRecSize - 1) {
        _M("CMainDoc: Field count is very large!");
        return FALSE;
    }

    //*** Dbf field structure

    DBF_FIELD *aDbfField = m_pDbfView1->aDbfField;

    //*** Maps view of dbf file for its first part (where are data)

    m_pDbfView2 = reinterpret_cast<DBF_FILE2 *>(
            &m_pDbfView1->aDbfField[m_nFldCount].acName[0]
    );

    BYTE cHdrEnd = 0;

    //*** Checks dbf reading

    try {
        cHdrEnd = m_pDbfView2->cHdrEnd;
  } catch(...) {
        _M("CMainDoc: Dbf file has wrong structure!");
        return FALSE;
    }

    //*** Checks dbf header record terminator

    if(cHdrEnd != HEADEREND) {  // = 0x0D (13)

        swprintf(
                szStr, 
                _T("CMainDoc: Header record terminator: %d doesn't equal to: %d!"),
                m_pDbfView2->cHdrEnd,
                HEADEREND
        );

        _M(szStr);
        return FALSE;
    }

    //*** Delete flag  // = " " or "*"

    //_M(pDbfFile2->aDbfRec[0]);


    //*** Dynamic array of field names

    m_aacFldName = new BYTE *[m_nFldCount];

    //*** Dynamic array of field types

    m_acFldType = new BYTE[m_nFldCount];

    //*** Dynamic array of offsets

    m_anOff = new UINT[m_nFldCount];
    
    //*** Dynamic array of lengths

    m_acLen = new BYTE[m_nFldCount];
    
    //*** Dynamic array of decimal places

    m_acDec = new BYTE[m_nFldCount];

    //*** Initializes arrays of length, offsets and etc. of dbf fields

    for(int i = 0; i < m_nFldCount; i++) {
        //*** Field name with a maximum of 10 characters with null terminator

        m_aacFldName[i] = new BYTE[11];

        //*** Copies field name

        for(int j = 0; j < 11; j++)
                m_aacFldName[i][j] = aDbfField[i].acName[j];

        //*** Field type

        m_acFldType[i] = aDbfField[i].cType;

        //*** Field lenght (in bytes)

        m_acLen[i] = aDbfField[i].cLen;

        //*** Number of decimal places (in bytes)

        m_anOff[i] = aDbfField[i].nOffset;

        //*** Number of decimal places (in bytes)

        m_acDec[i] = aDbfField[i].cDec;
    }

    //*** Testing for all field of j-th record

    /*
    //*** j-th record
    ULONG j = 11;
    
    //*** Line displacement of i-th field of j-th record
    ULONG ji = 0;
    
    for(i = 0; i < m_nFldCount; i++) {
        ji = j*m_nRecSize + m_anOff[i];

        //*** The copy of (j, i) field value of m_anLen[i]-th length
        // As it hasn't null terminator
        CString sFldVal((LPCSTR) &m_pDbfMap2->aDbfRec[ji], m_anLen[i]);

        sFldVal.TrimLeft();
        sFldVal.TrimRight();

        swprintf(
                szStr, 
                _T("%s : %c : %d : %d.%d :: '%s'"),
                (CString) aszFldName[i],  // As it has null terminator
                acFldType[i],
                m_anOff[i],
                m_anLen[i],
                m_anDec[i],
                sFldVal
        );

        _M(szStr);
    }
    */
    BYTE cFileEnd = 0;

    //*** Checks dbf reading

    try {
        cFileEnd = m_pDbfView2->aDbfRec[m_nRecCount * m_nRecSize];
  } catch(...) {
        _M("CMainDoc: Dbf file has wrong structure!");
        return FALSE;
    }

    //*** Checks dbf file record terminator

    if(cFileEnd != DBFEND) {  // = 0x1A (26)

        swprintf(
                szStr, 
                 _T("CMainDoc: Header record terminator: %d doesn't equal to: %d!"),
                m_pDbfView2->aDbfRec[m_nRecCount * m_nRecSize], 
                DBFEND
        );

        _M(szStr);
        return FALSE;
    }

    //*** Current table

    CListCtrlEx *pTable = m_pMainApp->m_apTable[m_eTable];
    
    if(!pTable) {
        _M("CMainDoc: Empty a CListCtrlEx object!");
        return FALSE;
    }

    //*** Sets the table rows count in the virtual mode (LVS_OWNERDATA)

    //*** Send messages LVN_GETDISPINFOW & HDM_LAYOUT

    //*** Calls the CListCtrlEx::DrawItem

    pTable->SetItemCount(m_nRecCount);

    //*** Shows the vertical scroll bar always

    //pTable->ShowScrollBar(SB_VERT);


    //*** Saves the current document

    m_pMainApp->m_apDoc[m_eTable] = this;

    return TRUE;
}  // OnOpenDocument

4. Creation of DBF files and filling their with static data

To create a file of database, it is needed preliminary to know its structure. In our demonstration program there are three such static structures which allow to create three different dbf-files. Changing their amount and content it is possible to create random enough files of the Visual FoxPro format of databases. Here are basic structures of our meta data:

//*** The dbf-file fields data structure
typedef struct {
    TCHAR *szFldName;  // Field name

    TCHAR *szFldType;  // Field type

    UINT nFldLen;  // Field length (in bytes)

    UINT nDecLen;  // Number of decimal places (in bytes)

} META_DATA;
//*** The meta table header structure
typedef struct {
    TCHAR *szHdrName;  // Column name

    DWORD nAdjust;  // Text formatting

    UINT nWidth;  // Column width

} META_HEADER;
//*** The meta table structure
typedef struct {
    TCHAR *szDbfName;  // Dbf name

    META_DATA *aMetaData;  // Dbf-file fields data structure

    TCHAR *szTblName;  // Table name

    META_HEADER *apMetaHeader;  // Meta table header structure

    DWORD dwStyle;  // Table style

    DWORD dwExStyle;  // Extended table style

    RECT *pFrmRect;  // Frame rectangle pointer

    RECT *pViewRect;  // View rectangle pointer

    CFont *pHdrFont;  // Table header font pointer

    CFont *pListFont;  // Table list font pointer

    UINT nHdrHeight;  // Table header height

    UINT nListHeight;  // Table list height

    UINT nColCount;  // Table header columns count

    UINT nRowCount;  // Table list row count

    TCHAR **apRowText;  // Table rows text array

} META_TABLE;

Using these structures the function CMainApp::CreateDocumentFile creates a necessary dbf-file:

/////////////////////////////////////////////////////////////////////////////
// CreateDocumentFile

/////////////////////////////////////////////////////////////////////////////
BOOL CMainDoc::CreateDocumentFile(TCHAR *szDbfName) {
    //*** The dbf file structure

    /*
    typedef struct {
        DBF_HEADER DbfHdr;
        DBF_FIELD aDbfField[FLDCOUNT];  // FLDCOUNT = (DbfHdr.wFirstRec-296)/32 for VFP
        BYTE cHdrEnd;  // Header record terminator = 0x0D (13)
        BYTE acDbcFile[263];  // A data range for associated *.dbc file, contains 0x00
        DBF_RECORD aDbfRec[RECCOUNT];  // RECCOUNT = DbfHdr.nRecCount
        BYTE cFileEnd;  // File record terminator = 0x1A (26)
    } DBF_FILE;
    */
    CFileStatus FileStatus;

    //*** If file szDbfName does exist simply return

    if(CFile::GetStatus(szDbfName, FileStatus))
            return TRUE;

    //*** Number of records in file

    ULONG nRecCount = m_MetaTable.nRowCount;

    //*** Data table fields count

    UINT nFldCount = m_MetaTable.nColCount;

    SYSTEMTIME SysTime = {0};

    //*** Gets system date and time

    GetSystemTime(&SysTime);

    //*** Dbf header structure


    DBF_HEADER DbfHdr = {0};

    DbfHdr.cType = VFPTYPE;  // DBF type

    DbfHdr.cYear = SysTime.wYear%BASEYEAR;  // Year of last update

    DbfHdr.cMonth = SysTime.wMonth;  // Month of last update

    DbfHdr.cDay = SysTime.wDay;  // Day of last update

    DbfHdr.nRecCount = nRecCount;  // Number of records in file

    //DbfHdr.wFirstRec = 0;    // Position of first data record

    //DbfHdr.wRecSize = 0;  // Length of one data record, including delete flag

    //DbfHdr.cFlag = 0;     // Table Flags (Only Visual FoxPro)

    DbfHdr.cCodePage = CP1252;    // Windows ANSI


    //*** Dbf field structure


    DBF_FIELD DbfField = {0};

    //*** Gets handle of dbf file

    HANDLE hDbfFile = ::CreateFile(
            szDbfName,  // Pointer to name of the dbf file

            GENERIC_WRITE,  // Access (read-write) mode

            0,  // Share mode

            NULL,  // Pointer to security attributes

            CREATE_ALWAYS,  // How to create

            FILE_ATTRIBUTE_NORMAL,  // File attributes

            NULL  // HANDLE hTemplateFile - Handle to file with attributes to copy

    );

    //*** Message buffer

    TCHAR szStr[MAXITEMTEXT];

    //*** Checks file creation

    if(hDbfFile == INVALID_HANDLE_VALUE) {
        swprintf(
                szStr,
                _T("CMainApp: Failed to create new file: '%s'!"),
                szDbfName
        );

        _M(szStr);
        ::CloseHandle(hDbfFile);
        return FALSE;
    }

    //*** Number of written bytes

    DWORD dwBytes = 0;

    //*** Writes bytes into file

    WriteFile(hDbfFile, &DbfHdr, sizeof(DbfHdr), &dwBytes, NULL);

    //*** Dinamic array of field length

    BYTE *acFldLen = new BYTE[nFldCount];

    //*** Dinamic array of field types

    BYTE *acFldType = new BYTE[nFldCount];

    UINT nOffset = 1;  // Skips delete byte


    //*** Writes array of DBF_FIELD aDbfField[FLDCOUNT] structures

    for(int i = 0; i < nFldCount; i++) {
        META_DATA MetaData = m_MetaTable.aMetaData[i];
        BYTE *acName = DbfField.acName;
        TCHAR *szFldName = MetaData.szFldName;

        //*** Field name with a maximum of 10 characters, a rest is padded 

        // with 0x00

        //for(int j = 0; j < 11; j++)

                //acName[j] = szFldName[j];

        
        //*** Simply copies

        while(*acName++ = *szFldName++);

        acFldType[i] = MetaData.szFldType[0];  // Field type

        DbfField.cType = acFldType[i];  // Field type

        DbfField.nOffset = nOffset;  // Displacement of field in record

        acFldLen[i] = MetaData.nFldLen;  // Length of field (in bytes)

        DbfField.cLen = acFldLen[i];  // Length of field (in bytes)

        DbfField.cDec = MetaData.nDecLen;  // Number of decimal places


        nOffset += acFldLen[i];

        //*** Writes bytes into file

        WriteFile(hDbfFile, &DbfField, sizeof(DbfField), &dwBytes, NULL);
    }

    //*** Length of one data record, including delete flag

    DbfHdr.wRecSize = nOffset;  // ARE NOT WRITTEN YET!


    //*** Header record terminator

    BYTE cHdrEnd = HEADEREND;  // = 0x0D (13)

    
    //*** Writes bytes into file

    WriteFile(hDbfFile, &cHdrEnd, sizeof(cHdrEnd), &dwBytes, NULL);

    //*** A data range for associated *.dbc file, contains 0x00

    BYTE acDbcFile[263] = {0};

    //*** Writes bytes into file

    WriteFile(hDbfFile, &acDbcFile, sizeof(acDbcFile), &dwBytes, NULL);

    //*** Gets current file pointer

    DWORD nCurOffset = SetFilePointer(hDbfFile, 0, NULL, FILE_CURRENT);

    //*** Position of first data record

    DbfHdr.wFirstRec = nCurOffset;  // ARE NOT WRITTEN YET!


    //*** The delete flag

    BYTE cDelete = 32;  // = 0x20 (" ")


    //*** Line table cell index

    UINT ji = 0;

    //*** Writes array of DBF_RECORD aDbfRec[RECCOUNT] structures

    for(ULONG j = 0; j < nRecCount; j++) {
        //*** Writes bytes into file

        WriteFile(hDbfFile, &cDelete, sizeof(cDelete), &dwBytes, NULL);

        //*** Writes array of BYTE's strings

        for(i = 0; i < nFldCount; i++) {
            ji = j*nFldCount + i;  // Line table cell index

            TCHAR *acRowText = m_MetaTable.apRowText[ji];

            BYTE cFldLen = acFldLen[i];
            BYTE cFldType = acFldType[i];

            //*** Dinamic array of field data

            BYTE *acFldData = new BYTE[cFldLen + 2];  // +2 for sake date format


            //*** Copies field data (into BYTEs from TCHARs)

            for(int k = 0; k < cFldLen; k++)
                    acFldData[k] = acRowText[k];

            //*** Formates our date string (DD.MM.YYYY) into dbf style (YYYYMMDD)

            if(cFldType == 68) {  // = 0x44 ("D") - Date

                if(cFldLen != 8) {
                    swprintf(
                            szStr, 
                            _T("CMainApp: Length of date format is %d. Must be 8!"), 
                            cFldLen
                    );

                    _M(szStr);
                    return FALSE;
                }

                if(cFldLen == 8) {  // Date length for dbf date format

                    //*** Our static date has 10 characters

                    acFldData[8] = acRowText[8];
                    acFldData[9] = acRowText[9];
                    
                    //*** Date format is d1d2.m1m2.y1y2y3y4 . Must be y1y2y3y4m1m2d1d2

                    acFldData[2] = acFldData[8];  // Writes y3

                    acFldData[5] = acFldData[4];  // Writes m2

                    acFldData[4] = acFldData[3];  // Writes m1

                    acFldData[3] = acFldData[9];  // Writes y4

                    acFldData[8] = acFldData[0];  // Saves d1

                    acFldData[9] = acFldData[1];  // Saves d2

                    acFldData[0] = acFldData[6];  // Writes y1

                    acFldData[1] = acFldData[7];  // Writes y2

                    acFldData[6] = acFldData[8];  // Writes d1

                    acFldData[7] = acFldData[9];  // Writes d2

                } 
                //*** Else do nothing

            }

            //*** Writes bytes into file

            WriteFile(hDbfFile, acFldData, cFldLen, &dwBytes, NULL);
        }
    }

    //*** File record terminator

    BYTE cFileEnd = DBFEND;  // = 0x1A (26)


    //*** Writes bytes into file

    WriteFile(hDbfFile, &cFileEnd, sizeof(cFileEnd), &dwBytes, NULL);

    //*** Calculates file pointer to DbfHdr.wFirstRec

    ULONG nPos = (ULONG) &DbfHdr.wFirstRec - (ULONG) &DbfHdr.cType;  // = 8

    WORD wFirstRec = DbfHdr.wFirstRec;
    WORD wRecSize = DbfHdr.wRecSize;

    //*** Sets file pointer in DbfHdr.wRecSize position

    SetFilePointer(hDbfFile, nPos, NULL, FILE_BEGIN);

    //*** Writes NOT WRITTEN YET bytes into file

    WriteFile(hDbfFile, &wFirstRec, sizeof(wFirstRec), &dwBytes, NULL);

    //*** Writes next NOT WRITTEN YET bytes into file

    WriteFile(hDbfFile, &wRecSize, sizeof(wRecSize), &dwBytes, NULL);

    ::CloseHandle(hDbfFile);
    
    return TRUE;
}  // CreateDocumentFile

5. Processing of data in the Virtual Mode

In conclusion we will show the code of LVN_GETDISPINFO handler, which is «responsible» for the virtual mode.

/////////////////////////////////////////////////////////////////////////////
// OnChildNotify

/////////////////////////////////////////////////////////////////////////////
BOOL CListCtrlEx::OnChildNotify(UINT message, WPARAM wParam, LPARAM lParam, LRESULT *pResult) {
    NMHDR *pNMHdr = reinterpret_cast<NMHDR *>(lParam);

    LV_DISPINFO *pLVDI = reinterpret_cast<LV_DISPINFO *>(lParam);
    LV_ITEM *pItem = &pLVDI->item;

    if(message == WM_NOTIFY) {
        switch(pNMHdr->code) {
            case LVN_GETDISPINFO: {
                if(pItem->mask & LVIF_TEXT) {
                    //*** Item row

                    UINT nRow = pItem->iItem;

                    //*** Item column

                    UINT nCol = pItem->iSubItem;

                    //*** The message buffer

                    TCHAR szStr[MAXITEMTEXT];

                    //*** Current document

                    m_pDoc = m_pMainApp->m_apDoc[m_eTable];

                    if(!m_pDoc) {
                        _M("CListCtrlEx::CListCtrlEx : Empty document!");
                        //*** Forces to exit from the application as else will be a lot messages

                        exit(-1);
                    }

                    //*** Number of fields in dbf file

                    ULONG nColCount = m_pDoc->m_nFldCount;  // = m_MetaTable.nColCount;


                    if(nColCount == 0) {
                        _M("CListCtrlEx::OnChildNotify : Column count = 0!");
                        //*** Forces to exit from the application as else will be a lot messages

                        exit(-1);
                    }

                    if(m_MetaTable.nColCount != nColCount) {
                        swprintf(
                                szStr, 
                                _T("CListCtrlEx::OnChildNotify : Table (%d) and Dbf (%d) columns are different!"), 
                                m_MetaTable.nColCount, 
                                nColCount
                        );

                        _M(szStr);
                        //*** Forces to exit from the application as else will be a lot messages

                        exit(-1);
                    }

                    //*** Number of records in dbf file

                    ULONG nRowCount = m_pDoc->m_nRecCount;

                    if(nRowCount == 0) {
                        _M("CListCtrlEx::OnChildNotify : Row count = 0!");
                        //*** Forces to exit from the application as else will be a lot messages

                        exit(-1);
                    }

                    //*** Sets row count into meta table

                    //if(m_MetaTable.nRowCount != nRowCount)

                    m_MetaTable.nRowCount = nRowCount;
                    
                    //*** Length of one data record (including delete flag)

                    ULONG nRowSize = m_pDoc->m_nRecSize;

                    if(nRowSize == 0) {
                        _M("CListCtrlEx::OnChildNotify : Row size = 0!");
                        //*** Forces to exit from the application as else will be a lot messages

                        exit(-1);
                    }

                    //*** Line displacement of nCol-th field of nRow-th record

                    ULONG nLineInd = nRow*nRowSize + m_pDoc->m_anOff[nCol];

                    if(nLineInd < 0) {
                        _M("CListCtrlEx::OnChildNotify : Line index < 0!");
                        //*** Forces to exit from the application as else will be a lot messages

                        exit(-1);
                    }

                    //*** Maps view of dbf file for its second part (where are data)

                    DBF_FILE2 *m_pDbfView2 = m_pDoc->m_pDbfView2;

                    if(!m_pDbfView2) {
                        _M("CListCtrlEx::OnChildNotify : Empty view of dbf file!");
                        //*** Forces to exit from the application as else will be a lot messages

                        exit(-1);
                    }

                    //*** The Visual FoxPro records

                    BYTE *aDbfRec = m_pDbfView2->aDbfRec;

                    if(!aDbfRec) {
                        _M("CListCtrlEx::OnChildNotify : Empty array of dbf file!");
                        //*** Forces to exit from the application as else will be a lot messages

                        exit(-1);
                    }

                    if(!aDbfRec) {
                        _M("CListCtrlEx::OnChildNotify : Empty records into dbf file!");
                        //*** Forces to exit from the application as else will be a lot messages

                        exit(-1);
                    }

                    //*** Line table cell index

                    UINT ji = nRow*nColCount + nCol;

                    if(ji < 0) {
                        _M("CListCtrlEx::OnChildNotify : Line cell index < 0!");
                        //*** Forces to exit from the application as else will be a lot messages

                        exit(-1);
                    }

                    if(ji < nRowCount*nColCount) {
                        try {
                            //*** Copies (j, i) field value of m_anLen[i]-th length

                            // We do this as it hasn't null terminator

                            CString sFldVal(
                                    (LPCSTR) &aDbfRec[nLineInd], 
                                    m_pDoc->m_acLen[nCol]
                            );

                            //*** Formates date string into German style

                            if(m_pDoc->m_acFldType[nCol] == 68) {  // = 0x44 ("D") - Date

                                CString sDate = 
                                        sFldVal.Right(2) + _T(".") + 
                                        sFldVal.Mid(4, 2) + _T(".") + 
                                        sFldVal.Left(4);
                                
                                sFldVal = sDate;
                            }

                            wcscpy((wchar_t *)(pItem->pszText), sFldVal);
                        } catch (...) {
                            //*** Forces to exit from the application as else will be a lot messages

                            exit(-1);
                        }
                    } else {
                        wcscpy((wchar_t *)(pItem->pszText), _T("***"));
                    }
                }

                break;
            }
        }
    }

    return CListCtrl::OnChildNotify(message, wParam, lParam, pResult);
}  // OnChildNotify

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Emery Emerald
Software Developer
Ukraine Ukraine
No Biography provided

You may also be interested in...

Pro
Pro

Comments and Discussions

 
GeneralVery good and interesting article. Pin
reinaldohf22-Jul-09 15:08
memberreinaldohf22-Jul-09 15:08 
GeneralRe: Very good and interesting article. Pin
Emery Emerald22-Jul-09 21:20
memberEmery Emerald22-Jul-09 21:20 
GeneralRe: Very good and interesting article. Pin
reinaldohf23-Jul-09 12:19
memberreinaldohf23-Jul-09 12:19 
Hi,

The Open Source Project xHarbour
www.xharbour.org
www.xharbour.com
www.nabble.com

Developers:
http://n2.nabble.com/harbour-devel-f1590103.html<a href="http://n2.nabble.com/harbour-devel-f1590103.html" target="_blank" title="New Window"


Here is the link for the place where the sources are.

http://sourceforge.net/projects/xharbour/files/Source%20distribution/1.20.01/xharbour-1.2.1.src.zip/download
GeneralRe: Very good and interesting article. Pin
Emery Emerald23-Jul-09 21:15
memberEmery Emerald23-Jul-09 21:15 
GeneralRe: Very good and interesting article. Pin
reinaldohf23-Jul-09 22:33
memberreinaldohf23-Jul-09 22:33 
GeneralRe: Very good and interesting article. Pin
Emery Emerald24-Jul-09 2:28
memberEmery Emerald24-Jul-09 2:28 
GeneralRe: Very good and interesting article. Pin
reinaldohf24-Jul-09 23:11
memberreinaldohf24-Jul-09 23:11 
GeneralRe: Very good and interesting article. Pin
Emery Emerald25-Jul-09 4:04
memberEmery Emerald25-Jul-09 4:04 
GeneralRe: Very good and interesting article. Pin
reinaldohf26-Jul-09 13:09
memberreinaldohf26-Jul-09 13:09 
GeneralRe: Very good and interesting article. Pin
Emery Emerald26-Jul-09 20:09
memberEmery Emerald26-Jul-09 20:09 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Terms of Use | Mobile
Web01 | 2.8.170424.1 | Last Updated 22 Jul 2009
Article Copyright 2009 by Emery Emerald
Everything else Copyright © CodeProject, 1999-2017
Layout: fixed | fluid