Dynamic string buffer allocation using IStream

• Download source files - 4.05 Kb

Introduction

Often, we find ourselves having to obtain sub-strings from various sources and accumulate them into a central string. For example, we may need to find out the values of various property settings in our program, store their values (in string form) together with some comments and finally display the entire report in a file or an Edit Box. A very familiar problem that we face in such situations is to estimate the sufficient amount of memory space to set aside for the central string buffer.

Often we over-allocate the amount of buffer space just so that we do not overrun memory. Another motivation for doing this is to allow for future maintenance where we anticipate more strings will need to be added. A good way to overcome this problem is to use good string manipulation classes like MFC's CString or its equivalents from other vendors. The CString, in particular, allows us to concatenate sub-strings without us having to worry about buffer space and its allocation/re-allocation.

However, not every project team is allowed to use MFC. The project that my team is working on, for instance, is strictly prohibited from using MFC due to legacy reasons and other historical issues. One time, one of my team members had to debug such a string concatenation problem that resulted in a GPF. The reason for the crash is none other than memory overrun.

Now, we had two ways to handle this problem :

• Pre-allocate more local stack memory for the central buffer.
• Resort to dynamic memory allocation with all the extra overheads of checking for sufficient left over-space and performing re-allocation where necessary.

Both alternatives were unattractive in their own ways. Then one of our guys read a very good article by Arnaud Aubert in the August 2001 issue of Windows Developer's Journal (in the Tech Tips section). Arnaud basically advocates the use of the OLE/COM IStream interface that can dramatically simplify memory management code. Using IStream, we can read/write data to a stream object backed by memory.

The wonderful thing about using the OLE/COM IStream APIs is that you need not worry about allocating the memory required for storing your data in the IStream object. OLE will manage this for us in the background. My source sample in this article uses this same basic principle to demonstrate how we can perform natural sub-string accumulation without worrying about the overheads of size and allocation management.

Overview

The sample source that I present in this article contains 4 main files :

1. main.h
2. main.cpp
3. memorystream.h
4. memorystream.cpp

main.h - Just a header file for declaring the functions defined in main.cpp.

main.cpp - Main running module. This file contains two main functions MemoryStringTest_Macro() and MemoryStringTest_Function() which demonstrate how to use IStream to concatenate sub-strings.

memorystream.h - Header file that contains several useful macro definitions. The macros here are used and tested in the function MemoryStringTest_Macro().

memorystream.cpp - Module that contains functions that work with an IStream pointer to create and manipulate a virtual string buffer.

Throughout, the direction I have taken is to have the notion of a string buffer in memory that we can keep concatenating without having to worry about its size. When we have finished concatenating this string with all our sub-strings, we make a copy of it and use it as we wish. You can use either the macros or the functions.

Detailed Analysis of the Macros

MEMORY_STRING_DECL

#define MEMORY_STRING_DECL\
    LPSTREAM lpStream = NULL;

This macro declares a pointer to an IStream. Other macros will assume the presence of this pointer.

MEMORY_STRING_CREATE

#define MEMORY_STRING_CREATE\
    CreateStreamOnHGlobal(NULL, TRUE, &lpStream);

This macro calls on the CreateStreamOnHGlobal() API to create a MEMORY stream object. Note that the first parameter (type HGLOBAL) is NULL which indicates to the CreateStreamOnHGlobal function to internally allocate a new shared memory block of size zero. This allocated memory block will grow automatically as we store more and more data into it. This is done in the background by the underlying OLE/COM engine. The second parameter is TRUE which indicates that the underlying handle for this stream object should be automatically freed when the stream object is released (i.e. when we call IStream::Release()).

MEMORY_STRING_RESET

#define MEMORY_STRING_RESET {\
    LARGE_INTEGER liBeggining = { 0 };\
    lpStream->Seek(liBeggining, STREAM_SEEK_SET, NULL);\
}

This macro will reset the internal pointer of the stream to the beginning of the stream. An IStream object is very much like a file. It has an internal pointer that points to BYTE locations offset from the beginning of the stream. You will notice that this macro is called in some other macros. I will explain its use in those macros when we cover those macros in detail.

MEMORY_STRINGCOPY

#define MEMORY_STRINGCOPY(lpString, lpbError) {\
    ULONG ulBytesWritten = 0;\
    ULONG ulSize = 0;\
    ULARGE_INTEGER uliSize = { 0 };\
\
    MEMORY_STRING_RESET\
\
    lpStream -> SetSize (uliSize);\
\
    ulSize = (ULONG)strlen(lpString);\
    lpStream->Write((void const*)lpString,\
       (ULONG)ulSize, (ULONG*)&ulBytesWritten);\
\
    if (lpbError)\
    {\
        if (ulSize == ulBytesWritten)\
        {\
            *((BOOL*)lpbError) = FALSE;\
        }\
        else\
        {\
            *((BOOL*)lpbError) = TRUE;\
        }\
    }\
}

This macro encapsulates the operation of a string copy from a source string (the first parameter) to the virtual string inside the IStream object. Notice that we first call MEMORY_STRING_RESET. When we copy a source string into a destination string, we would want the destination to start at the beginning. Data is stored into an IStream object via its Write() method. This method writes data at wherever the stream pointer points. This is why we specifically reset this pointer to the beginning of the stream. We next set the size of the stream to zero. This effectively shrinks the IStream object to size zero and causes all existing data in the stream object to be discarded away.

This last fact is important to maintain the notion that we are working with a string. Whenever we do a strcpy() into a string which has previous value, we accept the fact that this previous string value will be overwritten. We then calculate the character size of the source string (excluding terminating NULL character) and then copy the string into our IStream object. We do not copy the terminating NULL character because we can determine the length of the string via statistics information about the IStream object. More on this later.

We will indicate an error if the number of characters copied does not equal the number of characters in the source string. To keep this sample source simple, I have assumed that we will be working only with ANSI single-byte character set strings and not UNICODE nor double-byte character sets. Hence, my use of strlen().

MEMORY_STRINGCAT

#define MEMORY_STRINGCAT(lpString, lpbError) {\
    ULONG ulBytesWritten = 0;\
    ULONG ulSize = 0;\
\
    ulSize = strlen(lpString);\
    lpStream->Write((void const*)lpString, 
        (ULONG)ulSize, (ULONG*)&ulBytesWritten);\
\
    if (lpbError)\
    {\
        if (ulSize == ulBytesWritten)\
        {\
            *((BOOL*)lpbError) = FALSE;\
        }\
        else\
        {\
            *((BOOL*)lpbError) = TRUE;\
        }\
    }\
}

This macro will perform an equivalent of the usual strcat(). It is very similar to MEMORY_STRINGCOPY except that we do not reset the stream pointer and we do not call IStream::SetSize(). We let the size remain the same and simply write the new string into the existing one in the stream right after the end. Because there are no terminating NULL characters in the existing string, we effectively perform a concat. This is why we do not record any terminating NULL character in the stream. The concatenation operation is simplified.

MEMORY_STRING_READ

#define MEMORY_STRING_READ(lpReceiver, lpulSizeReceiver)\
{\
    STATSTG statstg;\
\
    MEMORY_STRING_RESET\
\
    memset (&statstg, 0, sizeof(statstg));\
    lpStream->Stat(&statstg, STATFLAG_NONAME);\
\
    if ((ULONG*)lpulSizeReceiver)\
    {\
        *((ULONG*)lpulSizeReceiver) = statstg.cbSize.LowPart;\
    }\
\
    if ((void*)lpReceiver)\
    {\
        memset ((void*)lpReceiver, 0,\
            statstg.cbSize.LowPart + sizeof(char));\
        lpStream->Read((void*)lpReceiver, \
            statstg.cbSize.LowPart, NULL);\
    }\
}

This macro reads the string in the stream and copies it into a user supplied buffer. This macro will store the required character size of the string into a receiving pointer to an unsigned long. Because you can use NULL pointers in both parameters, you can call MEMORY_STRING_READ with the first parameter NULL and a valid ULONG pointer in the second and this way, we can determine the required size of the receiving buffer first before calling MEMORY_STRING_READ again with a fully allocated buffer pointer.

An example of this can be seen in the function MemoryStringTest_Macro() :

MEMORY_STRING_READ (NULL, &ulSizeRequired)
pszString = (LPTSTR)malloc(ulSizeRequired + sizeof(char));
if (pszString)
{
    MEMORY_STRING_READ(pszString, NULL)
    free (pszString);
    pszString = NULL;
}

MEMORY_STRING_DESTROY

#define MEMORY_STRING_DESTROY\
    if (lpStream)\
    {\
        lpStream->Release();\
        lpStream = NULL;\
    }

The final macro simply releases our interface pointer to IStream and resets lpStream to NULL for safety.

Detailed Analysis of the Functions

StreamResetPointer()

HRESULT StreamResetPointer (LPSTREAM lpStream)
{
    LARGE_INTEGER liBeggining = { 0 };
    HRESULT hrRet = S_OK;

    hrRet = lpStream->Seek(liBeggining, STREAM_SEEK_SET, NULL);

    return hrRet;
}

This function is very similar in nature to the MEMORY_STRING_RESET macro. The way it works should be self-explanatory. See MEMORY_STRING_RESET for more details.

StreamStringCopy()

HRESULT StreamStringCopy (LPSTREAM lpStream, 
                            LPCTSTR lpString)
{
    ULONG ulBytesWritten = 0;
    ULONG ulSize = 0;
    ULARGE_INTEGER uliSize = { 0 };
    HRESULT hrRetTemp = S_OK;
    HRESULT hrRet = S_OK;

    hrRetTemp = StreamResetPointer ((LPSTREAM)lpStream);
    if (hrRetTemp != S_OK)
    {
        hrRet = hrRetTemp;
        goto StreamStringCopy_0;
    }

    hrRetTemp = lpStream->SetSize (uliSize);
    if (hrRetTemp != S_OK)
    {
        hrRetTemp = hrRet;
        goto StreamStringCopy_0;
    }

    ulSize = (ULONG)strlen(lpString);

    hrRetTemp = lpStream->Write((void const*)lpString, 
        (ULONG)ulSize, (ULONG*)&ulBytesWritten);
    if (hrRetTemp != S_OK)
    {
        hrRet = hrRetTemp;
        goto StreamStringCopy_0;
    }

    if (ulSize == ulBytesWritten)
    {
        hrRet = S_OK;
    }
    else
    {
        hrRet = S_FALSE;
    }

StreamStringCopy_0:

    return hrRet;
}

This function is the exact counterpart of the MEMORY_STRINGCOPY macro. Observe that StreamResetPointer() is also called at the beginning and that IStream::SetrSize() is also called.

StreamStringCat()

HRESULT StreamStringCat (LPSTREAM lpStream, 
                            LPCTSTR lpString)
{
    ULONG ulBytesWritten = 0;
    ULONG ulSize = 0;
    HRESULT hrRetTemp = S_OK;
    HRESULT hrRet = S_OK;

    ulSize = strlen(lpString);

    hrRetTemp = lpStream->Write((void const*)lpString, 
        (ULONG)ulSize, (ULONG*)&ulBytesWritten);
    if (hrRetTemp != S_OK)
    {
        hrRet = hrRetTemp;
        goto StreamStringCat_0;
    }

    if (ulSize == ulBytesWritten)
    {
        hrRet = S_OK;
    }
    else
    {
        hrRet = S_FALSE;
    }

StreamStringCat_0:

    return hrRet;
}

StreamStringCat() is the counter part of the MEMORY_STRINGCAT macro. Observe also that with the functions, a HRESULT is returned.

StreamStringRead()

HRESULT StreamStringRead (LPSTREAM lpStream, 
    LPTSTR lpszReceiver, ULONG* lpulSizeReceiver)
{
    STATSTG statstg;
    HRESULT hrRetTemp = S_OK;
    HRESULT hrRet = S_OK;

    hrRetTemp = StreamResetPointer ((LPSTREAM)lpStream);
    if (hrRetTemp != S_OK)
    {
        hrRet = hrRetTemp;
        goto StreamStringRead_0;
    }

    memset (&statstg, 0, sizeof(statstg));
    hrRetTemp = lpStream->Stat(&statstg, STATFLAG_NONAME);
    if (hrRetTemp != S_OK)
    {
        hrRet = hrRetTemp;
        goto StreamStringRead_0;
    }

    if ((ULONG*)lpulSizeReceiver)
    {
        *((ULONG*)lpulSizeReceiver) = statstg.cbSize.LowPart;
    }

    if ((void*)lpszReceiver)
    {
        memset ((void*)lpszReceiver, 0, 
            statstg.cbSize.LowPart + sizeof(char));
        lpStream->Read((void*)lpszReceiver, 
            statstg.cbSize.LowPart, NULL);
    }

StreamStringRead_0:

    return hrRet;
}

StreamStringRead() is no doubt the counterpart of the MEMORY_STRING_READ macro. Notice that in both StreamStringRead() and MEMORY_STRING_READ the size of the input string is assumed to be as large as the character length of the IStream string plus space for one more terminating NULL character.

Conclusion

Refer to MemoryStringTest_Macro() and MemoryStringTest_Function() for examples on the usage of the macros and the functions supplied in the sample source. I have tried to keep the code as simple as possible so that the reader is able to expand on the concept and perhaps create a C++ class for it. Best of luck.