A serialization primer - Part 1

This article is the first of a 3 part tutorial on serialization.

Part 1 introduces the basics of serialization.
Part 2 explains how to gracefully handle reading invalid data stores and support versioning.
Part 3 describes how to serialize complex objects.

Serialization is the process of writing or reading an object to or from a persistent storage medium, such as a disk file. Serializing an object requires 3 ingredients:

A CFile object representing the datafile
A CArchive object that provides the serialization context
The object being serialized

Step 1 - Open the datafile

To serialize an object to the file "foo.dat", open the file with the appropriate access mode. In this example, the file is opened for exclusive read/write access.

  // Open file "foo.dat"
  CFile* pFile = new CFile();
  ASSERT (pFile != NULL);
  if (!pFile->Open ("foo.dat", CFile::modeReadWrite | CFile::shareExclusive)) {
      // Handle error
      return;
  }

Step 2 - Hook up the archive

Next, a CArchive object is hooked up to the file. The archive provides an efficient conduit to persistent storage. Instead of directly reading and writing the file, you serialize data to and from the archive. The archive needs to know if you're going to be using it to read or write data. In this example, we'll assume we're writing data.

  // Create archive ...
  bool bReading = false;  // ... for writing
  CArchive* pArchive = NULL;
  try
  {
    pFile->SeekToBegin();
    UINT uMode = (bReading ? CArchive::load : CArchive::store);
    pArchive = new CArchive (pFile, uMode);
    ASSERT (pArchive != NULL);
  }
  catch (CException* pException)
  {
    // Handle error
    return;
  }

Step 3 - Serialize the object

Finally, we serialize the object by calling its serialize() method. serialize() is just a method we made up. It has nothing to with MFC's CObject::Serialize(). Also, you don't have to derive your object from CObject. Our serialize() method takes a pointer to a CArchive and returns an integer status.

  int CFoo::serialize
    (CArchive* pArchive)
  {
    int nStatus = SUCCESS;

    // Serialize the object ...
    ...
    
    return (nStatus);
  }

We'll get to the actual serialization process in a minute. Meanwhile, let's recognize a couple of important points:

The same method CFoo::serialize() is used to read/write the object from/to persistent storage.
CFoo::serialize() doesn't know anything about the datafile bring accessed.

Assume CFoo represents an employee record that contains a couple of data members.

  class CFoo
  {
    // Construction/destruction
    public:
      CFoo::CFoo();
      virtual CFoo::~CFoo();

    // Methods
    public:
      int serialize (CArchive* pArchive);

    // Data members
    public:
      CString  m_strName;  // employee name
      int      m_nId;      // employee id
  };

We use CArchive's streaming operators << and >> to read/write the data members from/to the archive. CArchive knows how to serialize simple data types like int, float, DWORD, and objects like CString. The archive also knows whether it's in read or write mode. You can query its mode by calling CArchive::IsStoring(). CFoo's serialization method can then be written as:

  int CFoo::serialize
    (CArchive* pArchive)
  {
    int nStatus = SUCCESS;

    // Serialize the object ...
    ASSERT (pArchive != NULL);
    try
    {
      if (pArchive->IsStoring()) {
         // Write employee name and id
         (*pArchive) << m_strName;
         (*pArchive) << m_nId;
      }
      else {
         // Read employee name and id
         (*pArchive) >> m_strName;
         (*pArchive) >> m_nId;
      }
    }
    catch (CException* pException)
    {
      nStatus = ERROR;
    }
    return (nStatus);
  }

Step 4 - Clean up

When you've finished serializing, you should clean up by closing the archive and datafile.

  pArchive->Close();
  delete pArchive;
  pFile->Close();
  delete pFile();

Conclusion

Well, there you have it - serialization in a (very small) nutshell. In Part 2, we'll see how to gracefully handle reading invalid data stores and support different versions of our object. In Part 3, we'll see how to serialize complex objects.

You must Sign In to use this message board.


Per page

Changing the format of the data in the file
AORD

13:26 29 Sep '07

What is the best way of making the data in the file unreadable? Currently I can see all the stings and integers when I open the file in notepad. Can the data be saved so the file is not readable by somebody opening with notepad etc. Cheers, AORD ___________________________ Here come the machines!
Sign In·View Thread·PermaLink

Re: Changing the format of the data in the file
Ravi Bhavnani

18:51 9 Oct '07

[ Sorry for the delay in replying. For some reason, I no longer receive CP emails when someone posts a note in my article forums. ]
A simple way to deter casual viewers is to invert the bits of the characters in a string before/after serialization. For example:

void InvertString
  (CString& strString)  // string to be inverted
{
  for (int nIndex=0; (nIndex < strString.GetLength()); nIndex++) {
      char ch = strString.GetAt (nIndex);
      ch = ~ch;
      strString.SetAt (nIndex, ch);
  }
}

Hope this helps!

/ravi

This is your brain on Celcius
Home | Music | Articles | Freeware | Trips ravib(at)ravib(dot)com

About the contents of the file
cristitomi

23:24 12 Feb '07

Hi! I am creating an object of type CFoo with some values for the data types, for example m_strName = "someName" m_nID = 98 and I am using the CArchive in store mode. But when I look at the file it shows me something like: someNameb instead of the right data. Does anyone know why this happens ? Thanks. cristitomi
Sign In·View Thread·PermaLink

Re: About the contents of the file
Ravi Bhavnani

2:34 13 Feb '07

cristitomi wrote:
Does anyone know why this happens ?

The file contains a binary representation of the data. If you'd like to see "98" in the file, you'd need to store the string version of m_nId.

Also, data stored for a CString will be preceded by its length (stored in binary form), which is required for the CArchive to know when to stop reading the CString.

/ravi

This is your brain on Celcius
Home | Music | Articles | Freeware | Trips ravib(at)ravib(dot)com

Re: About the contents of the file
cristitomi

3:22 13 Feb '07

Hi! Thanks for your helpful comments. I can tell you this is a great article, very helpful to me indeed. Thanks again.
Sign In·View Thread·PermaLink

Re: About the contents of the file
Ravi Bhavnani

4:25 13 Feb '07

Glad I could be of help, Cristi! /ravi This is your brain on Celcius Home \| Music \| Articles \| Freeware \| Trips ravib(at)ravib(dot)com
Sign In·View Thread·PermaLink

trouble in reading in serialized list structures
itsh11

13:01 15 Dec '04

I am trying to read in a list structure which is serialized as

m_DataPointerList.Serialize( ar );

where m_DataPointerList is defined as :

CTypedPtrList m_DataPointerList;

and class CDataPointer is declared as:

class CDataPointer : public CObject
{
public:
CDataPointer();

protected:
DECLARE_SERIAL(CDataPointer)

protected:

public:
unsigned long Time; unsigned char Channel;
unsigned char Rate;

public:
virtual void Serialize(CArchive& ar);
};

and defined as:

IMPLEMENT_SERIAL(CDataPointer, CObject, 1)
CDataPointer::CDataPointer()
{

}

void CDataPointer::Serialize(CArchive& ar)
{
if (ar.IsStoring())
{
ar << Time;
ar << Channel;
ar << Rate;
}
else
{
ar >> Time;
ar >> Channel;
ar >> Rate;
}
}

I am having some difficulties( i donno why?) in reading it in using the same serializing command ie:

m_DataPointerList.Serialize( ar );

is there any thing i am overlooking while i am doing this??

on the other hand,
I try to read this using a 'for' loop
which iterates for the number of occurances of CDataPointer. but i have to add an offset at the end of every iteration if I were to read the next iteration correctly.
something like

nDataCounter=ar.ReadCount();
for(i=0;i{
ar>>Time[i];
ar>>Channel[i];
ar>>Rate[i];

ar.Flush();
pFile->Seek(offSET, CFile::current);
}

i dont understand why i have to provide this offset.and i have trouble choosing the correct offSET as it varies for different lists. again is there something i am doing wrong ?

Can please you give me some insight into these two issues??

Thanks a lot
Hemanth

Re: trouble in reading in serialized list structures(typo corrected)
itsh11

5:11 16 Dec '04

m_DataPointerList has to be defined as CTypedPtrList < CObList,CDataPointer* > m_DataPointerList; and not as wrongly typed by me
Sign In·View Thread·PermaLink

Reading a file using serialization
hemanth_phk

12:55 9 Sep '04

I am trying to read in a file, which was written using serialization (by someone else). The size of the variables present in the file from which I need to read is way greater than the stack size. So this is giving me problems like stack overflow. Is there any way to solve such problems? Please give me your suggestions .Thanks a lot
Sign In·View Thread·PermaLink	3.67/5

Re: Reading a file using serialization
Ravi Bhavnani

13:28 9 Sep '04

See my reply in Part 3. No need to cross-post. /ravi My new year's resolution: 2048 x 1536 Home \| Articles \| Freeware \| Music ravib@ravib.com
Sign In·View Thread·PermaLink	1.33/5

Re: Reading a file using serialization
Thomas Mielke

15:58 11 Oct '07

Have just ran into it as well. Large data amounts are usually stored in pointer lists, but these are being processed recursively by CArchive resulting in extreme stack consumtion. Two workarounds that are not fully satisfying: disintegrate the list (by zeroing the 'next' pointer) and store each element with a distinct '<<' operation, or increase the default maximum stack size of 1 MB by using a linker option (for example /STACK:0xa00000 for 10 MB).

CArchive and .NET
Daniel S. Horwitz

7:37 12 Jul '04

Dear Folks, Is there a class in .NET that can read documents written using the MFC CArchive class? The problem is that I am porting an existing MFC C++ application to .NET. It uses CArchive to persist and restore its documents. I need the ported version of the code to be able to read the existing documents that were persisted by CArchive. Thanks for your help, Daniel S. Horwitz
Sign In·View Thread·PermaLink	2.00/5

In memory serialization
no_reg_name

3:59 7 May '04

Can you do this? Serialize an object and send it accross the wire, without actually storing it to the disk?
Sign In·View Thread·PermaLink

Re: In memory serialization
Ravi Bhavnani

7:02 7 May '04

I believe this can be done by using `CSocketFile` instead of `CFile`. /ravi My new year's resolution: 2048 x 1536 Home \| Articles \| Freeware \| Music ravib@ravib.com
Sign In·View Thread·PermaLink

Serialization using Dialog application
HybridLlama

17:57 12 Jan '04

How do i serialize data collected from a dialog using one button to save and one button to load.
Sign In·View Thread·PermaLink

Re: Serialization using Dialog application
Ravi Bhavnani

7:37 16 Jan '04

Store the data in an object, then serialize the object. /ravi My new year resolution: 2048 x 1536 Home \| Articles \| Freeware \| Music ravib@ravib.com
Sign In·View Thread·PermaLink

Re: Serialization using Dialog application
HybridLlama

10:56 16 Jan '04

Thanks, I got the serialization to work but I am having a problem loading the data. It is giving me an error something to the effect that an attempt was made to acessan unamed file past its end. How can I fix this?
Sign In·View Thread·PermaLink	2.00/5

Re: Serialization using Dialog application
Ravi Bhavnani

4:32 17 Jan '04

It's hard to say without looking at the code. Perhaps you could post a snippet? It's possible that you may not have opened the file or hooked up the archive correctly. /ravi My new year resolution: 2048 x 1536 Home \| Articles \| Freeware \| Music ravib@ravib.com
Sign In·View Thread·PermaLink

Re: Serialization using Dialog application
HybridLlama

9:28 17 Jan '04

Here is a small segment of the storeing code and the loading code:
Storing:

CFile file;

file.Open(fileName, CFile::modeCreate|CFile::modeWrite);

CArchive ar(&file, CArchive::store);

Serialize(ar);

ar.Close();
file.Close();

Loading:

CFile file;

file.Open(filename,CFile::modeRead);

CArchive ar(&file, CArchive::load);

Serialize(ar);

The program would load the information just fine until I added 5 strings to be stored, after that it went down hill. I hope the above is what you wanted.

Ditching.NET serialization in favor of MFC
MtnBiknGuy

17:49 28 Aug '03

Can anyone point me in the right direction for getting started using MFC serialization in place of .NET serialization?

We need to save large amounts (up to gigabytes) of complex objects and data to disk files. Most of the code is managed C#, but .NET serialization is way too slow. The processing time increases from 1 hr to over 15 hrs when we use .NET binary serialization (instead of just writing the data to ASCII files -- and the ASCII code even uses some reflection and other slow stuff).

Our desire is to keep most of our managed C# code, but substitute MFC serialization for .NET serialization in whatever way is required.

Any pointers or tips are apprecited.

Re: Ditching.NET serialization in favor of MFC
Ravi Bhavnani

5:33 29 Aug '03

MtnBiknGuy wrote: Any pointers or tips are apprecited Aren't the articles sufficient? /ravi Let's put "civil" back in "civilization" Home \| Articles \| Freeware \| Music ravib@ravib.com
Sign In·View Thread·PermaLink

Re: Ditching.NET serialization in favor of MFC
Gordon Brandly

7:48 31 Aug '05

Though I've been thinking of switching to .NET for our new data logging programs, this was something I've been worried about and a major reason I haven't switched yet. I'm glad to see that my fears are reasonable for once. Big Grin

Still, I do like the productivity of .NET, and I think my switchover is coming soon. When I do switch, I'll be using custom binary file formats like I do now. It's like working in assembly language: more work than using the built-in serialization libraries, but you can (potentially) get much faster performance.

On the other hand, I do less and less assembly language programming as time goes on, and I suspect that the .NET serialization support will get better in future versions.

Problem reading file
Count Zer0

10:19 21 Jul '03

I am trying to pull a bunch of CStrings out of a text file but I am not using an Object array
this is how I am do it

do
{
if( ar.IsLoading() )
{
ar >> temp;
}

if(temp.GetLength() > 1)
{
allnames[x] = temp;
x++;
}

}while(!temp.IsEmpty());

the problem is with ending the whole process I read all the CStrings but then it blows up when I read the next CString that is not there. How do I detect the end of file so that this will not happen.

thanks

Re: Problem reading file
Anonymous

6:43 16 Jan '04

When writing the strings, store the number of strings written first. James
Sign In·View Thread·PermaLink	2.00/5

library for XML serialization in C++ ?
Anonymous

2:15 17 Mar '03

Hello, Thanks to this article, I learnt a lot about serialization. My goal is to serialize in a readable format. XML would be ok. Moreover, I don't want to write everything from scratch, I prefer to use existing code. Do you know any good (commercial) library to implement XML serialization in C++ ? (I don't want to use managed C++). Regards, francois.
Sign In·View Thread·PermaLink	3.50/5