Click here to Skip to main content
Click here to Skip to main content

STL Serialization Library (STL-SL)

, 9 Oct 2006 CPOL
Rate this:
Please Sign up or sign in to vote.
The STL Serialization Library can serialize and load STL objects from a file. The serialization file format can be customised to suit your needs. It's fast, easy, and free!

A Quick User Guide

Using the STL Serialization Library (STL-SL) is a three-step process. In the first step, you declare the STL type you want to serialize using STL-SL.

value_trait < int > int_filer;
value_trait < std::list< bool > > bool_list_filer;
value_trait < std::map<std::string, std::multimap< float, bool > > > complex_stl_filer;

In the code snippet, three different types are declared. Each type has a varying degree of complexity. int_filer is the simplest of all, where an int is parameterized in the value_trait template class. int_filer can therefore be used to serialize and load an int from a serialization file.

Similarly, the declaration of bool_list_filer prepares it for serializing a list of bool types. And complex_stl_filer prepares a map of string and multimap data types where the multimap holds float, bool pairs as shown above.

In the next step, a file_interface is created that requires a serialization file name where the serialization data will be written-to or read-from.

stl_trait_writer file_interface (serialization_filename);

Note that file_interface does not require the type during declaration, meaning STL objects of different types may be serialized in the same serialization file.

Also, since the serialization file-interface declaration is not related to the data-file declaration, the above two steps may take each other's place.

In the third and the last stage, data is serialized into the file or loaded from it.

filer.serialize ( stl_object, file_interface );

filer.load ( stl_object, file_interface );

The following snippet illustrates the use of all the three steps described above:

//declare serialization-file interfaces
stl_trait_writer file_interface ( ".\\serialization-file.txt");

//declare the STL object 
std::vector < int > stl_object;


//declare the data filer 
value_trait < std::vector < int > > data_filer;


//...populate data in stl_object...// 
//serialize data in stl_object in the file pointed by file_interface
data_filer.serialize ( stl_object, file_interface );

//-- or --//

//load data in the file pointed by file_interface in stl_object STL object
data_filer.load ( stl_object, file_interface );
//...use data in stl_object...//

STL Serialization Library

This article presents a set of template classes capable of serializing STL objects in the user defined file format (by default, a text file).

The STL Serialization Library is composed of two major parts: Serialization Filer and Serialization Template Classes.

Serialization Filer

This consists of the following three classes:

//Serialization-file writer class.
//This guy writes the data to the file specified by 'file_path'.
//NOTE: This class does not recognize the data objects containing spaces, tabs, 
// new-line characters in them. This may be fixed by overloading '<<' operator and 
// adding escape-sequencing logic in it.
class stl_trait_writer: public std::ofstream
{
public:
      stl_trait_writer(const std::string& file_path):std::ofstream(file_path.c_str())
      {}
};

//Serialization-file reader class.
//This guy reads the data from the file specified by 'file_path'.
//NOTE: This class does not recognize the data objects containing spaces, tabs, 
// new-line characters in them. This may be fixed by overloading '>>' operator and 
// adding escape-sequencing logic in it.
class file_trait_reader: public std::ifstream
{
public:
      file_trait_reader(const std::string& file_path):std::ifstream(file_path.c_str())
      {}
};

//Serialization filer class.
//This guy presents the set of reader, writer objects responsible for reading and 
// writing to the serialization file.
template <class writer_trait , class reader_trait>
class filer_trait
{
public:
      typedef typename writer_trait writer_type;
      typedef typename reader_trait reader_type;
};

The stl_trait_writer and file_trait_reader classes provide the basic file I/O mechanism. The filer_trait class is a filer that only pairs the above two class types. An alternate approach may be where the filer_trait class implements the file I/O mechanism for simplicity at the cost of scalability.

The file writer class (stl_trait_writer) may be replaced with a user-class to change the serialization file format to, say for example, XML. A file reader class provided for loading the serialization data from a file (file_trait_writer) will have to be altered to a class capable of understanding the user file format.

Serialization Template Classes

The set of classes in this module comprises of the language mechanism for breaking down complex STL types into basic types and serializing/de-serializing the basic data types.

//Basic datatype serializer class.
//Triggers the read or write to the serialization file for the basic datatypes.
//NOTE: This class has been tweaked to work with the 'stl_trait_writer' class.
template <class val_trait, class val_filer_trait = 
              filer_trait<stl_trait_writer, file_trait_reader> >
class value_trait
{
public:
      typedef typename val_filer_trait::writer_type writer_trait;
      typedef typename val_filer_trait::reader_type reader_trait;

      void serialize(const val_trait& val, writer_trait &pen)
      {
            pen << val << "\n"; //a tweak for 'stl_trait_writer' class defined above.
            //pen << val; //correct code, this should replace above line of code should
                          //you choose to implement your own 'stl_trait_writer' class.

            pen.flush();
      }

      void load(val_trait& val, reader_trait &pen)
      {
            pen >> val;
      }
};

The value_trait class is responsible for serializing and loading basic data types. The code shown above is tweaked to be used with the Serialization Filer Classes described in the previous section.

The following excerpt illustrates the code that breaks down complex STL types into their primitive components.

//Sequence-list datatype serializer class.
//Triggers the read or write to the serialization file for the Sequence-list datatypes.
//This class takes care of STL types -- list, vector, stack, queue, deque
// and priority_queue
//NOTE: 'basic_string' type is not treated as sequence-list, but as basic type.
template <class sequence_list_type, class val_filer_trait >
class sequence_list_value_trait
{
public:
      typedef typename val_filer_trait::writer_type writer_trait;
      typedef typename val_filer_trait::reader_type reader_trait;

      typedef typename sequence_list_type::size_type size_type;   
      typedef typename sequence_list_type::value_type value_type;

      void serialize (sequence_list_type& val, writer_trait &pen )
      {
            value_trait<size_type, val_filer_trait> size_filer;
            size_filer.serialize (val.size(), pen);

            for(sequence_list_type::iterator i=val.begin(); i != val.end(); i++)
            {
                  value_trait<value_type, val_filer_trait> val_trait_key_filer;

                  val_trait_key_filer.serialize(*i,pen);
            }
      }

      void load (sequence_list_type& val, reader_trait &pen )
      {
            value_trait<size_type, val_filer_trait> size_reader;
            size_type val_size=0;
            size_reader.load(val_size, pen);

            for(; val_size > 0; val_size--)
            {
                  value_type element;

                  value_trait<value_type, val_filer_trait> val_trait_key_reader;

                  val_trait_key_reader.load(element, pen);

                  val.push_back(element);
            }
      }
};

The sequence_list_value_trait class breaks down the list, vector, stack, queue, deque, and priority_queue STL types to their finer component types. Thus, a vector of int type will be broken down into a list of int types in the sequence iterated by the default vector::iterator type. And the int type values are serialized by the value_trait class.

//Triggers the read or write to the serialization file for the Associative-list
// datatypes.
//This class takes care of STL types -- map, multimap, set, multiset
template <class associative_list_type, class val_filer_trait >
class associative_list_value_trait
{
public:

      typedef typename val_filer_trait::writer_type writer_trait;
      typedef typename val_filer_trait::reader_type reader_trait;

      typedef typename associative_list_type::size_type size_type;
      typedef typename associative_list_type::key_type key_type;
      typedef typename associative_list_type::mapped_type data_type;

      void serialize (associative_list_type& val, writer_trait &pen )
      {
            value_trait<size_type, val_filer_trait> size_filer;
            size_filer.serialize (val.size(), pen);

            for(associative_list_type::iterator i=val.begin(); i != val.end(); i++)
            {
                  value_trait<key_type, val_filer_trait> val_trait_key_filer;
                  value_trait<data_type, val_filer_trait> val_trait_data_filer;


                  val_trait_key_filer.serialize(i->first,pen);
                  val_trait_data_filer.serialize(i->second,pen);
            }
      }

      void load (associative_list_type& val, reader_trait &pen )
      {
            value_trait<size_type, val_filer_trait> size_reader;
            size_type val_size=0;
            size_reader.load(val_size, pen);

            for(; val_size > 0; val_size--)
            {
                  key_type key_element;
                  value_trait<key_type, val_filer_trait> val_trait_key_reader;
                  val_trait_key_reader.load(key_element, pen);

                  data_type data_element;
                  value_trait<data_type, val_filer_trait> val_trait_data_reader;
                  val_trait_data_reader.load(data_element, pen);

                  val.insert (std::pair<key_type, data_type> (key_element, data_element));
            }
      }
};

The associative_list_value_trait class is a complicated version of the sequence_list_value_trait class where in the map, multimap, set and multiset STL types are broken down into their key and data-element types and broken down further as required, or serialized.

The following extract depicts the mechanism by which a complex STL type initiates the breakdown process using the associative_list_value_trait and sequence_list_value_trait classes covered above.

//STL vector datatype serializer class.
//Triggers the read or write to the serialization file for the STL vector datatype.
template <class val_trait_key, class val_trait_data, class val_filer_trait  > 
class value_trait< std::vector<val_trait_key, val_trait_data> , val_filer_trait  >
{
public:

      typedef typename val_filer_trait::writer_type writer_trait;
      typedef typename val_filer_trait::reader_type reader_trait;

      typedef std::vector<val_trait_key, val_trait_data> vector_value_trait;

      void serialize (vector_value_trait& val, writer_trait &pen )
      {
            sequence_list_value_trait<vector_value_trait, 
                     val_filer_trait> sequence_list_value_filer;
            
            sequence_list_value_filer.serialize (val, pen);
      }

      void load (vector_value_trait& val, reader_trait &pen )
      {

            sequence_list_value_trait<vector_value_trait, 
                     val_filer_trait> sequence_list_value_reader;

            sequence_list_value_reader.load (val, pen);
      }
};

The above section of code initiates the dissolution of the vector STL type into granular components for further serialization. Similar logic is implemented for the list, stack, queue, deque, and priority_queue STL types.

//STL multimap datatype serializer class.
//Triggers the read or write to the serialization file for the STL multimap datatype .
template <class val_trait_key, class val_trait_data, class val_filer_trait  > 
class value_trait< std::multimap<val_trait_key, val_trait_data> , val_filer_trait  > 
{
public:

      typedef typename val_filer_trait::writer_type writer_trait;
      typedef typename val_filer_trait::reader_type reader_trait;

      typedef std::multimap<val_trait_key, val_trait_data> multimap_value_trait;

      void serialize (multimap_value_trait& val, writer_trait &pen )
      {

            associative_list_value_trait<multimap_value_trait, 
                        val_filer_trait> associative_list_value_filer;

            associative_list_value_filer.serialize (val, pen);
      }

      void load (multimap_value_trait& val, reader_trait &pen )
      {

            associative_list_value_trait<multimap_value_trait, 
                        val_filer_trait> associative_list_value_reader;

            associative_list_value_reader.load (val, pen);
      }
};

Like the above excerpt that dealt with fragmenting the vector type to finer components, this piece of code dissolves a multimap into its key type and data-type component for further serialization. Similar logic may be implemented for other associative containers such as map, set, and multiset.

Comments

It should be noted that the above library creates a class for every sub-type in the STL filer declaration. Although this dose not account to any extra runtime over-head, it doubles the number of classes that are serialized in a namespace. If the class count exceeds that supported by your compiler, you might want to re-factor the class hierarchy.

The above fact may be seen as a disadvantage, but it should be noted that the class count does not exert any overhead at run-time. The advantage is that STL-SL code is type safe.

STL-SL as presented here is the first revision; you may want to include the type information about the STL type that is being serialized in the serialization file, thus aiding run-time type checking in STL-SL.

Other libraries such as BOOST are available that are much more powerful but arguably not necessarily intuitive. These libraries address a multitude of other problems such as class versioning, pointer restoration, and data portability. The purpose of this article is not to replace them; this article should be seen as a light-weight, easy to use alternative. Moreover, code presented here may be viewed as an STL serialization engine for an even-fuller implementation of a serialization library.

License

This article, along with any associated source code and files, is licensed under The Code Project Open License (CPOL)

Share

About the Author

Aniruddha Jahagirdar
Web Developer
India India
No Biography provided

Comments and Discussions

 
QuestionI want to add number of structure in a list. But library shows error? PinmemberMaruthuj19-Oct-08 20:21 
GeneralCompiling error when the data is a class type PinmemberBing1688886-Mar-08 6:38 
GeneralDoes not compile with sets under VS2005 Pinmembersharkwatch29-Aug-07 20:04 
GeneralRe: Does not compile with sets under VS2005 PinmemberAniruddha Jahagirdar2-Sep-07 18:52 
GeneralRe: Does not compile with sets under VS2005 PinmemberKikoa20-Nov-07 1:34 
Generaldon't compile Pinmemberhenryleesd17-Nov-06 6:43 
GeneralRe: don't compile PinmemberAniruddha Jahagirdar2-Sep-07 18:56 
QuestionVC++ 6.0? PinmemberGHop26-Oct-06 7:46 
AnswerRe: VC++ 6.0? PinmemberAniruddha Jahagirdar2-Sep-07 19:00 
GeneralTo boost or not to boost? why this question: Pinmemberemilio_grv10-Oct-06 4:41 
GeneralRe: To boost or not to boost? why this question: PinmemberAnonymuos10-Oct-06 9:35 
GeneralRe: To boost or not to boost? why this question: Pinmemberemilio_grv11-Oct-06 6:01 
GeneralRe: To boost or not to boost? why this question: PinmemberStephen Hewitt15-Oct-06 20:55 
GeneralRe: To boost or not to boost? why this question: Pinmemberemilio_grv17-Oct-06 0:19 
May be you’re right, but may be I’m the 1 of 10 you spoken about.
 
Stephen Hewitt wrote:
That said, I’ve got enough experience in the trade to know that nine times out of ten, when you make such a home grown solution to a problem that’s been solved many times by many different people, the correct thing to do when you’ve got it working is to throw your code in the bin and use an off the self implementation. This is normally a win-win situation;

 
You have, in general to consider also these points:
  • The time you spend to seek an existing implementation
  • The “learning curve” you have about that implementation
  • The “dis-adaptation” it has with your design: feature that are include and you don’t need and missing feature you have to workaround.
There are many case that the problem is by itself so simple, that writing and testing yourself a code is more efficient than going inside all that stuff.
Unless the problem is so general you can think to solve it once forever.
 
My experience with boost is that it always have something more and something missing, and adapt to work with it is worst that avoid it.
Sometimes I got the feeling that authors want to show their intelligence blowing up features, rather than do a correct modularization.
 
More in general, OK to inform about possible choices, but a minimum of rational and comparison, instead of an “already done” without any explanation (that sounds like “shut-up, holy sh*t”) like mostly every time boost fans do, should be a minimum.
 


 

 

2 bugs found.
> recompile ...
65534 bugs found.
D'Oh! | :doh:

GeneralAlready been done - Boost Serialize lib! Pinmemberyafan9-Oct-06 6:15 
GeneralRe: Already been done - Boost Serialize lib! PinmemberAnonymuos9-Oct-06 6:59 
GeneralRe: Already been done - Boost Serialize lib! [modified] PinmemberJohn M. Drescher9-Oct-06 8:58 
GeneralRe: Already been done - Boost Serialize lib! PinmemberAniruddha Jahagirdar9-Oct-06 21:20 
GeneralRe: Already been done - Boost Serialize lib! PinmemberStephen Hewitt9-Oct-06 23:13 
GeneralRe: Already been done - Boost Serialize lib! PinmemberAniruddha Jahagirdar9-Oct-06 23:59 
GeneralRe: Already been done - Boost Serialize lib! PinmemberStephen Hewitt10-Oct-06 2:22 
GeneralRe: Already been done - Boost Serialize lib! PinmemberJohann Gerell10-Oct-06 2:36 
GeneralRe: Already been done - Boost Serialize lib! PinmemberAniruddha Jahagirdar10-Oct-06 20:48 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

| Advertise | Privacy | Terms of Use | Mobile
Web04 | 2.8.141223.1 | Last Updated 9 Oct 2006
Article Copyright 2006 by Aniruddha Jahagirdar
Everything else Copyright © CodeProject, 1999-2014
Layout: fixed | fluid