Click here to Skip to main content
13,551,964 members
Click here to Skip to main content
Add your own
alternative version


29 bookmarked
Posted 3 Jun 2003

IoBind, a serializer code factory.

, 29 Jun 2003
Rate this:
Please Sign up or sign in to vote.
IoBind proposes a new approach to object serialization.


IoBind is a highly flexible library for serializing objects to/from string. It uses meta-programming and policies to create reader and writer for complex objects at compile time.

In other words, IoBind is a code factory that produces customized readers and writers: you give the action to take and the objects to process and it will generate the corresponding code.

The latest versions of IoBind are available at SourceForge.


  • Quick examples
    • Converting a string to a string
    • Escaping string to XML
    • Combining policies
    • Reading back data
  • Compiling and instalation
  • The encode method
  • Policies
    • operator +
    • String manipulation
    • Base64
    • Zip
    • XML escaping, Latin1 escaping 
    • sequence container
    • pair structure
    • associative containers
  • History
  • References

Quick examples

Let's start with some quick examples to see what IoBind is about.

Converting a string to a ... string

Let see a first snippet:

<PRE lang=c++>string s("<xml/>"); // we convert s to a string s = encode( s, to_string_p ); cerr<<s<<endl;
-- output

This seems to be useless since I'm converting a string to string. Anyway, let's see what's in the second line:

  • encode is a template function that applies a conversion policy (see below) to s,
  • to_string_p is a conversion policy that takes a value and converts it to a string ( using boost::format library ).

Escaping string to XML

Now, suppose that you need to store this string into an xml document. You need replace escape characters (<, >, etc..) by &lt;, &gt;, etc:

<PRE lang=c++>s="escape me:&\"'<>" s=encode(s, escape_to_xml_p ); cerr<<s<<endl;
-- output
escape me:&amp;&quot;&apos;&lt;&gt;

What has happened here: the string has been converted to an escaped string using the escape_to_xml_p conversion policy.

Hence using predefined policies you can easily transform a string:

  • base64,
  • encryption,
  • zipping,
  • etc...

Combining policies

Having a set of basic policies is good but creating new policies by combining them really makes things interresting  (this is where meta-programming takes place).
Suppose that you have a vector of string, you want to convert it to string and to escape this string to xml:

<PRE lang=c++>vector<string> v_string; // filling v_int v_string.push_back("a string"); v_string.push_back("<string/>"); // transforming to escaped string: s = encode( v_string.begin(), v_string.end(), escape_to_xml_p * sequence_to_string_p ); cerr<<s<<endl;
-- output
a string,<string/>

In this call, sequence_to_string_p is a policy that converts an iterator range to a string. The operator + combines the two policies (similar to function composition) and creates a new conversion policy:

a * b applied to a value v is equivalent to a( b( v ) )

Note that you can also specify a policy to be applied to each value of the container. This, combined with a pair policy, can be used to serializes associatives containers.

Reading back data

All the "to" policies have their "from" counter part to read data. For example, we can easily read a vector<int> from a string: <PRE lang=c++>vector<int> v_int; string str("0,1,2,3,4"); // reading vector of ints v_int = encode(str, sequence_from_string_p % v_int << from_string<INT>() ); where

  • sequence_from_string_p is a generic conversion policy that read a sequence of strings. This conversion policy is a template class that depends on two template paramters:
    • Container, the type of container to fill,
    • Policy, the conversion policy to apply to each element, 
  • the operator % builds a new sequence_from_string conversion policy that adds values to a container of same type as v_int. Concretely, this operator replaces the Container template paramter with the type of v_int,
  • the operator << is similar to % but it works on the policy.

The above is rather confusing, let's go deeper to see how policies are built step by steps: <PRE lang=c++>sequence_from_string_p % v_int << from_string<INT>()

  1. sequence_from_string_p = A: this is a pre-defined sequence conversion policy: it reads a sequence of strings separated by commas,
    • Container = vector<string>
    • Policy = from_string<string>
  2. A % v_int = B: the container has been replaced by v_int type:
    • Container = vector<int>
    • Policy = from_string<string>
  3. B << from_string<INT>() </INT>: the conversion poliy applied to the elements reads a int from a string:
    • Container = vector<int>
    • Policy = from_string<int>

So with this little statement, we have build a new conversion policy that is a complex mixture of A and B.

Compiling and installation

You need Boost (see [1]) and a good compiler (don't think VC6 will manage to compile). IoBind heavily uses Boost: it uses type_traits, mpl, call_traits, spirit, (regex optional) and


The headers can be included by

<PRE lang=c++>#include <iobind/iobind.hpp>

Note also that all the IoBind classes are in the


The encode method

encode is a template method that applies a conversion policy to an object. It's return type depends on the policy return type. It comes with two overloads:

<PRE lang=c++>template< typename Value, typename Policy > typename Policy::return_type encode( Value const& value_, Policy const& policy_ ); <PRE lang=c++>// range version template< typename Iterator, typename Policy > typename Policy::return_type encode( Iterator begin_, Iterator end_, Policy const& policy_ );

This is the front end of IoBind to the user. Example of use of this method were given in the secion above.

Conversion policies

The individual steering behaviors [...] are components of a larger structure, like notes of a melody or words of a story. Craig Reynolds, Steering Behaviors for Autonomous Characters, GDC99.

Conversion policies are the constitutive pieces of IoBind. You can combine them to create complex serializers.

operator *

Takes two policies and combines them. If a and b are two policies, then 

a*b( value) = a( b( value ) ).

String manipulation

This is a basic string conversion policy not very useful used alone, but it become quite handy when combined with others.

  • <PRE lang=c++>struct to_string;converts the value using boost::format. If the value does not support this, the compiler will fail.
  • <PRE lang=c++>template<typename Value> struct from_string;transform a string to Value using ostringstream,


<PRE lang=c++>int i; string str=encode(i, to_string() ); i=encode(str, from_string<std::string><INT>() );

Note that almost all policies have predifined instances: the

policy is instanciated as to_string_p .


Converts streams to the base64 scheme.

  • <PRE lang=c++>struct to_base64;converts a stream to base64 notation,
  • <PRE lang=c++>struct from_base64;converts back a stream from base64 notation,


<PRE lang=c++>string str="test"; str=encode( str, to_base64_p); str=encode( str, from_base64_p);

These policies are base on the base64 iostream converter from Konstant Pilipchuk:

//  base64.hpp 
//  Autor Konstantin Pilipchuk

XML and Latin1 escaping

This policy takes care of transforming a string to XML or Latin1 conformant string. It replaces reserved characters <,>,... by &lt;, &gt;, etc...

  • <PRE lang=c++>struct escape_to_xml; struct escape_to_latin1;escapes an string to XML (< to &lt;) or Latin1,
  • <PRE lang=c++>struct unescape_from_xml; struct unescape_from_latin1;unescapes an string from XML (&lt; to <) or Latin1,

The usage is straight forward and similar to to_base64, from_base64.

Sequence container

This policy handles sequence containers such as vector, list, etc (as you will see later, it can also be used for associative containers).

  • <PRE lang=c++>template< typename Policy > struct sequence_to_string;converts a sequence to a string. This policy has the following constructor: <PRE lang=c++>sequence_to_string( policy_const_reference item_policy_, string_param_type begin_ = "", string_param_type delimiter_ = ",", string_param_type end_ = "" ) where item_<CODE>policy_ is the policy applied to the sequence elements, and the other parameters are used to separated the data. In fact, the core of the writer is: <PRE lang=c++>output <<m_begin .. <<m_policy.encode(value)<<m_delimiter, // this is done multiple times .. <<m_end;
  • <PRE lang=c++>template< typename Container, typename Policy > struct sequence_from_string;reads a sequence from a string and fills a container. This policy has the following constructor: <PRE lang=c++>sequence_from_string( policy_const_reference item_policy_, string_param_type begin_ = "", string_param_type delimiter_ = ",", string_param_type end_ = "" ) where the parameters have similar behavior as above. The item_policy_ is used to transform the string before adding it to the container.

These policies support other operators that take care of policy or container change:

  • <<, changes the policy,
  • %, changes the container type (only for pair_from_string).

Example converting elements of a vector<float> to base64 and back:

<PRE lang=c++>vector<float> v_f; for (i=1;i<5;++i) v_f.push_back( 1/static_cast<FLOAT>(i) ); str=encode( v_f.begin(), v_f.end(), sequence_to_string_p << to_base64() ); cerr<<"\tv (to_string, base64): "<<str<<endl; cerr<<"\tv is cleared..."<<endl; v_f.clear(); v_f=encode( str, sequence_from_string_b( v_f, from_string<float>() * from_base64() ) ); cerr<<"\tv (from_string from base64): "<<encode( v_f.begin(), v_f.end(), sequence_to_string_p ) <<endl;
-- output
        v (to_string, base64): MQA=,MC41AA==,MC4zMzMzMzMA,MC4yNQA=
        v is cleared...
        v (from_string from base64): 1,0.5,0.333333,0.25


This policy handles the famous std::pair structure.

  • template<
        typename FirstPolicy,
        typename SecondPolicy
    class pair_to_string;
    converts a pair to a string. This class has the following constructor: <PRE lang=c++>pair_to_string( first_policy_const_reference first_policy_, second_policy_const_reference second_policy_, string_param_type begin_ = "(", string_param_type delimiter_ = ":", string_param_type end_ = ")" ) where first/second_policy_ are the policies applied respectively to the first and second members of pair, and the other parameters are used to separated the data. In fact, the core of the writer is: <PRE lang=c++>output <<m_begin <<m_first_policy.encode(value.first) <<m_delimiter, <<m_second_policy.encode(value.second), <<m_end;
  • template<
        typename Pair,
        typename FirstPolicy,
        typename SecondPolicy
    class pair_from_string;
    reads a pair from a string. This class has the following constructor: <PRE lang=c++>pair_from_string( first_policy_const_reference first_policy_, second_policy_const_reference second_policy_, string_param_type begin_ = "(", string_param_type delimiter_ = ":", string_param_type end_ = ")" ) where first/second_policy_ are the policies applied respectively to the first and second members of pair, and the other parameters are used to separated the data. In fact, the core of the writer is: <PRE lang=c++>pair.first=m_frist_policy.encode(first_string); pair.second=m_second_policy.encode(second_string);

These policies support new operators that take care of policy, pair type change:

  • <<, changes the first policy,
  • >>, changes the second policy,
  • %, changes the pair type (only for pair_from_string).


<PRE lang?c++?>pair<int,string> p_fs(1,"second"); str=encode( p_fs, pair_to_string_p); cerr<<"\tpair (1,second): "<<str<<endl; cerr<<"\treseting pair"<<endl; p_fs.first=0; p_fs.second=""; p_fs=encode( str, pair_from_string_p % p_fs << from_string<int>() >> from_string<std::string>() ); cerr<<"\tpair (from string):"<<encode( p_fs, pair_to_string_p)<<endl;
-- output
        pair (1,second): (1:second)
        reseting pair
        pair (from string):(1:second)

Associative containers

Associative containers such as map,set, etc... are just a combination of a sequence container and a pair (speaking about serialization). Hence, using sequence_to/from_string and pair_to/from_string, you can easily build serializers for them, witout redifining new classes (the compiler will build them for you):

<PRE lang=c++>map<float,string> m_fs; const char* numbers[] = {"one", "two", "three", "four", "five"}; for (i=1;i<5;++i) m_fs[static_cast<float>(i)]=numbers[i]; // dumping to string str=encode( m_fs.begin(), m_fs.end(), sequence_to_string_p << pair_to_string_p ); cerr<<"\tm (to_string): "<<str<<endl; cerr<<"\tm is cleared..."<<endl; // reading back the data m_fs.clear(); m_fs=encode( str, sequence_from_string_p % m_fs << ( pair_from_string_p % pair<float,std::string>() << from_string<float>() >> from_string<std::string>() ) ); cerr<<"\tm (from_string): "<<encode( m_fs.begin(), m_fs.end(), sequence_to_string_p << pair_to_string_p ) <<endl;
-- output
-- associative container:
    combination of sequence_to/from_string and pair
        m (to_string): (1:two),(2:three),(3:four),(4:five)
        m is cleared...
        m (from_string): (1:two),(2:three),(3:four),(4:five)


The zip policy uses the famous zlib C library (see [2]). It compresses a buffer to another buffer. This policy, combined with base64 can be used to store byte buffers in XML files:

<PRE lang=c++>vector<unsigned char> buffer; string s; //zipping and converting to base64 s = encode( buffer, to_base64 * to_zip_p );


There is room for a lot of other policies:

  • encryption, decryption,
  • url encoding, decoding,
  • ...


  • 06-30-2003 - Added zip, latin1, case, crc, hex policies.
  • 05-04-2003 - Initial release.



This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

A list of licenses authors might use can be found here


About the Author

Jonathan de Halleux
United States United States
Jonathan de Halleux is Civil Engineer in Applied Mathematics. He finished his PhD in 2004 in the rainy country of Belgium. After 2 years in the Common Language Runtime (i.e. .net), he is now working at Microsoft Research on Pex (

You may also be interested in...


Comments and Discussions

GeneralSerializing a Spirit Grammar Pin
Karim Attaleb20-Sep-05 7:12
memberKarim Attaleb20-Sep-05 7:12 
QuestionVersion Control? Pin
Dave Handley13-Oct-04 11:47
memberDave Handley13-Oct-04 11:47 
Questiongreat article - similar to java chained streams? Pin
Jim Crafton4-Jun-03 6:48
memberJim Crafton4-Jun-03 6:48 
AnswerExample ? Pin
Jonathan de Halleux4-Jun-03 6:56
memberJonathan de Halleux4-Jun-03 6:56 
GeneralRe: Example ? Pin
Jim Crafton4-Jun-03 11:23
memberJim Crafton4-Jun-03 11:23 
GeneralRe: Example ? Pin
Jonathan de Halleux4-Jun-03 11:38
memberJonathan de Halleux4-Jun-03 11:38 
GeneralRe: Example ? Pin
Jim Crafton4-Jun-03 17:30
memberJim Crafton4-Jun-03 17:30 
GeneralRe: Example ? Pin
Jonathan de Halleux4-Jun-03 20:35
memberJonathan de Halleux4-Jun-03 20:35 
GeneralRe: Example ? Pin
Jim Crafton5-Jun-03 2:58
memberJim Crafton5-Jun-03 2:58 
General-&gt; WikiWeb Pin
Jonathan de Halleux5-Jun-03 3:35
memberJonathan de Halleux5-Jun-03 3:35 
GeneralRe: -&gt; WikiWeb Pin
Jim Crafton5-Jun-03 7:08
memberJim Crafton5-Jun-03 7:08 
GeneralRe: -&gt; WikiWeb Pin
Jonathan de Halleux5-Jun-03 20:22
memberJonathan de Halleux5-Jun-03 20:22 
GeneralRe: -&gt; WikiWeb Pin
Jim Crafton6-Jun-03 3:38
memberJim Crafton6-Jun-03 3:38 
GeneralRe: -&gt; WikiWeb Pin
Jonathan de Halleux6-Jun-03 3:49
memberJonathan de Halleux6-Jun-03 3:49 
QuestionExcellent but link? Pin
Michael A. Barnhart4-Jun-03 6:21
memberMichael A. Barnhart4-Jun-03 6:21 
AnswerRe: Excellent but link? Pin
Jonathan de Halleux4-Jun-03 6:25
memberJonathan de Halleux4-Jun-03 6:25 
AnswerRe: Excellent but link? Pin
Jonathan de Halleux4-Jun-03 6:29
memberJonathan de Halleux4-Jun-03 6:29 
GeneralRe: Excellent but link? Pin
Michael A. Barnhart4-Jun-03 9:29
memberMichael A. Barnhart4-Jun-03 9:29 

General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Praise Praise    Rant Rant    Admin Admin   

Use Ctrl+Left/Right to switch messages, Ctrl+Up/Down to switch threads, Ctrl+Shift+Left/Right to switch pages.

Permalink | Advertise | Privacy | Terms of Use | Mobile
Web03-2016 | 2.8.180515.1 | Last Updated 30 Jun 2003
Article Copyright 2003 by Jonathan de Halleux
Everything else Copyright © CodeProject, 1999-2018
Layout: fixed | fluid