IoBind, a serializer code factory.






4.89/5 (9 votes)
Jun 4, 2003
7 min read

80062

765
IoBind proposes a new approach to object serialization.
Introduction
IoBind
is a highly flexible library for serializing objects
to/from string
. It uses meta-programming and policies to create
reader and writer for complex objects at compile time.
In other words, IoBind
is a code factory that produces
customized readers and writers: you give the action to take and the objects to
process and it will generate the corresponding code.
The latest versions of IoBind are available at SourceForge.
Outline
- Quick examples
- Converting a string to a
string
- Escaping string to XML
- Combining policies
- Reading back data
- Converting a string to a
- Compiling and instalation
- The encode method
- Policies
- operator +
- String manipulation
- Base64
- Zip
- XML escaping, Latin1 escaping
- sequence container
- pair structure
- associative containers
- History
- References
Quick examples
Let's start with some quick examples to see what IoBind is about.
Converting a string to a ... string
Let see a first snippet:
string s("<xml/>");
// we convert s to a string
s = encode( s, to_string_p );
cerr<<s<<endl;
-- output
<xml/>
This seems to be useless since I'm converting a string to string. Anyway, let's see what's in the second line:
encode
is a template function that applies a conversion policy (see below) tos
,to_string_p
is a conversion policy that takes a value and converts it to a string ( usingboost::format
library ).
Escaping string to XML
Now, suppose that you need to store this string into an xml document. You
need replace escape characters (<
, >
, etc..) by
<
, >
, etc:
s="escape me:&\"'<>"
s=encode(s, escape_to_xml_p );
cerr<<s<<endl;
-- output
escape me:&"'<>
What has happened here: the string has been converted to an escaped string
using the escape_to_xml_p
conversion policy.
Hence using predefined policies you can easily transform a string:
- base64,
- encryption,
- zipping,
- etc...
Combining policies
Having a set of basic policies is good but creating new policies by
combining them really makes things interresting (this is where
meta-programming takes place).
Suppose that you have a vector of string, you
want to convert it to string and to escape this string to xml:
vector<string> v_string;
// filling v_int
v_string.push_back("a string");
v_string.push_back("<string/>");
// transforming to escaped string:
s = encode(
v_string.begin(),
v_string.end(),
escape_to_xml_p * sequence_to_string_p
);
cerr<<s<<endl;
-- output
a string,<string/>
In this call, sequence_to_string_p
is a policy that converts an
iterator range to a string. The operator +
combines the two
policies (similar to function composition) and creates a new conversion
policy:
Note that you can also specify a policy to be applied to each value of the container. This, combined with a pair policy, can be used to serializes associatives containers.
Reading back data
All the "to" policies have their "from" counter part to read data. For
example, we can easily read a vector<int>
from a
string:
vector<int> v_int;
string str("0,1,2,3,4");
// reading vector of ints
v_int = encode(str,
sequence_from_string_p
% v_int
<< from_string()
);
where
sequence_from_string_p
is a generic conversion policy that read a sequence of strings. This conversion policy is a template class that depends on two template paramters:Container
, the type of container to fill,Policy
, the conversion policy to apply to each element,
- the operator
%
builds a newsequence_from_string
conversion policy that adds values to a container of same type asv_int
. Concretely, this operator replaces theContainer
template paramter with the type ofv_int
, - the operator
<<
is similar to%
but it works on the policy.
The above is rather confusing, let's go deeper to see how policies are built step by steps:
sequence_from_string_p
% v_int
<< from_string()
sequence_from_string_p
= A: this is a pre-defined sequence conversion policy: it reads a sequence of strings separated by commas,- Container =
vector<string>
- Policy =
from_string<string>
- Container =
A % v_int
= B: the container has been replaced byv_int
type:- Container =
vector<int>
- Policy =
from_string<string>
- Container =
B << from_string
: the conversion poliy applied to the elements reads a() int
from a string:- Container =
vector<int>
- Policy =
from_string<int>
- Container =
So with this little statement, we have build a new conversion policy that is a complex mixture of A and B.
Compiling and installation
You need Boost (see [1]) and a good compiler (don't think VC6 will manage to
compile). IoBind
heavily uses Boost: it uses
type_traits
, mpl
, call_traits
,
spirit
, (regex
optional) and format
libraries.
The headers can be included by
#include <iobind/iobind.hpp>
Note also that all the IoBind classes are in the iobind
namespace.
The encode
method
encode
is a template method that applies a conversion policy to
an object. It's return type depends on the policy return type. It comes with two
overloads:
template<
typename Value,
typename Policy
>
typename Policy::return_type
encode(
Value const& value_,
Policy const& policy_
);
// range version
template<
typename Iterator,
typename Policy
>
typename Policy::return_type
encode(
Iterator begin_,
Iterator end_,
Policy const& policy_
);
This is the front end of IoBind
to the user. Example of use of
this method were given in the secion above.
Conversion policies
The individual steering behaviors [...] are components of a larger structure, like notes of a melody or words of a story. Craig Reynolds, Steering Behaviors for Autonomous Characters, GDC99.
Conversion policies are the constitutive pieces of IoBind
. You
can combine them to create complex serializers.
operator *
Takes two policies and combines them. If a
and b
are two policies, then
a*b( value) = a( b( value ) )
.
String manipulation
This is a basic string conversion policy not very useful used alone, but it become quite handy when combined with others.
struct to_string;
converts the value usingboost::format
. If the value does not support this, the compiler will fail.template<typename Value> struct from_string;
transform a string to Value usingostringstream
,
Example:
int i;
string str=encode(i, to_string() );
i=encode(str, from_string<std::string>() );
Note that almost all policies have predifined instances: the to_string
policy is instanciated as to_string_p
.
Base64
Converts streams to the base64 scheme.
struct to_base64;
converts a stream to base64 notation,struct from_base64;
converts back a stream from base64 notation,
Example:
string str="test";
str=encode( str, to_base64_p);
str=encode( str, from_base64_p);
These policies are base on the base64 iostream converter from Konstant Pilipchuk:
// base64.hpp // Autor Konstantin Pilipchuk // mailto:lostd@ukr.net
XML and Latin1 escaping
This policy takes care of transforming a string to XML or Latin1 conformant string. It replaces reserved characters <,>,... by <, >, etc...
struct escape_to_xml; struct escape_to_latin1;
escapes an string to XML (< to <) or Latin1,struct unescape_from_xml; struct unescape_from_latin1;
unescapes an string from XML (< to <) or Latin1,
The usage is straight forward and similar to to_base64
,
from_base64
.
Sequence container
This policy handles sequence containers such as vector
,
list
, etc (as you will see later, it can also be used for
associative containers).
template< typename Policy > struct sequence_to_string;
converts a sequence to astring
. This policy has the following constructor:sequence_to_string( policy_const_reference item_policy_, string_param_type begin_ = "", string_param_type delimiter_ = ",", string_param_type end_ = "" )
whereitem_<CODE>policy_
is the policy applied to the sequence elements, and the other parameters are used to separated the data. In fact, the core of the writer is:output <<m_begin .. <<m_policy.encode(value)<<m_delimiter, // this is done multiple times .. <<m_end;
template< typename Container, typename Policy > struct sequence_from_string;
reads a sequence from a string and fills a container. This policy has the following constructor:sequence_from_string( policy_const_reference item_policy_, string_param_type begin_ = "", string_param_type delimiter_ = ",", string_param_type end_ = "" )
where the parameters have similar behavior as above. Theitem_policy_
is used to transform the string before adding it to the container.
These policies support other operators that take care of policy or container change:
<<
, changes the policy,%
, changes the container type (only forpair_from_string
).
Example converting elements of a vector<float> to base64 and back:
vector<float> v_f;
for (i=1;i<5;++i)
v_f.push_back( 1/static_cast<FLOAT>(i) );
str=encode(
v_f.begin(),
v_f.end(),
sequence_to_string_p << to_base64()
);
cerr<<"\tv (to_string, base64): "<<str<<endl;
cerr<<"\tv is cleared..."<<endl;
v_f.clear();
v_f=encode(
str,
sequence_from_string_b( v_f, from_string<float>() * from_base64() )
);
cerr<<"\tv (from_string from base64): "<<encode(
v_f.begin(),
v_f.end(),
sequence_to_string_p
)
<<endl;
-- output
v (to_string, base64): MQA=,MC41AA==,MC4zMzMzMzMA,MC4yNQA=
v is cleared...
v (from_string from base64): 1,0.5,0.333333,0.25
Pair
This policy handles the famous std::pair
structure.
template< typename FirstPolicy, typename SecondPolicy > class pair_to_string;
converts apair
to a string. This class has the following constructor:pair_to_string( first_policy_const_reference first_policy_, second_policy_const_reference second_policy_, string_param_type begin_ = "(", string_param_type delimiter_ = ":", string_param_type end_ = ")" )
wherefirst/second_policy_
are the policies applied respectively to thefirst
andsecond
members ofpair
, and the other parameters are used to separated the data. In fact, the core of the writer is:output <<m_begin <<m_first_policy.encode(value.first) <<m_delimiter, <<m_second_policy.encode(value.second), <<m_end;
template< typename Pair, typename FirstPolicy, typename SecondPolicy > class pair_from_string;
reads apair
from a string. This class has the following constructor:pair_from_string( first_policy_const_reference first_policy_, second_policy_const_reference second_policy_, string_param_type begin_ = "(", string_param_type delimiter_ = ":", string_param_type end_ = ")" )
wherefirst/second_policy_
are the policies applied respectively to thefirst
andsecond
members ofpair
, and the other parameters are used to separated the data. In fact, the core of the writer is:pair.first=m_frist_policy.encode(first_string); pair.second=m_second_policy.encode(second_string);
These policies support new operators that take care of policy, pair type change:
<<
, changes the first policy,>>
, changes the second policy,%
, changes the pair type (only forpair_from_string
).
Example:
pair<int,string> p_fs(1,"second"); str=encode( p_fs, pair_to_string_p); cerr<<"\tpair (1,second): "<<str<<endl; cerr<<"\treseting pair"<<endl; p_fs.first=0; p_fs.second=""; p_fs=encode( str, pair_from_string_p % p_fs << from_string<int>() >> from_string<std::string>() ); cerr<<"\tpair (from string):"<<encode( p_fs, pair_to_string_p)<<endl;
-- output
pair (1,second): (1:second)
reseting pair
pair (from string):(1:second)
Associative containers
Associative containers such as map
,set
, etc... are
just a combination of a sequence container and a pair (speaking about
serialization). Hence, using sequence_to/from_string
and
pair_to/from_string, you can easily build serializers for them, witout
redifining new classes (the compiler will build them for you):
map<float,string> m_fs;
const char* numbers[] = {"one", "two", "three", "four", "five"};
for (i=1;i<5;++i)
m_fs[static_cast<float>(i)]=numbers[i];
// dumping to string
str=encode(
m_fs.begin(),
m_fs.end(),
sequence_to_string_p << pair_to_string_p
);
cerr<<"\tm (to_string): "<<str<<endl;
cerr<<"\tm is cleared..."<<endl;
// reading back the data
m_fs.clear();
m_fs=encode(
str,
sequence_from_string_p % m_fs
<< (
pair_from_string_p
% pair<float,std::string>()
<< from_string<float>()
>> from_string<std::string>()
)
);
cerr<<"\tm (from_string): "<<encode(
m_fs.begin(),
m_fs.end(),
sequence_to_string_p << pair_to_string_p
)
<<endl;
-- output
-- associative container:
combination of sequence_to/from_string and pair
m (to_string): (1:two),(2:three),(3:four),(4:five)
m is cleared...
m (from_string): (1:two),(2:three),(3:four),(4:five)
Zip
The zip policy uses the famous zlib C library (see [2]). It compresses a buffer to another buffer. This policy, combined with base64 can be used to store byte buffers in XML files:
vector<unsigned char> buffer;
string s;
//zipping and converting to base64
s = encode( buffer, to_base64 * to_zip_p );
Others
There is room for a lot of other policies:
- encryption, decryption,
- url encoding, decoding,
- ...
History
- 06-30-2003 - Added zip, latin1, case, crc, hex policies.
- 05-04-2003 - Initial release.