Click here to Skip to main content
Click here to Skip to main content

Tiny Template Library: variant

By , 23 Feb 2004
 

Introduction

My standard warning: Don't try to compile this project with MSVC v6.0/v7.0. This project requires a compliant compiler. MSVC v7.1 or GCC v3.2.3 will work just fine. In the article about typelist, I briefly mentioned variant. Here, I'd like to discuss it in more detail. The code in this article is to demonstrate the basic ideas only. The actual implementation can be found in TTL. In C++, it is not legal to have non-POD data types in union. For example the following code won't compile.

struct my_type
{
   int x;
   my_type() : x(0) {}
   virtual ~my_type();
};

union my_union
{
   my_type a;
   double b;
};

The variant template solves this problem and adds a lot of other cool features. One application of variant is heterogeneous containers.

typedef variant< my_type, double > mv;

main()
{
  my_type a;
  
  std::vector< mv > v;
  v.push_back(2.3); //add double 
  v.push_back(a); //add my_type
}

The variant semantic was inspired by boost::variant. A very good discussion about variant can be found in "Modern C++ Design" by A. Alexandrescu.

Implementation

variant has a variable number of template parameters.

variant<int>

variant<int, double>

...
To support variable numbers of template parameters, we use the technique that was suggested in the typelist article.

The main variant implementation ideas are:

  • Compile-time: convert variant template parameters to typelist.
  • Compile-time: using the typelist, find the largest element and reserve a buffer of this element size.
  • Run-time: use the reserved buffer to construct controlled objects in place.
  • Run-time: keep the index of the current instance type.
  • Run-time: if not initialized, variant is in a singular state.

    The variant pseudo-code looks like this:

    template < typename T1, typename T2, ... > 
    struct variant
    {
        //list of user types
        typedef meta::typelist< T1, T2,...> types;
        
        //initialization 
        variant() : p_(0) {}
        
        template< typename T >
        variant( const T& d ) : p_(0) 
        {
            //find the index of this type in the variant typelist
            which_ = find_type<T>::value;
            //in place construction
            p_ = new(buffer_) data_holder<T>(d);
        }
        
        virtual ~variant() { destroy(); }
        
        inline int which() const { return which_; }
        
        inline bool is_valid() const { return p_ != 0; }
        
        void destroy() 
        { 
            if(!is_valid()) return;
            p_->~data_holder_base(); 
            p_ = 0;
        }
        
    private:
        
        //data_holder is a wrapper for the user types
        
        struct data_holder_base 
        { 
            virtual ~data_holder_base() {}
        };
        
        temlate< typename T >
        struct data_holder : data_holder_base
        {
            T d_;
            data_holder( const T& d ) : d_(d) {}
        };
        
        //list of data holder types
        typedef meta::typelist<data_holder<T1>, data_holder<T2>,...> holder_types;
        
        //reserve enough space to hold the largest type
        char buffer_[find_largest_type<holder_types>::value];
    
    
        
        //pointer to the controlled object
        data_holder_base *p_;
        
        //object type index
        int which_;
    };
    
    Please note: the above code is only a pseudo-code. The actual implementation is more complex. It might take a small book to describe all the details. I'd rather talk about how to use variant in practice.

    Using variant

    Let's consider a simple example:
    typedef variant<int, double> my_variant;
    
    This typedef defines a data type that can contain a double or int variable. Suppose that we need to write a function that does something with my_variant depending on the variable data type. We can use a simple switch/case statement.
    void f( my_variant& v )
    {
        int n;
        double x;
        switch( v.which() )
        {
        case 0:  //int variable
            n = get<int>(v);
            //do something with int;
            break;
            
        case 1:  //double variable
            x = get<double>(v);
            //do something with the double;
            break;
        }
    };
    

    As you can see, the get<> function can be used to retrieve the typed data from variant<>. Obviously this switch statement is ugly and not very flexible. The function f() has to know the type indexes in my_variant. One way to solve these problems is to utilize the Gof visitor pattern ideas.

  • Define a variant visitor functor that has a separate operator() for all types in variant.
  • When applied to the variant, an appropriate visitor's operator() is called.

    TTL's variant has the apply_visitor<> function that takes care of calling the appropriate visitor operator(). Using this technique, the above example can be implemented as follows.
    typedef variant<int, double> my_variant;
    
    struct visitor
    {
        void operator()(int n)
        {
        //do something with the int;
        ...
        }
        void operator()(double x)
        {
        //do something with the double;
        ...
        }
        
        //ignore any other types
        template< typename T >
        void operator()( T d )
        {
        }
    };
    
    my_variant var;
    visitor vis;
    apply_visitor(var, vis);
    

    I think that it looks much nicer and we don't have to worry about type indexes or any other type identifiers for the same matter. The apply_visitor() function is implemented in TTL. apply_visitor does the following:

  • finds what type is identified by the which_ member;
  • casts the object's pointer to this type;
  • passes the casted pointer to the user supplied visitor.
  • the compiler automatically selects the appropriate operator().

    Another interesting implementation of variant is event dispatching. Suppose we have an event source that can generate multiple event types. For the simplicity sake, the event types are int and double.  We can define the event type as following:

    typedef variant< int, double > event;
    
    Now we need a way to specify a callback function that will be called by the event source to notify the client or observer. It is convenient to define callbacks using generic functors (see, TTL:implementing functors)

    typedef function< void (event&) > callback;
    
    Now we can put everything together:
    typedef variant< int, double > event;
    typedef function< void (event&) > callback;
    
    struct event_source
    {
        callback cb_
    
        event_source( callback& cb ) : cb_(cb) {}
    
        void do_something()
        {
            event ev;
    
            ...
            //generate int event
            ev = 1;
            cb_( ev );
    
            ....
            //generate double event
            ev = 2.3;
            cb_( ev );
        }
    };
    
    //define our event vistor
    struct event_visitor
    {
        //process int event
        void operator()(int n)
        { 
            cout << "got int:" << n;
        }
        
        //process double event
        void operator()(double n)
        { 
            cout << "got double:" << n;
        }
        
        //ignore any other events
        template< typename T >
        void operator()( T d )
        {
        }
    }
    
    void my_callback( event& e )
    {
        event_visitor vistor;
        apply_visitor(e, vistor);
    }
    
    main()
    {
        event_source src(my_callback);
        src.do_something();
    }
    
    You can find a working example in the samples/test folder. It is not hard to extend this example to a complete Observer pattern implementation w/o any polymorphic inheritances. As a result, a "strong" type checking is performed at compile time.
  • License

    This article has no explicit license attached to it but may contain usage terms in the article text or the download files themselves. If in doubt please contact the author via the discussion board below.

    A list of licenses authors might use can be found here

    About the Author

    kig
    Web Developer
    United States United States
    Member
    No Biography provided

    Sign Up to vote   Poor Excellent
    Add a reason or comment to your vote: x
    Votes of 3 or less require a comment

    Comments and Discussions

     
    You must Sign In to use this message board.
    Search this forum  
        Spacing  Noise  Layout  Per page   
    GeneralPerformance and Detailsmembertpolzin25 Aug '04 - 4:27 
    I looked at different possibilities for event dispatching, similiar to the one described in your example. I did some basic perfomance tests and found   the following:
     
    union, enum, switch-statement 0.05
    union, functionptr                  0.13
    base class/virtual method      0.52
    base class/virtual method using boost::object_pool lazy-delete
                                                 0.25
    base class/virtual method using boost::pool
                                                 0.14
    boost::bind                           0.44
    boost::variant                        0.28
    ttl::varaint                           0.25
     
    (microseconds on my PC, gcc 3.3.3 -O3 )
     
    I won't go into details how the different methods were implemented and the results are nothing to rely on. Overall it seems, that using variant has the best tradeoff between performance and ease of coding.
    Still, I was surprised that boost::pool is faster than variant and that the gap to using a hand coded union, _which enum and a switch statement   or a functionpoiter is noticable.
    Do you have an explaination for this?
     
    A disadvantage of using variant is that without optimization, it is the slowest. (>1 microsecond).
    A sidenote: Using gcc 3.4.1 each of the two variant's get slower, and ttl::variant is then slower than boost::variant.
     
    Concerning your article, I have some probably silly questions:
    1. What is the reasoning for making ~variant "virtual"?
    2. What does "p_->~data_holder_base();" do, as there is no destructor defined in data_holder_base or any data_holder?
    3. In the description of "apply_visitor": "the compiler automatically selects the appropriate operator()".
    Did I get it right that in the case the compiler cannot detect the current type at runtime (e.g. there is a vector of events, created at runtime) the following happens: Using typelists and inlining the compiler generates automatically an if-then-else chain: "if (which_ == 1 ) visitor( get<type1>(v)); else if (which_ == 2 ) ..."
     
    Regards,
       Tobias
    GeneralRe: Performance and Detailsmemberkig25 Aug '04 - 19:02 
    Cool! Have you tried to profile it?
     
    > Do you have an explaination for this?
     
    Perhaps compilers optimize switch/case statements
    much better than if/else chains.
    If it is the case then there is no surprise.
    boost::pool is basically constructing objects
    in a reserved space just like variant does it
    while there is no if/else chains.
    I think that it is quite possible to change
    variant to switch/case from if/else.
    The type_switch code in typlist.h won't be
    that elegant but it is possible.
     
    How do you results depend on number of
    different event types if at all?
     
    Another reason could be all this overhead
    associated with accessing data and constructing
    temporary objects on the stack in variant.
     
    > Concerning your article, I have some probably silly questions:
     
    No, they are not silly on the contrary.
     
    > 1. What is the reasoning for making ~variant "virtual"?
     
    I guess if you want variant to be a base class.
    However I agree that it may not be a compelling reason.
     
    > 2. What does "p_->~data_holder_base();" do, as there is no destructor defined in data_holder_base or any data_holder?
     
    Please download the latest version from sourceforge http://tinytl.sourceforge.net.
    It doesn't call ~data_holder_base() any more.
    data_holder_base used to have a virtual destructor.
    However I still have to call p->~data_holder< T >();
    variant constructs data_holder objects in-place in a reserved buffer that is equal to the size
    of the largest data_holder type. data_holder constructs and stores user defined objects.
    I use data_holder (to keep the user data) instead of the user types directly for several reasons,
    one of which is the alignment.
    template< typename T >;
    struct data_holder { T d_; };
    sizeof(data_holder< T >) could be very different from sizeof(T) depending on the T alignment.
    By calling p_->~data_holder() I am destroying the user object that is stored
    in the data_holder.
    Does it make sense?
     
    > 3. In the description of "apply_visitor": "the compiler automatically selects the appropriate operator()".
    > Did I get it right that in the case the compiler cannot detect the current type at runtime
     
    You are absolutely correct. I meant that compiler creates associations between
    operator() overloads and variant types.
     
    > Using typelists and inlining the compiler generates automatically an if-then-else chain: "if (which_ == 1 ) visitor(
     
    get(v)); else if (which_ == 2 ) ..."
     
    Correct. The magic happens in type_switch (typlist.hpp).
     
    I think that the biggest performance difference between
    boost::variant and ttl::variant happens in assignments
    in certain cases.
     
    If a boost::variant type is controlling only
    non-POD types (that might throw), it'll allocate
    a default constructible object in the memory heap
    every time you do an assignment. It is done
    to provide basic exception safety guarantees
    because boost::variant's semantic
    doesn't allow singular (when the variant
    content is undefined) variant states.
    ttl::variant allows singular states. So it never
    uses memory heap.
    Does it make sense?
     
    Kig

    GeneralRe: Performance and Detailsmembertpolzin25 Aug '04 - 23:26 
    Thanks for your answer.
     
    > Cool! Have you tried to profile it?
     
    No, probably it´s not worth it. It tempting, because one could do it, but in the end it will probably never be even noticeable in a real application if one operation takes 0.05 oder 0.2 mikroseconds. Probably one could also twist the way the objects are stored or created and change the values.
     
    I did not mention, that I measured the times for the creation phase and the calling seperately.
     

    enum
    creation time: 0.05
    calling   time: 0.01
     
    functionpointer
    creation time: 0.05
    calling   time: 0.06
     
    virtual
    creation time: 0.3
    calling   time: 0.23
     
    virtual object_pool
    creation time: 0.15
    calling   time: 0.08
     
    virtual pool
    creation time: 0.1
    calling   time: 0.06
     
    boost::bind
    creation time: 0.29
    calling   time: 0.1
     
    boost::variant
    creation time: 0.2
    calling   time: 0.1
     
    ttl::variant
    creation time: 0.18
    calling   time: 0.05
     
    > > Do you have an explaination for this?
    > Perhaps compilers optimize switch/case
    > statements much better than if/else chains.
     
    I looked into the sources and I saw there there is a call to apply_visitor, even if one increases --finline-limit. Thus, switch / if is probably not the problem.
     
    > How do you results depend on number of
    > different event types if at all?
     
    I did not test.
     
    > Another reason could be all this overhead
    > associated with accessing data and constructing
    > temporary objects on the stack in variant.
     
    This could be the problem. Or even that because of the alignment the size of the data for variant is bigger than for boost_pool. Or the initialisation of the vector of variants. Or perhaps one save one copying of the data:
    variant: construction of temporary event -> variant -> vector
    boost_pool: construction of temporary event -> boost_pool. In the vector there is strored only a pointer in this case.
    To understand this, you probably have to see my test-code, but I don´t dare to post it. Wink | ;-)
     
    >> 2. What does "p_->~data_holder_base();" do, as there is no destructor defined in data_holder_base or any data_holder?
     
    > However I still have to call p->~data_holder< T >();
    > variant constructs data_holder objects in-place in a reserved buffer that is equal to the size
    >of the largest data_holder type. data_holder constructs and stores user defined objects.
    > By calling p_->~data_holder() I am destroying the user object that is stored
    in the data_holder.
    > Does it make sense?
     
    Yes, I did not think of the "default destructor".
     
    Different issue:
    I wanted to have the code for "visiting" defined at the classes that are put together by variant. And I wanted to add an additional parameter. I did this by using a visitor
     
    struct my_visitor
    {
       int _param;
       void setParam(int _param)
          { _param = param; }
       template< typename T >
       void operator()( T d )
          {
             d.process_event( _param );
          }
    };
     
    [...]
       visitor.setParam( param )
       apply_visitor( visitor, event )
    [...]
     

    Is this the way one should do this?
     
    Regards,
       Tobias
     
    P.S. Is there a way to quote replys in the codeproject forum?
    GeneralRe: Performance and Detailsmemberkig26 Aug '04 - 18:16 
    Interesting.
    The calling time of ttl::variant is 2 times faster than boost::variant.
     
    The variant's calling time is in line with functionpointer.
    This is what I would expect.
     
    >variant: construction of temporary event -> variant -> vector
    boost_pool: construction of temporary event -> boost_pool. In the vector there is strored only a pointer in this case.
     
    This would make a big difference.
     
    > Different issue: ...
     
    Yes, it looks good... some comments.
     
    Using undescores before identifiers is not recommended.
     
    You may want to consider putting param in constructor.
    struct my_visitor
    {
       int _param;
       my_visitor( int p ) : _param(p) {}
     
       template< typename T >
       void operator()( const T& d )
                               ^^^^^^^^^^
          {
             d.process_event( _param );
          }
       ...
    };
     
    apply_visitor( my_visitor(param), event );
     
    In some cases, sending events by reference could be more efficient.
     
    I am not completely clear is process_event() an event's method?
    It seems a bit unusual, event is processing itself. Smile | :)
     
    > P.S. Is there a way to quote replys in the codeproject forum?
     
    Sorry, I don't understant what you mean.
     
    Best,
    kig
     

    GeneralRe: Performance and Detailsmembertpolzin26 Aug '04 - 22:58 
    > Interesting.
    > The calling time of ttl::variant is 2 times faster than boost::variant.
     
    Remeber: On gcc-3.4.1 it´s
     
    boost::variant
    create 0.17
    call   0.09
     
    ttl::variant
    create 0.23
    call   0.07
     
    And again the warning: This is not too serious testing. I just noticed, when one changes the order in which the tests are performed, boost::variants calling times and ttl::variants calling time are much closer (0.10 / 0.07).
    I tried adding an <int> to the boost variant. And although I thought for a struct with some ints this should not make a difference, I notice a small improvement. (0.02 improvement in creating, 0.01 in calling)
     
    > The variant's calling time is in line with functionpointer.
    > This is what I would expect.
     
    Concidering the possible twisting possibilities and measurement errors, all but the enum and the virtual method are more or less in line. Smile | :)
     
    > Using undescores before identifiers is not recommended.
     
    It´s mostly a matter of taste. And even more, it´s a matter of convention. This is the defined coding style in my group. I believe having any convention for this is better than having none.
     
    > You may want to consider putting param in   constructor.
    [...]
    >apply_visitor( my_visitor(param), event );
     
    Do I have an outdated version of variant.hpp? One has to add some new methods in variant.hpp, because the temporary my_visitor is const.
     
    funcptr.cpp:659: error: invalid initialization of non-const reference of type '
       ttl_visitor&' from a temporary of type 'ttl_visitor'
    ttl/var/variant.hpp:269: error: in passing argument 1 of `void
       ttl::var::apply_visitor(F&, V&) [with F = ttl_visitor, V =
       ttl::var::variant<const this_c, const that_c, const nothing_c,
       ttl::empty_type, t[...]ttl::empty_type>]'
     

    Finally I get this to work and it´s much handier to use.
     
    > I am not completely clear is process_event() an event's method?
    > It seems a bit unusual, event is processing itself.
     
    My application is probably somewhat different from classical events. As my "events" are something like "at a specified time, perform some action, some parameters of this action are known at the creation time of the action, other at calling time", it feels more natural to put the processing code into the "event".
    If you are interested I could explain in more detail.
    > > P.S. Is there a way to quote replys in the codeproject forum?
    > Sorry, I don't understant what you mean.
     
    How can I quote your mail, such that there is your text with leading "> " and I can write in between.
     
    Regards,
       Tobias
    GeneralRe: Performance and Detailsmemberkig1 Sep '04 - 20:50 
    > Do I have an outdated version of variant.hpp? One has to add some new methods in variant.hpp, because the temporary my_visitor is const.
     
    Some compilers have problems with matching temporaries and template parameter references.
    I get the same error with gcc. VC++7.1
    seems to be working just fine.
    How did you get it to work with GCC?
     
    > My application is probably somewhat different from classical events. As my "events" are something like "at a specified time, perform some action,
     
    I think that I can see how it could be useful.
     
    >How can I quote your mail, such that there is your text with leading "> " and I can write in between.
     
    I don't know. I just prefix the first line manually.
     
    Best,
    kig
     

    GeneralRe: Performance and Detailsmembertpolzin2 Sep '04 - 1:15 
    > How did you get it to work with GCC?
     
    I stupidly added/replaced "const" versions to existing "non-const" versions.
     
    I was wondering why there is a "ttl/variant.hpp" and a "ttl/var/variant.hpp". I just used the latter.
     

     
    < const F& f_;
    ---
    > F& f_;
     
    < visitor_adapter_ref( const F& f, V v ) : f_(f), v_(v) {}
    ---
    > visitor_adapter_ref( F& f, V v ) : f_(f), v_(v) {}
     

    < template< typename F, typename V >
    < void apply_visitor( const F& f, V& v )
    < {
    < if( v.is_singular() ) return;
    < meta::type_switch ts;
    < impl::visitor_adapter_ref va(f, v);
    < ts(v.which(), va);
    < }
    <
     
    Regards,
    Tobias
    GeneralRe: Performance and Detailsmemberkig6 Sep '04 - 15:45 
    ttl/var/variant.hpp is the right file. I forgot to delete ttl/variant.hpp.

    QuestionWhat's the difference to Boost.Variant?memberHartmut Kaiser25 Feb '04 - 21:47 
    Hi,
     
    could you please explain to me, what're the different points if it comes to a comparision between your library and Boost.Variant?
     
    Regards Hartmut

    AnswerRe: What's the difference to Boost.Variant?memberkig26 Feb '04 - 7:01 
    Boost.Variant is a fine piece of code.
    However there are few important differences between boost and ttl semantics.
     
    1. ttl::variant can be in an uninitialized (singular) state. It means that the variant doesn't have a valid data. This concept is similar to 'union' that hasn't been initialized except that ttl::variant has the is_singular() method to check it. ttl::variant becomes singular if it has not been initialized or there was an exception during assignment. On the other hand boost::variant always contains a user defined type. If the assignment fails the content of boost::variant might be changed under the covers to a default-constrictable value. boost::variant visitors cannot assume that the variant content has not been changed by the variant internally. I find it disturbing. On the other hand, ttl::variant visitors simply won't be called if the variant is in singular state. So that if a visitor is called, the variant content is guaranteed to be valid (that was set by the user).
     

    2. To support the requirement of non-singularity, boost::variant has to sometimes allocate objects on the heap.
    If none of your variant types has a non-throw default constructible type, boost::variant creates (during assignements) an instance on the heap. For example the following variant type will allocate a default constrictable instance of the user type on heap each time you do an assignment.
     
    struct my_type {}
    typedef boost::variant< my_type > my_variant.
     
    ttl::variant never touches the memory heap! So in some cases ttl::variant can be a much more efficient solution.
     
    You can always prevent boost::variant from using the memory heap by adding a a non-throw default-constrictable type such as 'int'. For example the following variant won't touch the memory heap.
     
    typedef boost::variant< my_type, int > my_variant.
     
    However you'll have to add handlers for this dummy 'int' to all of your visitors. Also adding 'int' might be problemtic in generic context.
     

     
    3. ttl::variant visitors are not required to be derived from another class like boost::variant visitors need to be derived from boost::variant::static_visitor.
     
    4. ttl visitors cannot return a value. This limitation has a simple workaround with a wrapper class.
     
    int visitor( my_type& x );
     
    struct visitor_wrapper
    {
    int return_value;
     
    void operator()( my_type& x )
    {
    return_value = visitor(x);
    }
    };
     
    variant< my_type& > var;
    visitor_wrapper vw;
    apply_visitor(var, vw);
    int r = vw.return_value;
     
    I am planning to generalize such a wrapper latter.
     
    5. IMHO, ttl::variant implementation is much simpler. As a result the compilation times are shorter.
     
    Best,
    kig

    GeneralRe: What's the difference to Boost.Variant?memberHartmut Kaiser26 Feb '04 - 9:08 
    HI,
     
    thanks for your thorough explanations.
    BTW: Have you considered to discuss these issues with the Boost.Variant author?
     
    Regards Hartmut
    GeneralRe: What's the difference to Boost.Variant?memberkig26 Feb '04 - 11:28 
    I discussed some these issues with the boost community. Actually Eric was kind enough to modify some of the original Boost.Variant semantic according to my suggestions, but he seems to think that non-singular variants are still better. I respect his point of view. Perhaps a policy based approach is the way to go. I can imaging variant having a Singularity policy.
     
    Best,
    kig

    AnswerRe: What's the difference to Boost.Variant?memberkig26 Feb '04 - 11:49 
    I forgot to add, TTL (http://tinytl.sourceforge.net/) doesn't support recursive variants like in Boost.Variant.

    GeneralRe: What's the difference to Boost.Variant?memberHartmut Kaiser26 Feb '04 - 19:40 
    Thanks for the clarification. I would like to encourage you to discuss that with Eric further, because this will bring your both implementations forward and will help the community as a whole.
     
    Regards Hartmut

    General General    News News    Suggestion Suggestion    Question Question    Bug Bug    Answer Answer    Joke Joke    Rant Rant    Admin Admin   

    Permalink | Advertise | Privacy | Mobile
    Web03 | 2.6.130523.1 | Last Updated 24 Feb 2004
    Article Copyright 2004 by kig
    Everything else Copyright © CodeProject, 1999-2013
    Terms of Use
    Layout: fixed | fluid