Introduction
JSON is a text file format similar to XML, but less verbose. It has been called "XML lite". This article describes JSON Spirit, a C++ library that reads and writes JSON files or streams. It is written using the Boost Spirit parser generator. If you are already using Boost, you can use JSON Spirit without any additional dependencies.
The library supports Unicode. A new feature from version 4.0 is the option to use an std::map implementation for JSON Objects instead of the original std::vector implementation.
The JSON Spirit source code is available as a Microsoft Visual Studio 2005 C++ "solution". However, it should compile and work on any platform compatible with Boost. JSON Spirit has been built and tested with Visual C++ 2005, 2008, and G++ version 4.2.3 on Linux. It has been tested with Visual C++ using Boost versions 1.34.0, 1.37.0 and 1.39.0. It has also been tested with STLPort.
Platform independent CMake files are included, kindly supplied by Uwe Arzt.
The Visual C++ solution consists of four projects:
- The JSON Spirit library
- Two small programs demonstrating how to use JSON Spirit
- An application running the library's unit tests
Using the Code
The Visual C++ solution builds an object library. You can link to this object library or, alternately, you can add the JSON Spirit source files directly to your project. All JSON Spirit declarations are in the namespace json_spirit.
The include files needed to read JSON text are json_spirit_reader.h and json_spirit_value.h. To generate JSON text, you need json_spirit_writer.h and json_spirit_value.h. Alternately, you can include json_spirit.h, which includes all three of the above.
Reading JSON
You can read JSON data from a stream or a string:
bool read( const std::string& s, Value& value );
bool read( std::istream& is, Value& value );
For example:
ifstream is( "json.txt" );
Value value;
read( is, value );
You can also read JSON data by supplying a pair of string iterators.
bool read( std::string::const_iterator& begin,
std::string::const_iterator end, Value& value );
After a successful read, the iterator "begin" will point one past the last character of the text for the object just read. This allows the decoding of a string containing multiple top level objects. A subsequent call to read will read the next object in the string.
Similarly the stream reading functions now allow a sequence of top-level objects to be read one at a time. Before version 3.0, a stream was converted to a string before being parsed. This was fine for files, but not if for example you want to read multiple JSON values from a socket.
JSON Spirit Value
A JSON value can hold either a JSON array, JSON object, string, integer, double, bool , or null. The interface of the JSON Spirit Value class is shown below. The Value class for Unicode is analogous; for details, see the section on Unicode support.
enum Value_type{ obj_type, array_type, str_type,
bool_type, int_type, real_type, null_type };
class Value
{
public:
Value();
Value( const char* value );
Value( const std::string& value );
Value( const Object& value );
Value( const Array& value );
Value( bool value );
Value( int value );
Value_impl( boost::int64_t value );
Value_impl( boost::uint64_t value );
Value( double value );
bool operator==( const Value& lhs ) const;
Value_type type() const;
const std::string& get_str() const;
const Object& get_obj() const;
const Array& get_array() const;
bool get_bool() const;
int get_int() const;
boost::int64_t get_int64() const;
boost::uint64_t get_uint64() const;
double get_real() const;
Object& get_obj();
Array& get_array();
template< typename > T get_value() const;
bool is_uint64() const;
bool is_null() const;
static const Value null;
private:
...
};
You obtain the Value's type by calling Value::type(). You can then call the appropriate getter function. Generally, you will know a file's format, so you will know what type the JSON values should have.
The template getter function get_value() is an alternative to get_int(), get_real(), etc. Example usage would be:
int i = value_1.get_value< int >();
double d = value_2.get_value< double >();
A top level Value read from a file or stream normally contains an Array or an Object. An Array is a std::vector of values. An Object is, by default, a std::vector of JSON pairs.
typedef std::vector< Pair > Object;
typedef std::vector< Value > Array;
A Pair is a structure that holds a std::string and a Value.
struct Pair
{
Pair( const std::string& name, const Value& value );
bool operator==( const Pair& lhs ) const;
std::string name_;
Value value_;
};
JSON Arrays and Objects can themselves contain other Arrays or Objects, forming a tree.
JSON Spirit provide an alternative std::map based Object, see the Map Implementation section below.
Writing JSON
To output JSON, you first create a Value object containing your data, then, write the created Value to a stream or string. There are two versions of each function: one outputs the JSON data without any white-space, the other formats the data by adding white-space and line breaks.
void write ( const Value& value, std::ostream& os );
void write_formatted( const Value& value, std::ostream& os );
std::string write ( const Value& value );
std::string write_formatted( const Value& value );
The following example shows how to create a small JSON file containing an object with three members:
Object addr_obj;
addr_obj.push_back( Pair( "house_number", 42 ) );
addr_obj.push_back( Pair( "road", "East Street" ) );
addr_obj.push_back( Pair( "town", "Newtown" ) );
ofstream os( "address.txt" );
write_formatted( addr_obj, os );
os.close();
The object addr_obj is automatically converted into a Value as it is passed to write_formatted. The file address.txt will contain:
{
"house_number" : 42,
"road" : "East Street",
"town" : "Newtown"
}
Unicode Support
Unicode support is provided by std::wstring versions of the JSON Spirit Value, Array, Object, and Pair types. These are called wValue, wArray, wObject, and wPair. There are also std::wstring versions of each reader and writer function.
Note that there is no support for reading Unicode files and converting them to wstrings as this is not a task specific to JSON.
The Value and wValue classes are actually instantiations of the template class Value_impl.
Std::map Implementation
Before version 4.00, the JSON Spirit Object type was a std::vector of name/value Pairs. You now have the option of using mObject which is a name/value std::map. For the std::map version, use mValue instead of Value, mObject instead of Object and mArray instead of Array. For the Unicode map version, use wmValue, wmObject and wmArray.
The following table shows the times in seconds it takes on my PC to extract the data from a single object of varying sizes. The methods used are as per the demo programs. The vector version is faster until the number of object members reached around 10, but then gets exponentially slower.
| size |
vector |
map |
| 2 |
2.03253e-007 |
3.24672e-007 |
| 5 |
7.63788e-007 |
9.59516e-007 |
| 10 |
2.26948e-006 |
2.00929e-006 |
| 15 |
4.68965e-006 |
3.28509e-006 |
| 20 |
8.19667e-006 |
4.75871e-006 |
| 30 |
1.72695e-005 |
7.64189e-006 |
| 50 |
4.63171e-005 |
1.39316e-005 |
| 75 |
0.000102418 |
2.27205e-005 |
| 100 |
0.000179923 |
3.17732e-005 |
| 200 |
0.000709335 |
7.3061e-005 |
| 500 |
0.00440386 |
0.00022076 |
| 1000 |
0.0175385 |
0.000495554 |
| 10000 |
1.76533 |
0.00658268 |
| 100000 |
178.718 |
0.127325 |
| 1000000 |
17800.718 |
1.99467 |
Note with a vector object members will be written out in the same order they were read in. A map will sort members alphabetically. A vector object also allows members to have duplicate names. This might be useful in some circumstances but would be non-standard.
Error Detection
From version 3.00, JSON Spirit provides functions that report the position of format errors in text being parsed. These functions are identical to the normal read functions except that instead of returning false if an error is found, they throw a json_spirit::Error_position exception. Note, these functions run about three times slower than the normal read functions.
The Error_position structure holds the line and column number where the first error was found.
struct Error_position
{
...
unsigned int line_;
unsigned int column_;
std::string reason_;
};
Using JSON Spirit with Multiple Threads
If you intend to use JSON Spirit in more than one thread, you will need to uncomment the following line near the top of json_spirit_reader.cpp.
In this case, Boost Spirit will require you to link against Boost Threads.
History
- Version 1.00, 10th August 2007
- 12th August 2007
- Part of the Boost Spirit Applications Repository
- Version 1.01, 20th August 2007
- Fixed bug outputting escape characters
- Version 1.02, 9th October 2007
- Fixed bug inputting "\/"
- No longer attaches a semantic action
c_ecscape_ch_p, simplifying the code
- Speed optimizations
- Version 2.00, 9th November 2007
- Unicode support
- Writes out "/" without a "\" escape character
- Version 2.0, 114th December 2007
- Increased precision of floating point number output
- Version 2.02, 12th February 2008, bug fixes
- Value construction from explicit
const char*
- Writes out extended ASCII
- Version 2.03, 1st March 2008
- Added support for 64-bit integers
- Version 2.04, 22nd April 2008.
- Allows the reading of top level values other than
Arrays and Objects, see Daniel Friederich's message thread below
- Fixed bug where e.g.. "
[ 1 2 ]" is read as "[ 1, 2 ]"
- Added template getter function
get_value()
- Version 2.05, 2nd June 2008
- Linux version added
- Added commented out
#define BOOST_SPIRIT_THREADSAFE
- Version 2.06, 12th September 2008
- Now works when MSVC 2005 is using STLPort
- Version 3.00, 31st January 2009
- Multiple top-level objects can now be read from a
string or stream
- Addition of functions that take a pair of
string iterators
- Addition of functions that report the position of format errors
- Speed optimisations
- Added utility functions
obj_to_map, map_to_obj, find_value
- Version 3.01, 11th May 2009
- Added CMake files, many thanks to Uwe Arzt for supplying these
- json_spirit_writer_test.cpp now includes limits.h, needed on some platforms
- Version 4.00, 9th June 2009
- Added alternative
std::map implementation for objects
- Added support for unsigned 64 bit integers
- Allow numbers such as
3 to be read as integers or floating point numbers
- Version 4.01, 13th June 2009
- Changed source code license to the MIT License
- Stopped gcc warning about missing
switch statement
- Version 4.02, 11th September 2009
- Fixed compilation errors in the unit tests that occurred with 64 bit Linux GCC
- Changed the internals of the
Value_impl class to use boost variant, giving a speed improvement
- Small changes to prevent some pedantic warning messages
| You must Sign In to use this message board. |
|
|
 |
|
 |
Hi,
It might be nice to show how easy it is to switch between using std::vector and std::map Value implementations in your demo.
Very nice code.
-- Mark
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
 |
Hi Mark, I am pleased you like the code.
Are you saying that there should be a demo on how to convert a std::vector Value to an std::map Value, and vice versa? I am not sure that this is something that many people would want to do. I have thought that people would stick to one or the other implementation.
Regards John
PS. The following code converts a map value to the corresponding vector value.
Value mvalue_to_value( const mValue& mv );
Object mobject_to_object( const mObject& mo ) { Object o;
for( mObject::const_iterator i = mo.begin(); i != mo.end(); ++i ) { o.push_back( Pair( i->first, mvalue_to_value( i->second ) ) ); }
return o; }
Array marray_to_array( const mArray& ma ) { Array a;
for( unsigned int i = 0; i < ma.size(); ++i ) { a.push_back( mvalue_to_value( ma[i] ) ); }
return a; }
Value mvalue_to_value( const mValue& mv ) { Value v;
switch( mv.type() ) { case obj_type: v = mobject_to_object( mv.get_obj() ); break; case array_type: v = marray_to_array( mv.get_array() ); break; case str_type: v = mv.get_str(); break; case bool_type: v = mv.get_bool(); break; case int_type: v = mv.get_int(); break; case real_type: v = mv.get_real(); break; case null_type: v = Value::null; break; default: assert( false ); }
return v; }
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
 |
I wasn't sure which implementation made more sense for my situation, so I made the following trivial changes to your json_demo.cpp file:
//#define USE_MAP #ifdef USE_MAP typedef mValue Demo_Value; typedef mArray Demo_Array; typedef mObject Demo_Object; #else typedef Value Demo_Value; typedef Array Demo_Array; typedef Object Demo_Object; #endif
...
void write_address( Demo_Array& a, const Address& addr ) { Demo_Object addr_obj;
#ifdef USE_MAP addr_obj["house_number"] = addr.house_number_; addr_obj["road"] = addr.road_; addr_obj["town"] = addr.town_; addr_obj["county"] = addr.county_; addr_obj["country"] = addr.country_; #else addr_obj.push_back( json_spirit::Pair( "house_number", addr.house_number_ ) ); addr_obj.push_back( json_spirit::Pair( "road", addr.road_ ) ); addr_obj.push_back( json_spirit::Pair( "town", addr.town_ ) ); addr_obj.push_back( json_spirit::Pair( "county", addr.county_ ) ); addr_obj.push_back( json_spirit::Pair( "country", addr.country_ ) ); #endif
a.push_back( addr_obj ); }
...
Address read_address( const Demo_Object& obj ) { Address addr;
for( Demo_Object::const_iterator i = obj.begin(); i != obj.end(); ++i ) { #ifdef USE_MAP const string& name = i->first; const Demo_Value& value = i->second; #else const string& name = i->name_; const Demo_Value& value = i->value_; #endif
...
-- Mark
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
 |
I see what you mean. Putting code like the above in a demo would highlight the differences between the two methods but it might make it less clear how to use either, so I will probably leave the the two demos as they are.
Thanks for the feedback, Regards John
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
 |
The library doesn't compile under Visual C++ 2010 Beta2 without some changes. All the errors consist mostly of conflicts between new stuff in std:: and the stuff in boost::, since you are using namespace std; and using namespace boost; in one of the files. The fix consists of dropping the using namespace directives and replcaing them by usings specific for the types you're using. In addition, the binds need to be explicitly qualified because it still conflicts with std due to ATL on the placeholders I believe. Should be simple fixes.
Thank you for writing this great, reliable and readable library. =)
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
 |
Thanks for the information. I suppose the reason for the errors is that some of the boost libraries have become C++ standard libraries. I will download a copy of VC++ 2010 beta and make the required changes.
Thanks John
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
 |
Hi,
at first i would like to thank you for your work with this parser. It is really nice.
However, i have some Questions which you could hopefully answer.
If i have a json Object and want to change a deeper Property directly, it is hard work.
Example json-Object:
[ {}, { 'first': { 'firstfirst': { 'firstfirstfirst': { 'property' : 'abc' } } } }, {} ]
Currently i have to do something like this, if i want to change the value of property:
json_spirit::mValue v; json_spirit::read("[{},{'first':{'firstfirst':{'firstfirstfirst':{'property':'abc'}}}},{}]",v); json_spirit::mObject f, ff, fff; json_spirit::mArray a; std::string s, s2="def";
a = v.get_array(); f = a.at(1).get_obj(); ff = f["firstfirst"].get_obj(); fff = ff["firstfirstfirst"].get_obj(); if (fff["property"].get_str() != s2) { fff["property"] = s2; } ff["firstfirstfirst"] = fff; f["firstfirst"] = ff; a[1] = f;
s = json_spirit::write_formatted(f);
It would be nice if i could get a pointer or a reference to fff, so i could change it directly so i do not have to put everything back.
Or is there already a chance to change a Value or Object directly?
Thanks, Jens.
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
 |
The Value_impl class has non-const get_obj and get_array method that return references allowing direct access. You just need to change your object and array variables to be references.
Cheers John
|
| Sign In·View Thread·PermaLink | 5.00/5 |
|
|
|
 |
|
|
 |
|
 |
Just to clarify here is your example modified to do what you want:
json_spirit::mValue v; json_spirit::read("[{},{\"first\":{\"firstfirst\":{\"firstfirstfirst\":{\"property\":\"abc\"}}}},{}]",v);
cout << json_spirit::write_formatted(v);
json_spirit::mArray& a = v.get_array(); json_spirit::mObject& f = a.at(1).get_obj(); json_spirit::mObject& ff = f["first"].get_obj(); json_spirit::mObject& fff = ff["firstfirst"].get_obj(); json_spirit::mObject& ffff = fff["firstfirstfirst"].get_obj();
ffff["property"] = "def";
cout << json_spirit::write_formatted(v);
Regards John
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
 |
I'm currently working with the 2.06 codebase and don't want to go through the pain of upgrading to the latest just yet. The issue I have is I can't specify information as hex. e.g.:
{ "Id": \u0004 }
Json.org implies this is legal but when I try to pass it to JSON Spirit I get Invalid JSON Structure. Can you do a quick test and see if that is still the result in 4.02?
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
 |
I bit the bullet and upgraded to 4.02 (SVN had my back). Upgrade was painless but my problem still remains. I can't submit hex values.
FYI, json_spirit/json_spirit builds in Fedora 11 under g++ (GCC) 4.3.2 20081105 (Red Hat 4.3.2-7)
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
 |
I don't think that { "Id": \u0004 } is valid JSON. I read the spec to say that numbers are either decimal integers or floats, see http://www.json.org/ and the RFC. The hex notation can only be used inside strings for characters.
I wouldn't be adverse to adding this as an extension especially if it is required to read JSON produced by other programs.
Regards John
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
 |
You are correct. I didn't look at the spec closely enough. Hex is not an allowed number type. Hex is related to string usage. And more specifically a multibyte character. Thanks.
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
 |
Is it possible to get this library wrapped in a dll ( or just a project to generate a dll)? The library is quite large and getting bigger for static linking.
Thanks.
habiria
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
 |
Thanks for your message. Are you referring to the size of the .lib file, i.e. 69M? This is definitely large but the resultant exes when linking to it statically are small, e.g. 120K for the demo program. I don't see what benefit's a DLL would give. The DLL would be the size of the .lib and the exes using the DLL would only be slightly smaller than if they were linked to the library.
Regards John
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
 |
Thanks John for your nice lib and more so for the regular updates and support you give us. 
Since JSON is a subset of YAML, could your lib be extended to support YAML 1.2 or maybe as a seperate lib? Would that be a big effort for you? I would need YAML but I'm not proficient enough to code it in Spirit or ANTLR without help.
Thanks anyways, pat 
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
 |
|
 |
I understand. Maybe you could then extend your article with how you created the parsing part in Spirit. How you got from EBNF to working code. That would be most helpful to me.
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
 |
I don't think I could add anything that has not already been said in existing Spirit tutorials. There are a number on the Code Project that might be helpful.
Regards John
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
 |
Redundancy has never been an issue on Codeproject. And new versions of Spirit are not always immediatly covered. Few authors make regular updates of their articles. And most of all I would appreciate to see a different approach with a clear workflow as described above. So in fact there are many good reasons to write about it... if you want. 
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
 |
There is not much to say. I developed the code incrementally starting with just parsing data without any semantic actions, i.e. without building up any data structures. I think I first got JSON arrays to work before moving onto Objects. The first version only worked with ASCII, not Unicode, which simplified things, fewer templates.
Throughout he development I maintained a comprehensive set of unit tests. This allowed me to continually refactor the code to keep it clean. The tests are also a great help for regression testing to ensure I didn't break anything when adding new features.
I coded an odd hour here and there which helped with the design because I had a long time to think how to solve problems that arose.
Cheers John
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
 |
Hello,
Just a simple question, is possible to change the output to xml format (to read and write in a valid xml format)? If yes then how this support the attributes and so on?
Thanks
Nuno
|
| Sign In·View Thread·PermaLink | |
|
|
|
 |
|
|
 |
|
 |
This library uses Classic Spirit for note. Spirit2.1 (currently in Boost Trunk and will be in Boost 1.41) is fully complete (they are just finishing documentation and building up a useful set of components). Spirit2.1 is a *great* deal faster then Classic Spirit.
I have already created a completely header file based json parser in Spirit2.1 (that I intend to add to the Spirit2.1 external components section) that vastly outperforms anything that can be created in Classic Spirit. Value in my version is just a Boost.Variant with all the normal Boost.Variant power (can directly pull out a value as with the article in this version, as well as able to to compile time visitation on it for absolute speed), and it takes a *lot* less memory use then the Value class in this article (why did the article writer not use boost::variant?). It uses boost::unordered_map for the map object (since the json standard specifies that it is an unordered map, seems the most logical to use since it is so efficient), int64 (based on the JSON standard), double (based on the JSON standard), etc... I tried to base everything on the JSON standard as much as possible. I am waiting to commit it until I write the karma version (Spirit.Karma in Spirit2.1 is the output version of the normal Spirit.QI input, so it will be able to output as well as input). You can expect it in Boost Trunk when I get around to finishing the karma part, or you can wait until Boost 1.41 when it is fully released. I have already released the parser part (not the karma part yet) on the Spirit mailing list a while back, but I am cleaning up the interface as well. It is also input agnostic, as long as the iterator is a back-tracking iterator it will store the characters in whatever format it is, so if you give it an std::string iterator, you get you a JSON Value using char's to store string, or if you use unicode, you will get whatever the embedded char type is, etc... and so forth. Only thing you do is set a macro to the encoding type you will use (ascii, wide, etc...) as-is supported by Spirit2.1 and it will use whatever you give it to parse, much more powerful. And, as stated, it is header-only, so no needing to include any cpp files in your project (and when it is added to the Spirit components then you will just include like any normal part of spirit anyway).
|
| Sign In·View Thread·PermaLink | 1.00/5 |
|
|
|
 |
|
|