Reading JSON File with C++ and Boost

Read & Parse a JSON file c++ BOOST

I'm not completely sure what the problem is. It seems to work (once you make the JSON valid):

UPDATE: Boost JSON

Boost 1.75.0 introduced Boost JSON, far superior way to actually deal with Json: Live On Wandbox

#include <boost/json.hpp>
#include <boost/json/src.hpp>
#include <iostream>
#include <iterator>
#include <fstream>
namespace json = boost::json;

struct Rec {
    int64_t number;
    std::string string;

    friend Rec tag_invoke(json::value_to_tag<Rec>, json::value const& v) {
        auto& o = v.as_object();
        return {
            o.at("number").as_int64(),
            value_to<std::string>(o.at("string")),
        };
    }

    friend void tag_invoke(json::value_from_tag, json::value& v, Rec const& rec)
    {
        v = json::object{
            {"number", rec.number},
            {"string", rec.string},
        };
    }
};

int main() {
    std::ifstream ifs("input.txt");
    std::string   input(std::istreambuf_iterator<char>(ifs), {});

    using Recs = std::vector<Rec>;
    Recs recs  = value_to<std::vector<Rec>>(json::parse(input));

    for (auto& [n, s] : recs) {
        std::cout << "Rec { " << n << ", " << std::quoted(s) << " }\n";

        // some frivolous changes:
        n *= 2;
        reverse(begin(s), end(s));
    }

    std::cout << "Modified json: " << json::value_from(recs) << "\n";
}

Printing

Rec { 1234, "hello world" }
Rec { 5678, "foo bar" }
Modified json: [{"number":2468,"string":"dlrow olleh"},{"number":11356,"string":"rab oof"}]

Live On Coliru

#include "boost/property_tree/ptree.hpp"
#include "boost/property_tree/json_parser.hpp"

int main() {
    using boost::property_tree::ptree;

    std::ifstream jsonFile("input.txt");

    ptree pt;
    read_json(jsonFile, pt);

    for (auto & array_element: pt) {
        for (auto & property: array_element.second) {
            std::cout << property.first << " = " << property.second.get_value < std::string > () << "\n";
        }
    }
}

With input.txt containing:

[{"number": 1234, "string": "hello world"},{"number": 5678, "string": "foo bar"}]

Prints

number = 1234
string = hello world
number = 5678
string = foo bar

Reading JSON file with C++ and BOOST

Because the data structure in the other answer was deemed "very complex" and the target data structure was suggested to be:

struct Data {
    struct Folder { int id, parent_id; std::string path; };
    struct File   { int id, parent_id; std::string path, name, md5_hash; };

    using Folders = std::vector<Folder>;
    using Files   = std::vector<File>;

    Folders folders;
    Files   files;
};

I ended up writing a transformation from generic "JSON" to that data structure (see the other answer: Reading JSON file with C++ and BOOST).

However, perhaps the OP will be more pleased if we "skip the middle man" and parse the JSON specifically into the shown Data structure. This "simplifies" the grammar making it specific for this type of document only:

start    = '{' >> 
           (folders_ >> commasep) ^
           (files_ >> commasep)
         >> '}';

folders_ = prop_key(+"folders") >> '[' >> -(folder_ % ',') >> ']';
files_   = prop_key(+"files")   >> '[' >> -(file_   % ',') >> ']';

folder_  = '{' >> (
                (prop_key(+"id")        >> int_  >> commasep) ^
                (prop_key(+"parent_id") >> int_  >> commasep) ^
                (prop_key(+"path")      >> text_ >> commasep)
            ) >> '}';
file_    = '{' >> (
                (prop_key(+"id")        >> int_  >> commasep) ^
                (prop_key(+"parent_id") >> int_  >> commasep) ^
                (prop_key(+"path")      >> text_ >> commasep) ^
                (prop_key(+"name")      >> text_ >> commasep) ^
                (prop_key(+"hash")      >> text_ >> commasep)
            ) >> '}';

prop_key = lexeme ['"' >> lazy(_r1) >> '"'] >> ':';
commasep = &char_('}') | ',';

This grammar allows

  • insignificant whitespace,
  • re-ordering of properties within objects
  • and omitted object properties

Benefits:

  • early checking of property value types
  • lower compile times
  • less code indeed: 37 fewer LoC (not counting the sample JSON lines that's ~22%)

That last benefit has a flip side: if ever you want to read slightly different JSON, now you need to muck with the grammar instead of just writing a different extraction/transform. At 37 lines of code, my preference is with the other answer but I'll leave it to you to decide.

Here's the same demo program using this grammar directly:

Live On Coliru

//#define BOOST_SPIRIT_DEBUG
#include <boost/fusion/adapted.hpp>
#include <boost/spirit/include/qi.hpp>
#include <boost/spirit/include/phoenix.hpp>

namespace qi = boost::spirit::qi;

static std::string const sample = R"(
    {
        "folders" :
        [{
                "id" : 109,
                "parent_id" : 110,
                "path" : "\/1\/105\/110\/"
            },
            {
                "id" : 110,
                "parent_id" : 105,
                "path" : "\/1\/105\/"
            }
        ],

        "files" :
        [{
                "id" : 26,
                "parent_id" : 105,
                "name" : "picture.png",
                "hash" : "md5_hash",
                "path" : "\/1\/105\/"
            },
            {
                "id" : 25,
                "parent_id" : 110,
                "name" : "another_picture.jpg",
                "hash" : "md5_hash",
                "path" : "\/1\/105\/110\/"
            }
        ]
    })";

struct Data {
    struct Folder { int id, parent_id; std::string path; };
    struct File   { int id, parent_id; std::string path, name, md5_hash; };

    using Folders = std::vector<Folder>;
    using Files   = std::vector<File>;

    Folders folders;
    Files   files;
};

BOOST_FUSION_ADAPT_STRUCT(Data::Folder, (int,id)(int,parent_id)(std::string,path))
BOOST_FUSION_ADAPT_STRUCT(Data::File,   (int,id)(int,parent_id)(std::string,path)(std::string,name)(std::string,md5_hash))
BOOST_FUSION_ADAPT_STRUCT(Data,         (Data::Folders,folders)(Data::Files,files))

namespace folder_info { // adhoc JSON parser

    template <typename It, typename Skipper = qi::space_type>
    struct grammar : qi::grammar<It, Data(), Skipper>
    {
        grammar() : grammar::base_type(start) {
            using namespace qi;

            start    = '{' >> 
                       (folders_ >> commasep) ^
                       (files_ >> commasep)
                     >> '}';

            folders_ = prop_key(+"folders") >> '[' >> -(folder_ % ',') >> ']';
            files_   = prop_key(+"files")   >> '[' >> -(file_   % ',') >> ']';

            folder_  = '{' >> (
                            (prop_key(+"id")        >> int_  >> commasep) ^
                            (prop_key(+"parent_id") >> int_  >> commasep) ^
                            (prop_key(+"path")      >> text_ >> commasep)
                        ) >> '}';
            file_    = '{' >> (
                            (prop_key(+"id")        >> int_  >> commasep) ^
                            (prop_key(+"parent_id") >> int_  >> commasep) ^
                            (prop_key(+"path")      >> text_ >> commasep) ^
                            (prop_key(+"name")      >> text_ >> commasep) ^
                            (prop_key(+"hash")      >> text_ >> commasep)
                        ) >> '}';

            prop_key = lexeme ['"' >> lazy(_r1) >> '"'] >> ':';
            commasep = &char_('}') | ',';

            ////////////////////////////////////////
            // Bonus: properly decoding the string:
            text_   = '"' >> *ch_ >> '"';

            ch_ = +(
                    ~char_("\"\\")) [ _val += _1 ] |
                       qi::lit("\x5C") >> (               // \ (reverse solidus)
                       qi::lit("\x22") [ _val += '"'  ] | // "    quotation mark  U+0022
                       qi::lit("\x5C") [ _val += '\\' ] | // \    reverse solidus U+005C
                       qi::lit("\x2F") [ _val += '/'  ] | // /    solidus         U+002F
                       qi::lit("\x62") [ _val += '\b' ] | // b    backspace       U+0008
                       qi::lit("\x66") [ _val += '\f' ] | // f    form feed       U+000C
                       qi::lit("\x6E") [ _val += '\n' ] | // n    line feed       U+000A
                       qi::lit("\x72") [ _val += '\r' ] | // r    carriage return U+000D
                       qi::lit("\x74") [ _val += '\t' ] | // t    tab             U+0009
                       qi::lit("\x75")                    // uXXXX                U+XXXX
                            >> _4HEXDIG [ append_utf8(qi::_val, qi::_1) ]
                    );

            BOOST_SPIRIT_DEBUG_NODES((files_)(folders_)(file_)(folder_)(start)(text_))
        }
    private:
        qi::rule<It, Data(),            Skipper> start;
        qi::rule<It, Data::Files(),     Skipper> files_;
        qi::rule<It, Data::Folders(),   Skipper> folders_;
        qi::rule<It, Data::File(),      Skipper> file_;
        qi::rule<It, Data::Folder(),    Skipper> folder_;
        qi::rule<It, void(const char*), Skipper> prop_key;

        qi::rule<It, std::string()> text_, ch_;
        qi::rule<It> commasep;

        struct append_utf8_f {
            template <typename...> struct result { typedef void type; };
            template <typename String, typename Codepoint>
            void operator()(String& to, Codepoint codepoint) const {
                auto out = std::back_inserter(to);
                boost::utf8_output_iterator<decltype(out)> convert(out);
                *convert++ = codepoint;
            }
        };
        boost::phoenix::function<append_utf8_f> append_utf8;
        qi::uint_parser<uint32_t, 16, 4, 4> _4HEXDIG;
    };

    template <typename Range, typename It = typename boost::range_iterator<Range const>::type>
    Data parse(Range const& input) {
        grammar<It> g;

        It first(boost::begin(input)), last(boost::end(input));
        Data parsed;
        bool ok = qi::phrase_parse(first, last, g, qi::space, parsed);

        if (ok && (first == last))
            return parsed;

        throw std::runtime_error("Remaining unparsed: '" + std::string(first, last) + "'");
    }
}

int main()
{
    auto parsed = folder_info::parse(sample);

    for (auto& e : parsed.folders) 
        std::cout << "folder:\t" << e.id << "\t" << e.path << "\n";
    for (auto& e : parsed.files) 
        std::cout << "file:\t"   << e.id << "\t" << e.path << "\t" << e.name << "\n";
}

Output:

folder: 109 /1/105/110/
folder: 110 /1/105/
file:   26  /1/105/ picture.png
file:   25  /1/105/110/ another_picture.jpg

boost: parse json file and get a child

Use a JSON library. Property Tree is not a JSON library. Using Boost JSON and JSON Pointer:

Live On Coliru

#include <boost/json.hpp>
#include <boost/json/src.hpp> // for header-only
#include <iostream>
#include <string_view>

auto getField(std::string_view json, std::string_view field) {
  return boost::json::parse(json).at_pointer(field);
}

int main() {
  auto output = R"({
    "data": {
        "device_id": 67,
        "place_open": {
            "monday": ["10:15", "19:30"]
        }
    }
})";

  for (auto pointer : {
           "/data",
           "/data/device_id",
           "/data/place_open/monday",
           "/data/place_open/monday/0",
           "/data/place_open/monday/1",
       })
    std::cout << pointer << " -> " << getField(output, pointer) << std::endl;
}

Prints

/data -> {"device_id":67,"place_open":{"monday":["10:15","19:30"]}}
/data/device_id -> 67
/data/place_open/monday -> ["10:15","19:30"]
/data/place_open/monday/0 -> "10:15"
/data/place_open/monday/1 -> "19:30"

Read a sub json using boost property tree

The body is not a string.

So, getting the child object would be in order:

for (auto const& v : jsonBatchResponse.get_child("responses")) {
    std::string  strID     = v.second.get<std::string>("id", "");
    std::string  strStatus = v.second.get<std::string>("status", "");
    ptree const& body      = v.second.get_child("body");
}

If you add some output to that loop with e.g.

    std::cout << std::quoted(strID) << "\n";
    std::cout << std::quoted(strStatus) << "\n";
    write_json(std::cout, body);

It will print Live On Coliru

"1"
"200"
{
    "createdDateTime": "2021-04-22T09:24:59.394Z",
    "displayName": "Test1",
    "visibility": "public",
    "isMembershipLimitedToOwners": "false",
    "discoverySettings": {
        "showInTeamsSearchAndSuggestions": "true"
    },
    "memberSettings": {
        "allowCreateUpdateChannels": "true",
        "allowCreateUpdateRemoveConnectors": "true"
    },
    "guestSettings": {
        "allowCreateUpdateChannels": "true",
        "allowDeleteChannels": "false"
    },
    "messagingSettings": {
        "allowUserEditMessages": "true",
        "allowChannelMentions": "true"
    },
    "funSettings": {
        "allowGiphy": "true",
        "allowCustomMemes": "true"
    }
}

BONUS: Using a proper JSON library instead

Boost Property Tree is NOT a JSON library, and therefore has a lot of limitations.

Instead I suggest using Boost JSON:

Live On Coliru

#include <boost/json.hpp>
#include <boost/json/src.hpp>
#include <iostream>

namespace json = boost::json;

extern std::string sample;

int main() {
    json::object jsonBatchResponse = json::parse(sample).as_object();

    for (auto& v : jsonBatchResponse["responses"].as_array()) {
        auto& res = v.as_object();
        json::value id  = res["id"],     // string
            status      = res["status"], // integer
            body        = res["body"];   // object
        std::cout << id << "\n";
        std::cout << status << "\n";
        std::cout << body << "\n";
    }
}

std::string sample = R"(
{
    "responses": [{
        "id": "1",
        "status": 200,
        "headers": {
            "OData-Version": "4.0",
            "Content-Type": "application/json;odata.metadata=minimal;odata.streaming=true"
        },
        "body": {
            "createdDateTime": "2021-04-22T09:24:59.394Z",
            "displayName": "Test1",
            "visibility": "public",
            "isMembershipLimitedToOwners": false,
            "discoverySettings": {
                "showInTeamsSearchAndSuggestions": true
            },
            "memberSettings": {
                "allowCreateUpdateChannels": true,
                "allowCreateUpdateRemoveConnectors": true
            },
            "guestSettings": {
                "allowCreateUpdateChannels": true,
                "allowDeleteChannels": false
            },
            "messagingSettings": {
                "allowUserEditMessages": true,
                "allowChannelMentions": true
            },
            "funSettings": {
                "allowGiphy": true,
                "allowCustomMemes": true
            }
        }
    }]
}
)";

Prints

"1"
200
{"createdDateTime":"2021-04-22T09:24:59.394Z","displayName":"Test1","visibility":"public","isMembershipLimitedToOwners":false,"discoverySettings":{"showInTeamsSearchAndSuggestions":true},"memberSettings":{"allowCreateUpdateChannels":true,"allowCreateUpdateRemoveConnectors":true},"guestSettings":{"allowCreateUpdateChannels":true,"allowDeleteChannels":false},"messagingSettings":{"allowUserEditMessages":true,"allowChannelMentions":true},"funSettings":{"allowGiphy":true,"allowCustomMemes":true}}

CMake, JNI boost read json file - Android

I used this code to get it working, basically you copy the file from Android raw dir to some location on the device that the C++ JNI code can see.

MainActivity code:

private static final String RES_RAW_CONFIG_PATH_ENV_VAR = "RES_RAW_CONFIG_PATH";
private static final String RES_RAW_CONFIG_FILE_NAME = "res_raw_config.json";

@Override
protected void onCreate(Bundle savedInstanceState) {
    initResRawConfig(true); // this will copy the file
    ...
}

private boolean existsInFilesDir(String fileName) {
    val file = File(filesDir, fileName)
    return file.exists()
}

private void initResRawConfig(boolean forceCopy) {
    if (existsInFilesDir(RES_RAW_CONFIG_FILE_NAME) && !forceCopy) {
        Log.d(TAG, "Config file: " + RES_RAW_CONFIG_FILE_NAME + " already exists in " + getFilesDir().toString());
    }
    else {
        copyFileFromResToFilesDir(R.raw.res_raw_config, RES_RAW_CONFIG_FILE_NAME);
        Log.d(TAG, "Config file: " + RES_RAW_CONFIG_FILE_NAME + " copied to " + getFilesDir().toString());
    }

    // Set Environment Variable to get path from native
    try {
        Os.setenv(RES_RAW_CONFIG_PATH_ENV_VAR, getFilesDir().getAbsolutePath(), true);
    } catch (ErrnoException e) {
        e.printStackTrace();
    }
}

private void copyFileFromResToFilesDir(int fromResId, String toFile) {
    InputStream is =  getResources().openRawResource(fromResId);
    byte[] buffer = new byte[4096];
    try
    {
        FileOutputStream fos = openFileOutput(toFile, Context.MODE_PRIVATE);

        int bytes_read;
        while ((bytes_read = is.read(buffer)) != -1)
            fos.write(buffer, 0, bytes_read);

        fos.close();
        is.close();
    }
    catch (FileNotFoundException e)
    {
        e.printStackTrace();
    }
    catch (IOException e)
    {
        e.printStackTrace();
    }
}

public String getResRawConfigDir()
{
    return getFilesDir().toString();
}

JNI code:

// get file path from java method
std::string getResRawConfigDirFromJava(JNIEnv *env, jobject obj) {
    jclass clazz = env->GetObjectClass(obj); // or env->FindClass("com/example/myapp/MainActivity");
    jmethodID method = env->GetMethodID(clazz, "getResRawConfigDir", "()Ljava/lang/String;");
    jobject ret = env->CallObjectMethod(obj, method);

    auto jConfigDirPath = (jstring) ret;

    return std::string(env->GetStringUTFChars(jConfigDirPath, nullptr));
}

extern "C" JNIEXPORT jstring
sampleFunc(JNIEnv *env, jobject thiz) {
    std::string configPath = getResRawConfigDirFromJava(env, thiz);
    std::string filePath = configPath + "/" + "res_raw_config.json";
}

Ref: https://github.com/nkh-lab/ndk-config-provider/blob/master/app/src/main/java/com/example/myapp/MainActivity.java

Reading json files in C++

  1. Yes you can create a nested data structure people which can be indexed by Anna and Ben. However, you can't index it directly by age and profession (I will get to this part in the code).

  2. The data type of people is of type Json::Value (which is defined in jsoncpp). You are right, it is similar to the nested map, but Value is a data structure which is defined such that multiple types can be stored and accessed. It is similar to a map with a string as the key and Json::Value as the value. It could also be a map between an unsigned int as key and Json::Value as the value (In case of json arrays).

Here's the code:

//if include <json/value.h> line fails (latest kernels), try also:
//  #include <jsoncpp/json/json.h>
#include <json/value.h>
#include <fstream>

std::ifstream people_file("people.json", std::ifstream::binary);
Json::Value people;
people_file >> people;

cout<<people; //This will print the entire json object.

//The following lines will let you access the indexed objects.
cout<<people["Anna"]; //Prints the value for "Anna"
cout<<people["ben"]; //Prints the value for "Ben"
cout<<people["Anna"]["profession"]; //Prints the value corresponding to "profession" in the json for "Anna"

cout<<people["profession"]; //NULL! There is no element with key "profession". Hence a new empty element will be created.

As you can see, you can index the json object only based on the hierarchy of the input data.


Related Topics

Conversion from Int** to Const Int**

Does Copy List Initialization Invoke Copy Ctor Conceptually

Efficiently Convert Between Hex, Binary, and Decimal in C/C++

Glut Deprecation in MAC Osx 10.9, Ide: Qt Creator

Is It Legal/Well-Defined C++ to Call a Non-Static Method That Doesn't Access Members Through a Null Pointer

Netbeans and Mingw-W64

Append Int to Std::String

Why "Not All Control Paths Return a Value" Is Warning and Not an Error

How to Compare Strings

Auto' as a Template Argument Placeholder for a Function Parameter

Lambdas and Capture by Reference Local Variables:Accessing After the Scope

Popen Equivalent in C++

Ramifications of C++20 Requiring Two's Complement

Function Already Defined Error in C++

Lifetime of Object Is Over Before Destructor Is Called

Correct Format for Http Post Using Qnetworkrequest

Recursion in C++ Factorial Program

Numpy Style Arrays for C++


Leave a reply



Submit