Copy a Std::Vector to a Repeated Field from Protobuf with Memcpy

Copy a std::vector to a repeated field from protobuf with memcpy

Since this isn't here yet and I like one-liners:

*fMessage.mutable_samples() = {fData.begin(), fData.end()};

Protobuf Partially Copy vector into repeated filed

I created a program to test this, and it seems that using std::copy works!

syntax = "proto3";

message messagetest{
    repeated float samples = 6;
}

#include <iostream>
#include <vector>

#include "message.pb.h"

int main(){
  std::vector<float> fData(10);
  messagetest fMessage;
  std::generate(fData.begin(),fData.end(),[&fMessage](){
    static float num = 0.0;
    num += 1.0;
    fMessage.add_samples(0.0);
    return num;
  });
  for(const float& f : fData)
     std::cout << "[" << f << "]";
  std::cout << std::endl;
  
  std::copy(
    fData.begin() + 3, fData.end() - 2,
    fMessage.mutable_samples()->begin() + 3
  );
  
  for(const float& f : fMessage.samples())
     std::cout << "[" << f << "]";
  std::cout << std::endl;
  
  
  return 0;
}

output:

[1][2][3][4][5][6][7][8][9][10]
[0][0][0][4][5][6][7][8][0][0]

How to add vector to repeated field protobuf c++

You have to iterate of the given vector and add the objects to your protobuf message manually. You can not use a memcpy operation for that.

Following code is written out of my mind without testing ... but should point you in the correct direction. btw: I assume Journey inherits from gen_Journey in this case. Otherwise you have to adjust the "this->" statement accordingly.

Journey::Journey(const std::vector<gen_ProposedSegment *> &proposedSegment) {
    auto copy = [&](const gen_ProposedSegment *) {
        auto temp_seg = this->add_proposedsegments();
        temp_seg->CopyFrom(*gen_ProposedSegment);
    };
    std::for_each(proposedSegment.cbegin(), proposedSegment.cend(), copy);
}

How to append multiple elements to a repeatedField in Protocol buffers?

I don't think your sample code would build. My_msg::mutable_my_arr() returns a pointer to a RepeatedField (not the first element of an array). Trying to index it would segfault at best.

In terms or performance, if you have your data in an std::vector you will always need to copy - so you could just try to make that faster.

You can call RepeatedField::Reserve before. Then you can either write a loop, or use RepeatedFieldBackInserter:

void set_data(const std::vector<double>& table, My_msg* message){
  message->mutable_my_arr()->Reserve(message->my_arr_size() + table.size());
  std::copy(
      table.begin(),
      table.end(), 
      RepeatedFieldBackInserter(message->mutable_my_arr()));
}

Type conversion to vector in C++

That should be easy, since repeated fields act as containers:

void foo(RepeatedField<google::protobuf::uint32> const & f)
{
    std::vector<double> v(f.begin(), f.end());

    // use v
}

Nested protobuf structure with repeated field leads to debug assertion failure

I finally figured out what was the problem, so for everybody who has a similar problem:

1 For the first described heap problems make sure you use the correct libraries for the appropriate configuration (debug/ release). I used libprotobuf.lib/dll instead of libprotobufd.lib/dll.

2 For the problem with the string passed to the ParseFromString() method: I had _ITERATOR_DEBUG_LEVEL=0 set in the program's Preprocessor Definitions. This was incompatible with the dll generated by vcpkg. To make it compatible you have to include _ITERATOR_DEBUG_LEVEL=0 to the compilation of the protobuflib. You can do this by adding

set(VCPKG_C_FLAGS_DEBUG "/D_ITERATOR_DEBUG_LEVEL=0")
set(VCPKG_CXX_FLAGS_DEBUG "/D_ITERATOR_DEBUG_LEVEL=0")

to the cmake file used for compiling the proto library in vcpkg. E.g. C:\Program Files\vcpkg\triplets\x86-windows.cmake if invoking vcpkg install protobuf protobuf:x86-windows (best to create a new cmake file and invoke it after the colon in the install command). Then use the newly created lib/dll files in your project.

std::memcpy and copying data from 2D vector

It's not possible to use a single std::memcpy to go from a flattened matrix (int[]) to a proper matrix (std::vector<std::vector<int>>). For two reasons. First, there is no way to tell std::memcpy the matrix dimensions. Second, the 'rows' in the std::vector matrix are not required to be sequential in memory, but std::memcpy only copies to a single address.

---

Related Topics