Fails to build with newer protobuf library

[russss]

(macOS 13.4, amd64, Apple clang version 14.0.3)

I have Protobuf 23.3 installed here (through Homebrew). Running make fails with the following error due to missing std symbols:

In file included from include/osmformat.pb.cc:4:
In file included from ./include/osmformat.pb.h:24:
In file included from /usr/local/include/google/protobuf/io/coded_stream.h:134:
In file included from /usr/local/include/absl/strings/cord.h:78:
In file included from /usr/local/include/absl/container/inlined_vector.h:53:
In file included from /usr/local/include/absl/container/internal/inlined_vector.h:30:
In file included from /usr/local/include/absl/container/internal/compressed_tuple.h:40:
/usr/local/include/absl/utility/utility.h:164:12: error: no member named 'in_place_t' in namespace 'std'
using std::in_place_t;
      ~~~~~^
/usr/local/include/absl/utility/utility.h:165:12: error: no member named 'in_place' in namespace 'std'
using std::in_place;
      ~~~~~^
/usr/local/include/absl/utility/utility.h:181:12: error: no member named 'in_place_type' in namespace 'std'
using std::in_place_type;
      ~~~~~^
/usr/local/include/absl/utility/utility.h:182:12: error: no member named 'in_place_type_t' in namespace 'std'
using std::in_place_type_t;
      ~~~~~^
/usr/local/include/absl/utility/utility.h:198:12: error: no member named 'in_place_index' in namespace 'std'
using std::in_place_index;
      ~~~~~^
/usr/local/include/absl/utility/utility.h:199:12: error: no member named 'in_place_index_t' in namespace 'std'
using std::in_place_index_t;
      ~~~~~^

Changing -std=c++14 to -std=c++17 in the Makefile fixes this and it all compiles fine, but it fails with this linker error:

c++ -O3 -Wall -Wno-unknown-pragmas -Wno-sign-compare -std=c++17 -pthread -fPIE -DTM_VERSION=v2.4.0  -o tilemaker include/osmformat.pb.o include/vector_tile.pb.o src/mbtiles.o src/pbf_blocks.o src/coordinates.o src/osm_store.o src/helpers.o src/output_object.o src/read_shp.o src/read_pbf.o src/osm_lua_processing.o src/write_geometry.o src/shared_data.o src/tile_worker.o src/tile_data.o src/osm_mem_tiles.o src/shp_mem_tiles.o src/attribute_store.o src/tilemaker.o src/geom.o -I/usr/local/include -isystem ./include -I./src -I/usr/local/include/luajit-2.1 -DLUAJIT -L/usr/local/lib -lz -lluajit-5.1 -lboost_program_options -lsqlite3 -lboost_filesystem -lboost_system -lboost_iostreams -lprotobuf -lshp -pthread -pagezero_size 10000 -image_base 100000000
ld: warning: Linking with PIE, -image_base will be ignored
ld: warning: non-standard -pagezero_size is deprecated when targeting macOS 13.0 or later
Undefined symbols for architecture x86_64:
  "absl::lts_20230125::log_internal::LogMessageFatal::LogMessageFatal(char const*, int, std::__1::basic_string_view<char, std::__1::char_traits<char>>)", referenced from:
      BlobHeader::~BlobHeader() in osmformat.pb.o
      Blob::~Blob() in osmformat.pb.o
      HeaderBlock::HeaderBlock(HeaderBlock const&) in osmformat.pb.o
      HeaderBlock::SharedDtor() in osmformat.pb.o
      HeaderBBox::~HeaderBBox() in osmformat.pb.o
      HeaderBBox::~HeaderBBox() in osmformat.pb.o
      HeaderBBox::~HeaderBBox() in osmformat.pb.o
      ...
  "absl::lts_20230125::log_internal::LogMessageFatal::~LogMessageFatal()", referenced from:
      BlobHeader::~BlobHeader() in osmformat.pb.o
      Blob::~Blob() in osmformat.pb.o
      HeaderBlock::HeaderBlock(HeaderBlock const&) in osmformat.pb.o
      HeaderBlock::SharedDtor() in osmformat.pb.o
      HeaderBBox::~HeaderBBox() in osmformat.pb.o
      HeaderBBox::~HeaderBBox() in osmformat.pb.o
      HeaderBBox::~HeaderBBox() in osmformat.pb.o
      ...
  "std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>* absl::lts_20230125::log_internal::MakeCheckOpString<void const*, void const*>(void const*, void const*, char const*)", referenced from:
      HeaderBlock::HeaderBlock(HeaderBlock const&) in osmformat.pb.o
      PrimitiveBlock::PrimitiveBlock(PrimitiveBlock const&) in osmformat.pb.o
      PrimitiveGroup::PrimitiveGroup(PrimitiveGroup const&) in osmformat.pb.o
      StringTable::StringTable(StringTable const&) in osmformat.pb.o
      BlobHeader::MergeImpl(google::protobuf::Message&, google::protobuf::Message const&) (.cold.1) in osmformat.pb.o
      Blob::MergeImpl(google::protobuf::Message&, google::protobuf::Message const&) (.cold.1) in osmformat.pb.o
      HeaderBlock::MergeImpl(google::protobuf::Message&, google::protobuf::Message const&) (.cold.1) in osmformat.pb.o
      ...
  "std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char>>* absl::lts_20230125::log_internal::MakeCheckOpString<long long, long long>(long long, long long, char const*)", referenced from:
      DenseInfo::DenseInfo(DenseInfo const&) in osmformat.pb.o
      ChangeSet::ChangeSet(ChangeSet const&) in osmformat.pb.o
      Node::Node(Node const&) in osmformat.pb.o
      DenseNodes::DenseNodes(DenseNodes const&) in osmformat.pb.o
      Way::Way(Way const&) in osmformat.pb.o
      Relation::Relation(Relation const&) in osmformat.pb.o
      google::protobuf::RepeatedField<int>::GrowNoAnnotate(int, int) in osmformat.pb.o
      ...
ld: symbol(s) not found for architecture x86_64
clang: error: linker command failed with exit code 1 (use -v to see invocation)

Looks like they've added a dependency on Abseil (presumably as at protobuf version 23).

Appending the (copious) output of pkg-config --libs protobuf to the LIB line in the Makefile results in the linker succeeding and Tilemaker appears to work.

[systemed]

Oh, ugh. Maybe this should be a prompt to move to protozero…

[pnorman]

Or use a library for at least reading OSM files that abstracts it away.

[systemed]

Yep. We need to write out pbf vector tiles though so protozero might be worth it for the speed boost.

[WHM1]

how to add [ pkg-config --libs protobuf ]. can you show me you makefile code. thks

[systemed]

Just to say, a PR to fix the makefile would be very happily welcomed (after countless bad experiences I'm now a Homebrew-free zone, so probably can't reproduce this exactly).

[russss]

Just to say, a PR to fix the makefile would be very happily welcomed (after countless bad experiences I'm now a Homebrew-free zone, so probably can't reproduce this exactly).

I'm not sure this is possible in a reliable way without using autotools or cmake, which is why I didn't already :)

[systemed]

💡

We do have a cmake build already, of course. Does that work?

[russss]

I didn't spot that! The cmake build does indeed work fine.

[FunMiles]

cmake build does not work for me. The version of cmake on which it is based is too old. It needs to be moved to more modern cmake to avoid the linking issue. Maybe because the version of absl found is not the one protobuf (installed by brew) was built with. In any case, modern cmake does get the correct dependencies directly. I will make a PR.

[systemed]

That would be brilliant - yes please.

[FunMiles]

@systemed Done: #555

[systemed]

v3.0 fixes this by removing the protobuf dependency :)