Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

general crayCC compilation failure #1091

Closed
cpranger opened this issue May 16, 2018 · 12 comments
Closed

general crayCC compilation failure #1091

cpranger opened this issue May 16, 2018 · 12 comments

Comments

@cpranger
Copy link

I am very interested to use this amazing JSON library for my project. However, I am virtually tied to using the Cray Compiling Environment (CCE) and the Cray C++ compiler (crayCC). I realise that what I am reporting below may well be outside the scope of the developer(s), and that moreover, CrayCC is not on the list of supported compilers. Yet I want to make mention of this, and leave it up to the author to decide what to do with this information. Any help would be appreciated!

I have constructed a minimal working example (hello world) which does absolutely nothing with JSON except include the single header file:

#include <iostream>
#include "nlohmann/json.hpp"

int main()
	{ std::cout << "hello world!" << std::endl; }

this program is compiled with either gcc 7.2.0 producing expected behaviour (hello world!), and with crayCC 8.6.5 (https://pubs.cray.com/content/S-2179/8.6/cray-c-and-c++-reference-manual-s-2179-86/about-the-cray-and-c++-reference-manual).

> CC hello_json.cpp

the output is:

/tmp/pe_38754/hello_json_1.o: In function `std::_Sp_counted_base<(__gnu_cxx::_Lock_policy)2>::_M_release()':
/opt/gcc/6.1.0/snos/include/g++/bits/shared_ptr_base.h:147: undefined reference to `_ZN23_INTERNAL_b13fd261_main9__gnu_cxx27__exchange_and_add_dispatchEPii$$CFE_id_b13fd261_main'
/opt/gcc/6.1.0/snos/include/g++/bits/shared_ptr_base.h:162: undefined reference to `_ZN23_INTERNAL_b13fd261_main9__gnu_cxx27__exchange_and_add_dispatchEPii$$CFE_id_b13fd261_main'
/tmp/pe_38754/hello_json_1.o: In function `std::__shared_ptr<nlohmann::detail::input_adapter_protocol, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr()':
/users/cpranger/garnet/experiments/json_compilation/hello_json.cpp:1: undefined reference to `_ZN23_INTERNAL_b13fd261_main9__gnu_cxx27__exchange_and_add_dispatchEPii$$CFE_id_b13fd261_main'
/users/cpranger/garnet/experiments/json_compilation/hello_json.cpp:1: undefined reference to `_ZN23_INTERNAL_b13fd261_main9__gnu_cxx27__exchange_and_add_dispatchEPii$$CFE_id_b13fd261_main'
/tmp/pe_38754/hello_json_1.o: In function `std::__shared_ptr<nlohmann::detail::output_adapter_protocol<char>, (__gnu_cxx::_Lock_policy)2>::~__shared_ptr()':
/users/cpranger/garnet/experiments/json_compilation/hello_json.cpp:7488: undefined reference to `_ZN23_INTERNAL_b13fd261_main9__gnu_cxx27__exchange_and_add_dispatchEPii$$CFE_id_b13fd261_main'
/tmp/pe_38754/hello_json_1.o:/users/cpranger/garnet/experiments/json_compilation/hello_json.cpp:7488: more undefined references to `_ZN23_INTERNAL_b13fd261_main9__gnu_cxx27__exchange_and_add_dispatchEPii$$CFE_id_b13fd261_main' follow
/tmp/pe_38754/hello_json_1.o: In function `std::unique_ptr<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >* nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer>::create<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*)#1}>::~unique_ptr()':
/opt/gcc/6.1.0/snos/include/g++/bits/unique_ptr.h:236: undefined reference to `_ZZN8nlohmann10basic_jsonISt3mapSt6vectorNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEblmdSaNS_14adl_serializerEE6createIS8_JRKS8_EEEPT_DpOT0_ENKUlPS8_E_clESJ_$$CFE_id_b13fd261_main'
/opt/cray/pe/cce/8.6.5/binutils/x86_64/x86_64-pc-linux-gnu/bin/ld: link errors found, deleting executable `a.out'
/usr/bin/X11/sha1sum: a.out: No such file or directory

FYI, _ZN23_INTERNAL_b13fd261_main9__gnu_cxx27__exchange_and_add_dispatchEPii$$CFE_id_b13fd261_main demangles to

_INTERNAL_b13fd261_main::__gnu_cxx::__exchange_and_add_dispatch(int*, int)$$CFE_id_b13fd261_main

and _ZZN8nlohmann10basic_jsonISt3mapSt6vectorNSt7__cxx1112basic_stringIcSt11char_traitsIcESaIcEEEblmdSaNS_14adl_serializerEE6createIS8_JRKS8_EEEPT_DpOT0_ENKUlPS8_E_clESJ_$$CFE_id_b13fd261_main demangles to

std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >* nlohmann::basic_json<std::map, std::vector, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, bool, long, unsigned long, double, std::allocator, nlohmann::adl_serializer>::create<std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&>(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> > const&)::{lambda(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*)#1}::operator()(std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >*) const$$CFE_id_b13fd261_main

Please let me know if any more information is required!

Casper

@cpranger
Copy link
Author

It seems I cannot edit my own issue (older browser?), so I will add here that I am using the version I cloned this morning from the development branch.

@OvermindDL1
Copy link

That doesn't look like a json library issue, that looks like that the included system headers for the C++ standard std::shared_ptr is using compare-and-swap intrinsics (which is a good thing) that the compiler itself is not resolving. This looks like an issue either with the compiler (less likely though to be honest) or with the standard libraries being used (this is the most likely issue).

@OvermindDL1
Copy link

OvermindDL1 commented May 16, 2018

Try compiling something like this using the same steps as you used above and tell what happens (don't run it, just compile it):

#include <memory>

void main() {
  std::shared_ptr<void*> test;
  test = &test;
  test.reset();
}

@cpranger
Copy link
Author

Thank you for your comment. You may well be right that this is a compiler issue. shared_ptr and unique_prt are used extensively throughout my project without any issues. Your provided test compiles neither in gcc version 7.2.0 nor with crayCC version 8.6.5 (nor in my head, to be honest), with crayCC complaining that:

  No operator "=" matches these operands.
            The operand types are:  std::shared_ptr<void *> =
                      std::shared_ptr<void *> *.
    test = &test;

and gcc making the same complaint in different wording. FYI, I believe the crayCC compiler is using the STL of gcc 6.1.0

Thanks again for your input,

Casper

@OvermindDL1
Copy link

Eh ignore the test = &test part, it was just a quick add to test equality but feel free to assign 0 to it. ^.^;

I believe the crayCC compiler is using the STL of gcc 6.1.0

Yeah which is why it makes me thing that the json library is using shared_ptr more completely than your project...

It's definitely hard to tell though, I've never even heard of that compiler, I thought all other systems have basically migrated to clang backends by now instead of inventing their own compilers... ^.^;

@cpranger
Copy link
Author

Both compilers accept your code example with the assignment test = 0;. I'm sorry this issue can be reproduced by only few people due to the exotic compiler! I presume Cray uses their own compilers for optimal performance on their proprietary hardware (in this case Cray XC40 and XC50). If you can propose further tests, I'd be happy to try them! In the mean time, I can try compiling against another STL version.

Thanks again,

Casper

@OvermindDL1
Copy link

Is the cray compiler a modified gcc? If so then do you know what version it is based on? Any updated one? If it is not then another standard library might help? Maybe a command-line flag to disable atomic operations?

@cpranger
Copy link
Author

These are hard questions! The compiling environment is kind of opaque to me, so I can not really answer them at this moment. Since you are essentially suggesting that the issue is likely not with the JSON library, I have contacted support of the machine in question and will report back as soon as there is news! In the meantime, thank your for helping!

Casper

@OvermindDL1
Copy link

I'm exceptionally curious about this. ^.^
Whatever you find out, please report back! It 'could' indeed be a bug in this library but based on all information given thus far I'd bet hard money that it's not. :-)

@cpranger
Copy link
Author

cpranger commented May 18, 2018

A helpful computational scientist working at the site of the machine came up with an interesting solution. While my original example didn't compile with crayCC, the following does:

#include <iostream>
#include "nlohmann/json.hpp"

int main()
{
    using nlohmann::json;
    auto a = std::make_shared<double>(5.0);
    std::stringstream ss1;
    ss1 << "[\"foo\",1,2,3,false,{\"one\":1}]";
    json j = json::parse(ss1);
    std::cout << "hello world!" << std::endl;
    std::cout << j << std::endl;
}

It is not clear to me why this would be necessary, and what the general underlying idea is of this fix (though I have asked for clarification on that). It seems that simply using the shared_ptr gets rid of the first few compilation errors, and that actually using JSON gets rid of the last. I'll have to see how this works out in my larger project since JSON will be an optional component, but this is a good start!

@OvermindDL1
Copy link

That... is very odd! I'm quite curious in this bug... What is that compiler doing...

@cpranger
Copy link
Author

@OvermindDL1, thank you very much for thinking with/for me! I have just implemented the fix posted above in my bigger project and it compiles fine now. Since at the moment I have no further information sources that could lead to a more general solution, I will close this issue. If I get any more feedback (from e.g. technical support of the machine) I will be sure to share!

All the best,

Casper

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants