Ubuntu - Documentation for Intel compiler build

[ghost]

Build System based on Ubuntu 14.04 and Intel Compiler 16.

Using these exact commits of the develop branches:

• CORESRC=/home/libweb3core
  fa1943c144121ae78b3f390a6a6fcdbeef59d953
• HELPERSRC=/home/webthree-helpers
  cf112efa34baa01a9cf60ec699537c840d34e9dd
• LIBSRC=/home/libethereum
  34b7273f69e98b789257dc03ad4fa6d89ba9b39c

Build with GNU compiler generates valid binaries.
Build with Intel Compiler generates boost::multiprecision Error

/home/libweb3core/libdevcore/CommonData.h(120): error: no suitable conversion function from "dev::u256" to "uint8_t={unsigned char}" exists o_out[i - 1] = (typename Out::value_type)(uint8_t)v; detected during instantiation of "void dev::toBigEndian(T, Out &) [with T=dev::u256, Out=std::string]" at line 137

The build logs are attached.
Any suggestions?
Ubuntu14.04_GNU_libethereum-build_bot-20160325182012.txt
Ubuntu14.04_INTEL_libethereum-build_bot-20160325183854.txt

[bobsummerwill]

Build with GCC? :-)

But seriously, I assume that you have a specific reason for using the Intel compiler, but it looks like you are the first to try that particular combination and have hit some elements in our code (or its dependencies) which the Intel compiler doesn't like.

The "no suitable conversion ones" are happening for boost types, ie ...

using u256 = boost::multiprecision::number<boost::multiprecision::cpp_int_backend<256, 256, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>>;

So you'll have to either work out for yourself what is happening there or seek help from the Boost team or Intel compiler team. We're just using Boost, and it works on multiple platforms with multiple compilers, but maybe there is some shortfall for the Intel compiler? I am afraid I must plead ignorance there.

Just spotted this in the log :-) Which is quite true. I'm afraid that the Intel compiler isn't something which we're claiming to support, but if you can get it to work, we'd be happy to take back pull-requests to add that support.

CMake Warning at webthree-helpers/cmake/EthCompilerSettings.cmake:93 (message):
Your compiler is not tested, if you run into any issues, we'd welcome any
patches.probl

There are a few more warnings in your log too, which it is hard to tell whether are serious or not. I've just gone through the effort of fixing all our warnings and enabling warnings-as-errors, so we would want to get to the bottom of them all, and ensure we could build warning-free with the Intel compiler before pulling back changes.

Are you up for battling through them? I'll do what I can to help, but that will mainly be cheerleading, I am afraid. Good luck!

[ghost]

I will systematically resolve all issues to enable compilation with ICPC16.
Using the GNU binaries and the vanilla ethash_cl_miner_kernel.cl the k1om x100 5110P generates ~15Mhs.
I expect that a native k1om application will outperform 15Mhs as the Xeon Phi supports 512bit SIMD instructions.

root@coast77:/home# /home/webthree-umbrella/build/libethereum/ethminer/ethminer --list-devices
[OPENCL]:
Listing OpenCL devices.
FORMAT: [deviceID] deviceName
[0] Intel(R) Many Integrated Core Acceleration Card
CL_DEVICE_TYPE: ACCELERATOR
CL_DEVICE_GLOBAL_MEM_SIZE: 6053646336
CL_DEVICE_MAX_MEM_ALLOC_SIZE: 2017882112
CL_DEVICE_MAX_WORK_GROUP_SIZE: 8192

[bobsummerwill]

Excellent. All power to you! Thanks.

[ghost]

Greetings, got most of the errors resolved using defines and convert_to.

#ifdef __INTEL_COMPILER
#else
#endif

.convert_to<uint8_t>()
.convert_to<unsigned>()
.convert_to<size_t>()
.convert_to<uint16_t>()
.convert_to<int>()
.convert_to<double>()
.convert_to<byte>()
.convert_to<string>()
.convert_to<uint64_t>()

Regarding \libweb3core\libdevcore\Base64.h:
The problem occurs with templates.

Do you have any advice for resolution of this particular case?

/home/libweb3core/libdevcore/Base64.h(48): error: no suitable conversion function from "boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<160U, 160U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off>" to "unsigned int" exists
        unsigned r = (unsigned)(a - a / 36 * 36); // boost's % is broken
                               ^
          detected during instantiation of "std::string dev::toBase36(const dev::FixedHash<N> &) [with N=20UL]" at line 114 of "/home/libethereum/libethcore/ICAP.cpp"

I tried convert_to, but got an error.

template <size_t N> inline std::string toBase36(FixedHash<N> const& _h)
{
    static char const* c_alphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ";
    typename FixedHash<N>::Arith a = _h;
    std::string ret;
    for (; a > 0; a /= 36)
    {
#ifdef __INTEL_COMPILER
        unsigned r = (unsigned)((a - a / 36 * 36).convert_to<unsigned>()); // boost's % is broken
#else
        unsigned r = (unsigned)(a - a / 36 * 36); // boost's % is broken
#endif
        ret = c_alphabet[r] + ret;
    }
    return ret;
}

/home/libweb3core/libdevcore/CommonData.h(121): error: type name is not allowed
        o_out[i - 1] = (typename Out::value_type)(uint8_t)(v.convert_to<uint8_t>());
                                                                        ^

/home/libweb3core/libdevcore/CommonData.h(121): error: expected an expression
        o_out[i - 1] = (typename Out::value_type)(uint8_t)(v.convert_to<uint8_t>());
                                                                                 ^

Regarding the template with toBigEndian, just enumerated like this, definitely requires rework.

/// Converts a templated integer value to the big-endian byte-stream represented on a templated collection.
/// The size of the collection object will be unchanged. If it is too small, it will not represent the
/// value properly, if too big then the additional elements will be zeroed out.
/// @a Out will typically be either std::string or bytes.
/// @a T will typically by unsigned, u160, u256 or bigint.
template <class T, class Out>
inline void toBigEndian(T _val, Out& o_out)
{
    static_assert(std::is_same<bigint, T>::value || !std::numeric_limits<T>::is_signed, "only unsigned types or bigint supported"); //bigint does not carry sign bit on shift
    for (auto i = o_out.size(); i != 0; _val >>= 8, i--)
    {
        T v = _val & (T)0xff;
        o_out[i - 1] = (typename Out::value_type)(uint8_t)v;
    }
}
#ifdef __INTEL_COMPILER
template <class Out>
inline void toBigEndian(u256 _val, Out& o_out)
{
    for (auto i = o_out.size(); i != 0; _val >>= 8, i--)
    {
        u256 v = _val & (u256)0xff;
        /* u256 

        o_out[i - 1] = (typename Out::value_type)(uint8_t)v;

        TODO:
        Resolve all cases cleanly
        */

        uint8_t vv;
        boost::multiprecision::number<boost::multiprecision::cpp_int_backend<256, 256, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>> bb;
        bb = (boost::multiprecision::number<boost::multiprecision::cpp_int_backend<256, 256, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>>)v;
        vv = (uint8_t)bb.convert_to<uint8_t>();
        o_out[i - 1] = (typename Out::value_type)vv;
    }
}
template <class Out>
inline void toBigEndian(u160 _val, Out& o_out)
{
    for (auto i = o_out.size(); i != 0; _val >>= 8, i--)
    {
        u160 v = _val & (u160)0xff;
        /* u160 

        o_out[i - 1] = (typename Out::value_type)(uint8_t)v;

        TODO:
        Resolve all cases cleanly
        */
        uint8_t vv;
        boost::multiprecision::number<boost::multiprecision::cpp_int_backend<160, 160, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>> bb;

        bb = (boost::multiprecision::number<boost::multiprecision::cpp_int_backend<160, 160, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>>)v;
        vv = (uint8_t)bb.convert_to<uint8_t>();
        o_out[i - 1] = (typename Out::value_type)vv;
    }
}
template <class Out>
inline void toBigEndian(bigint _val, Out& o_out)
{
    for (auto i = o_out.size(); i != 0; _val >>= 8, i--)
    {
        bigint v = _val & (bigint)0xff;
        /* bigint 

        o_out[i - 1] = (typename Out::value_type)(uint8_t)v;

        TODO:
        Resolve all cases cleanly
        */
        uint8_t vv;
        boost::multiprecision::number<boost::multiprecision::cpp_int_backend<>> bb;

        bb = (boost::multiprecision::number<boost::multiprecision::cpp_int_backend<>>)v;
        vv = (uint8_t)bb.convert_to<uint8_t>();
        o_out[i - 1] = (typename Out::value_type)vv;
    }
}
template <class Out>
inline void toBigEndian(boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<512U, 512U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off> _val, Out& o_out)
{
    for (auto i = o_out.size(); i != 0; _val >>= 8, i--)
    {
        boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<512U, 512U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off> v = _val & (boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<512U, 512U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off>)0xff;
        /* boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<512U, 512U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off> 

        o_out[i - 1] = (typename Out::value_type)(uint8_t)v;

        TODO:
        Resolve all cases cleanly
        */
        uint8_t vv;
        boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<512U, 512U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off> bb;

        bb = (boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<512U, 512U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off>)v;
        vv = (uint8_t)bb.convert_to<uint8_t>();
        o_out[i - 1] = (typename Out::value_type)vv;
    }
}
template <class Out>
inline void toBigEndian(boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<64U, 64U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off> _val, Out& o_out)
{
    for (auto i = o_out.size(); i != 0; _val >>= 8, i--)
    {
        boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<64U, 64U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off> v = _val & (boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<64U, 64U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off>)0xff;
        /* boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<64U, 64U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off> 

        o_out[i - 1] = (typename Out::value_type)(uint8_t)v;

        TODO:
        Resolve all cases cleanly
        */
        uint8_t vv;
        boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<64U, 64U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off> bb;

        bb = (boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<64U, 64U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off>)v;
        vv = (uint8_t)bb.convert_to<uint8_t>();
        o_out[i - 1] = (typename Out::value_type)vv;
    }
}
template <class Out>
inline void toBigEndian(boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<128U, 128U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off> _val, Out& o_out)
{
    for (auto i = o_out.size(); i != 0; _val >>= 8, i--)
    {
        boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<128U, 128U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off> v = _val & (boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<128U, 128U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off>)0xff;
        /* boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<128U, 128U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off> 

        o_out[i - 1] = (typename Out::value_type)(uint8_t)v;

        TODO:
        Resolve all cases cleanly
        */
        uint8_t vv;
        boost::multiprecision::number<boost::multiprecision::cpp_int_backend<>> bb;

        bb = (boost::multiprecision::number<boost::multiprecision::cpp_int_backend<>>)v;
        vv = (uint8_t)bb.convert_to<uint8_t>();
        o_out[i - 1] = (typename Out::value_type)vv;
    }
}
#endif

In order to minize build for ethminer binary, is it safe to disable Eth::evm and Eth::lll?
When trying to link ethminer:

[100%] Building CXX object libethereum/ethminer/CMakeFiles/ethminer.dir/main.cpp.o
Linking CXX executable ethminer
../libethereum/libethereum.so: undefined reference to `dev::eth::ExtVMFace::ExtVMFace(dev::eth::EnvInfo const&, dev::FixedHash<20u>, dev::FixedHash<20u>, dev::FixedHash<20u>, boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<256u, 256u, (boost::multiprecision::cpp_integer_type)0, (boost::multiprecision::cpp_int_check_type)0, void>, (boost::multiprecision::expression_template_option)0>, boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<256u, 256u, (boost::multiprecision::cpp_integer_type)0, (boost::multiprecision::cpp_int_check_type)0, void>, (boost::multiprecision::expression_template_option)0>, dev::vector_ref<unsigned char const>, std::vector<unsigned char, std::allocator<unsigned char> >, dev::FixedHash<32u> const&, unsigned int)'
../libethereum/libethereum.so: undefined reference to `dev::eth::compileLLL(std::string const&, bool, std::vector<std::string, std::allocator<std::string> >*)'
../libethereum/libethereum.so: undefined reference to `dev::eth::VMFactory::create()'
../libethereum/libethereum.so: undefined reference to `dev::eth::VMFactory::create(dev::eth::VMKind)'
../libethereum/libethereum.so: undefined reference to `dev::eth::instructionInfo(dev::eth::Instruction)'
make[2]: *** [libethereum/ethminer/ethminer] Error 1

Base64Error_libethereum-build_bot-20160327024444.txt
Base64_fail_libethereum-build_bot-20160329004639.txt
Linker_libethereum-build_bot-20160329012817.txt

[bobsummerwill]

Hey @mancoast!
Sorry for the lag ..

I would ignore ethminer binary if I were you. For the time being at least.

Why don't you do a fork-and-clone and create some PRs for the changes in process which we can discuss?

[bobsummerwill]

For the type conversion issues, perhaps @chriseth will know a little better than me?

[chriseth]

Oh, this looks quite horrible. I have seen the following in the boost documentation, perhaps that helps:
int i = z.template convert_to<int>(); // sets i to 2

[bobsummerwill]

Yikes to that syntax! Did it get you further on?

[ghost]

Yea, all the requirements have been fulfilled with the single exception of Base64.h.
https://github.com/pwntoast512/libethereum
https://github.com/pwntoast512/libweb3core

After completion of toBase36 I will fork using this username for a cleaner, single commit, pull request.
@bobsummerwill @chriseth
Please advise fulfillment of the following requirement:

template <size_t N> inline std::string toBase36(FixedHash<N> const& _h)
{
    static char const* c_alphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ";
    typename FixedHash<N>::Arith a = _h;
    std::string ret;
    for (; a > 0; a /= 36)
    {
#ifdef __INTEL_COMPILER
        //TODO
        //unsigned r = (unsigned)((a - a / 36 * 36).convert_to<unsigned>()); // boost's % is broken
        unsigned r = 0;
#else
        unsigned r = (unsigned)(a - a / 36 * 36); // boost's % is broken
#endif
        ret = c_alphabet[r] + ret;
    }
    return ret;
}

Thanks!

[bobsummerwill]

"Please advise fulfillment of the following requirement:"

Sorry - I don't know what exactly you mean. Please could you rephrase? There would need to be some working expression inside the Intel #ifdef, wouldn't there? Or the code would be broken.

[ghost]

Restored dev::toBase36 to vanilla && resolved, via ICAP modification.

#ifdef __INTEL_COMPILER
        static char const* c_alphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ";
        typename FixedHash<Address::size>::Arith a = m_direct;
        std::string dd;
        for (; a > 0; a /= 36)
        {
            unsigned r = (unsigned)((a - a / 36 * 36)).convert_to<unsigned>(); // boost's % is broken
            dd = c_alphabet[r] + dd;
        }
        std::string d = dd;
#else
        std::string d = toBase36<Address::size>(m_direct);
#endif

[bobsummerwill]

Have you still got outstanding breaks, or have you got to the link step yet?

[ghost]

ICC issue is resolved, on to the next one: Xeon PHI x100 Co-Processor
i am currently recompiling all required libraries for k1om, might be that simple...

[ghost]

libethereum-build_bot-20160401193304.txt
ethmin-trials-20160401200517.txt

Accidentally clicked the close button.
Binaries are confirmed functional with x64.

Looks like dual E5-2603 are hashing at ~.25 to .30 Mhs.
Code is functional, but requires cleanup.

[bobsummerwill]

Great - getting there, eh!
Thanks for all your work on this. I just commented on one of the PRs.
I'd like to try to eliminate most of the #ifdefs if at all possible, to make it easier to maintain. And if that means changes to the existing code, that is fine.

[ghost]

Greetings!

Questions:
1.) should i close the pull requests?
2.) How to handle scenarios that still require different logic for the ICPC [still need pound defines]?
3.) is pull request with multiple commits acceptable?

Please advise this decision point: How to proceed: a.) or b.)?
a.) add new commits to pull request
b.) close pull request, generate new diff, open new pull request.

Thanks,
coast

[bobsummerwill]

Just keep making further commits on those branches until we are both happy with the content.
The usual flow is to keep going with that, and then "squash" them just before committing.

Like http://gitready.com/advanced/2009/02/10/squashing-commits-with-rebase.html

[ghost]

Alrighty, most of the defines have been removed.
The use of member function convert_to remains an issue with template parameters.
How should we proceed?

[bobsummerwill]

Just adding links to the two PRs:

ethereum/libethereum#223
ethereum/libweb3core#67

[bobsummerwill]

Just a few more questions.

1. For the toBigEndian() functions, are those OVERRIDES to existing functions? Where are they called from? Without those, what build errors do you get? What fails? I'm sure that what you have is correct, but I would just like to better understand where they are used.
2. For the toBase36 one in libethereum, should that actually be an override to that function within an ifdef in the base64.h in libdevcore? rather than inlined in the call site? Or would that not work?

Beyond that I would just want to check with @chriseth on the test run results we are seeing for these 2 PRs, which I think are issues in our automation, and nothing to do with your changes, and then I think we would be good to go.

No doubt further breaks will pop up in the future which we would be blind to, but obviously feel free to submit PRs as we go.

Did you get the performance improvements you were hoping for?

[bobsummerwill]

Oh - and the squashes will still need doing, but we would defer that until we're 100% happy with the content. Thanks again for your persistence!

[ghost]

toBigEndian: T=u256 || T=u160

/home/cpp-ethereum/libdevcore/../libdevcore/CommonData.h(111): error: no suitable conversion function from "dev::u256" to "uint8_t={unsigned char}" exists
        o_out[i - 1] = (typename Out::value_type)(uint8_t)v;
                                                          ^
          detected during instantiation of "void dev::toBigEndian(T, Out &) [with T=dev::u256, Out=std::string]" at line 128

/home/cpp-ethereum/libdevcore/../libdevcore/CommonData.h(111): error: no suitable conversion function from "dev::u160" to "uint8_t={unsigned char}" exists
        o_out[i - 1] = (typename Out::value_type)(uint8_t)v;
                                                          ^
          detected during instantiation of "void dev::toBigEndian(T, Out &) [with T=dev::u160, Out=std::string]" at line 129

/home/cpp-ethereum/libdevcore/../libdevcore/CommonData.h(111): error: no suitable conversion function from "dev::u256" to "uint8_t={unsigned char}" exists
        o_out[i - 1] = (typename Out::value_type)(uint8_t)v;
                                                          ^
          detected during instantiation of "void dev::toBigEndian(T, Out &) [with T=dev::u256, Out=dev::bytes]" at line 130

/home/cpp-ethereum/libdevcore/../libdevcore/CommonData.h(111): error: no suitable conversion function from "dev::u160" to "uint8_t={unsigned char}" exists
        o_out[i - 1] = (typename Out::value_type)(uint8_t)v;
                                                          ^
          detected during instantiation of "void dev::toBigEndian(T, Out &) [with T=dev::u160, Out=dev::bytes]" at line 131

compilation aborted for /home/cpp-ethereum/libdevcore/FileSystem.cpp (code 2)

toBigEndian: T=bigint

/home/libweb3core/libdevcore/CommonData.h(120): error: no suitable conversion function from "dev::bigint" to "uint8_t={unsigned char}" exists
        o_out[i - 1] = (typename Out::value_type)(uint8_t)v;
                                                          ^
          detected during:
            instantiation of "void dev::toBigEndian(T, Out &) [with T=dev::bigint, Out=dev::bytes]" at line 195
            instantiation of "dev::bytes dev::toCompactBigEndian(T, unsigned int) [with T=dev::bigint]" at line 125 of "/home/libweb3core/rlp/main.cpp"

compilation aborted for /home/libweb3core/rlp/main.cpp (code 2)

toBigEndian: T=cpp_int_backend<512U>

/home/libweb3core/libdevcore/CommonData.h(120): error: no suitable conversion function from "boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<512U, 512U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off>" to "uint8_t={unsigned char}" exists
        o_out[i - 1] = (typename Out::value_type)(uint8_t)v;
                                                          ^
          detected during:
            instantiation of "void dev::toBigEndian(T, Out &) [with T=boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<512U, 512U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off>, Out=std::array<byte={uint8_t={unsigned char}}, 64UL>]" at line 72 of "/home/libweb3core/libdevcore/FixedHash.h"
            instantiation of "dev::FixedHash<N>::FixedHash(const dev::FixedHash<N>::Arith &) [with N=64U]" at line 91 of "/home/libweb3core/libdevcore/CommonJS.h"
            instantiation of "dev::FixedHash<N> dev::jsToFixed<N>(const std::string &) [with N=64U]" at line 37 of "/home/libethereum/libethcore/CommonJS.h"

compilation aborted for /home/libethereum/libethcore/CommonJS.cpp (code 2)

toBigEndian: T=cpp_int_backend<64U>

/home/libweb3core/libdevcore/CommonData.h(120): error: no suitable conversion function from "boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<64U, 64U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off>" to "uint8_t={unsigned char}" exists
        o_out[i - 1] = (typename Out::value_type)(uint8_t)v;
                                                          ^
          detected during:
            instantiation of "void dev::toBigEndian(T, Out &) [with T=boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<64U, 64U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off>, Out=std::array<byte={uint8_t={unsigned char}}, 8UL>]" at line 72 of "/home/libweb3core/libdevcore/FixedHash.h"
            instantiation of "dev::FixedHash<N>::FixedHash(const dev::FixedHash<N>::Arith &) [with N=8U]" at line 59 of "/home/libweb3core/libdevcore/Base64.h"
            instantiation of "dev::FixedHash<N> dev::fromBase36<N>(const std::string &) [with N=8UL]" at line 151 of "/home/libethereum/libethcore/ICAP.cpp"

compilation aborted for /home/libethereum/libethcore/ICAP.cpp (code 2)

toBigEndian: T=cpp_int_backend<128U>

/home/libweb3core/libdevcore/CommonData.h(120): error: no suitable conversion function from "boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<128U, 128U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off>" to "uint8_t={unsigned char}" exists
        o_out[i - 1] = (typename Out::value_type)(uint8_t)v;
                                                          ^
          detected during:
            instantiation of "void dev::toBigEndian(T, Out &) [with T=boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<128U, 128U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off>, Out=std::array<byte={uint8_t={unsigned char}}, 16UL>]" at line 72 of "/home/libweb3core/libdevcore/FixedHash.h"
            instantiation of "dev::FixedHash<N>::FixedHash(const dev::FixedHash<N>::Arith &) [with N=16U]" at line 735 of "/home/libethereum/ethkey/KeyAux.h"

compilation aborted for /home/libethereum/ethkey/main.cpp (code 2)

toBase36: initial error

[ 44%] Building CXX object libethereum/libethcore/CMakeFiles/ethcore.dir/ICAP.cpp.o
/home/libweb3core/libdevcore/Base64.h(48): error: no suitable conversion function from "boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<160U, 160U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off>" to "unsigned int" exists
        unsigned r = (unsigned)(a - a / 36 * 36); // boost's % is broken
                               ^
          detected during instantiation of "std::string dev::toBase36(const dev::FixedHash<N> &) [with N=20UL]" at line 118 of "/home/libethereum/libethcore/ICAP.cpp"

compilation aborted for /home/libethereum/libethcore/ICAP.cpp (code 2)

toBase36: member function fail

/home/libweb3core/libdevcore/Base64.h(49): error: type name is not allowed
        unsigned r = (unsigned)((a - a / 36 * 36).convert_to<unsigned>()); // boost's % is broken
                                                             ^

/home/libweb3core/libdevcore/Base64.h(49): error: expected an expression
        unsigned r = (unsigned)((a - a / 36 * 36).convert_to<unsigned>()); // boost's % is broken
                                                                       ^

compilation aborted for /home/libethereum/libethcore/ICAP.cpp (code 2)

I would prefer to use an template "override".
Is this different from C++11 override?
http://en.cppreference.com/w/cpp/language/override

What is the exact signature required for the additional toBase36 function?

I have tried using 20UL, but the function remains uncalled and icpc throws an error: no suitable conversion.

#ifdef __INTEL_COMPILER
inline std::string toBase36(FixedHash<20UL> const& _h)
{
    static char const* c_alphabet = "0123456789ABCDEFGHIJKLMNOPQRSTUVWXYZ";
    typename FixedHash<20UL>::Arith a = _h;
    std::string ret;
    for (; a > 0; a /= 36)
    {
        unsigned r = (unsigned)((a - a / 36 * 36)).convert_to<unsigned>(); // boost's % is broken
        ret = c_alphabet[r] + ret;
    }
    return ret;
}
#endif

/home/libweb3core/libdevcore/Base64.h(48): error: no suitable conversion function from "boost::multiprecision::number<boost::multiprecision::backends::cpp_int_backend<160U, 160U, boost::multiprecision::unsigned_magnitude, boost::multiprecision::unchecked, void>, boost::multiprecision::et_off>" to "unsigned int" exists
        unsigned r = (unsigned)(a - a / 36 * 36); // boost's % is broken
                               ^
          detected during instantiation of "std::string dev::toBase36(const dev::FixedHash<N> &) [with N=20UL]" at line 118 of "/home/libethereum/libethcore/ICAP.cpp"

compilation aborted for /home/libethereum/libethcore/ICAP.cpp (code 2)

[ghost]

The baseline native performance for mic, requires all the libraries recompiled for mic.
Currently completed is gmp6_1 and boost1_55.
Many more libraries remain to be cross compiled before the ethereum binaries will link.

After establishing a baseline, it will be interesting to tune the ethminer binary.

[bobsummerwill]

If you are only on boost 55, it might be worth your while updating that.

1.60 is the latest version.

There might be a case for sending them some patches too, eh?

[bobsummerwill]

I'm still not clear on where toBigEndian() comes from. Is that boost API?

[ghost]

I tested with boost 1-60 release, but there was no resolution. My guess is that the Intel compiler has more strict lexi. I would not consider that unexpected from Intel platform. As for additional boost patch, yes the "using bigint =.." causes another error. That is separate scope, but will be analysed later.

ToBigEndian is a template core function of libweb3core, not related to boost. The error occurs when this libweb3core template function is instantiated with boost's wide types.

[ghost]

confirmed resolved

[gcolvin]

Thanks for clearing that up John @jzmaddock
And thanks for sticking with this so long @mancoast

[bobsummerwill]

Thanks for your magnificently detailed reply, @jzmaddock! Much appreciated.

And thanks indeed for your long and noble work on getting this going, @mancoast! I am going to reopen this issue, though, because we aren't quite done yet.

After all of this work, I want to ensure that other people are able to build eth using the Intel compiler as well, and we're not quite there yet. I want to lock all of this learning and effort in :-)

Our CMake compiler settings don't have any flows for Intel compiler.

https://github.com/ethereum/webthree-helpers/blob/develop/cmake/EthCompilerSettings.cmake

Did you just have some local edits for this, @mancoast? Or command-line overrides? What settings did you use?

I would like us to have instructions for Intel compiler builds to add to http://www.ethdocs.org/en/latest/ethereum-clients/cpp-ethereum/building-from-source/index.html. What elements of your local setup are we missing, @mancoast? What instructions do we need to add to get random people in a position where they can replicate your results?

Also, in terms of the specific version requirements, etc. we should get those encoded into the CMake files as well, within that new "if Intel" conditional.

[bobsummerwill]

Oh yes, one more thing.

If we don't need the convert_to<> changes anymore, I would like to undo them, because they are proving to be a bit of a merge nightmare with a major set of refactorings which @gcolvin is working on in the same codebase. If they were winning us something we would probably just leave them in, but given that they are causing us pain AND they appear no longer to be required, reverting them back out seems best.

More generally, it looks like our codebase is a pretty horrible mess of C-style casts in general, but that is another story entirely :-)

[bobsummerwill]

So I think that would mean reverting the following:

• https://github.com/ethereum/libethereum/pull/224/files
• https://github.com/ethereum/libethereum/pull/223/files (most of, but not 100%)
• https://github.com/ethereum/libweb3core/pull/68/files
• https://github.com/ethereum/libweb3core/pull/67/files (most of, but not 100%)

It is probably best if YOU actually create PRs for these, eh, @mancoast, because you can verify that we don't over-strip in the process, and re-break your Intel builds.

BTW - Are you able to build ALL the executables now?

[ghost]

Revert convert_to usage:
ethereum/libethereum#252
ethereum/libweb3core#76

[ghost]

Intel x64 cmake command line:

# Build x64 #
USRINC="-I$SRC/microhttpd-0.9.28/build/include/ -I$SRC/json-rpc-cpp-0.5.0/build/gen/jsonrpccpp/common/ -I$SRC/json-rpc-cpp-0.5.0/build/gen/"
cd $BUILDTREE \
&& CC=icc CXX=icpc \
CXXFLAGS="-std=c++11 -O2 -g -DETH_RELEASE -pedantic -DSHAREDLIB -fPIC $USRINC" \
CFLAGS="-02 -g" \
BOOST_ROOT=$SRC/boost_1_60_0 \
BOOST_LIBRARYDIR=$SRC/boost_1_60_0/build/lib \
Boost_NO_SYSTEM_PATHS=ON \
cmake3 \
-DGMP_INCLUDE_DIR=$SRC/gmp-6.1.0/build/ \
-DGMP_LIBRARY=$SRC/gmp-6.1.0/build/lib/libgmp.a \
-DJSONCPP_INCLUDE_DIR=$SRC/jsoncpp-1.6.2/include/ \
-DJSONCPP_LIBRARY=$SRC/jsoncpp-1.6.2/build/src/lib_json/libjsoncpp.a \
-DLEVELDB_INCLUDE_DIR=$SRC/leveldb-1.3/include/ \
-DLEVELDB_LIBRARY=$SRC/leveldb-1.3/build/libleveldb.so \
-DCRYPTOPP_INCLUDE_DIR=$SRC/cryptopp-5.6.2/build/include/ \
-DCRYPTOPP_LIBRARY=$SRC/cryptopp-5.6.2/build/lib/libcryptopp.a \
-DJSON_RPC_CPP_INCLUDE_DIR=$SRC/json-rpc-cpp-0.5.0/src/ \
-DJSON_RPC_CPP_COMMON_LIBRARY=$SRC/json-rpc-cpp-0.5.0/build/lib/libjsonrpccpp-common.so \
-DJSON_RPC_CPP_SERVER_LIBRARY=$SRC/json-rpc-cpp-0.5.0/build/lib/libjsonrpccpp-server.so \
-DJSON_RPC_CPP_CLIENT_LIBRARY=$SRC/json-rpc-cpp-0.5.0/build/lib/libjsonrpccpp-client.so \
-DCURL_INCLUDE_DIR=/usr/include/ \
-DCURL_LIBRARY=/usr/lib64/libcurl.so \
-DFATDB=OFF \
-DGUI=OFF \
-DTESTS=OFF \
-DETHASHCL=OFF \
-DEVMJIT=OFF \
-DSOLIDITY=OFF \
-DMINIUPNPC=OFF \
-G "Eclipse CDT4 - Unix Makefiles" \
$SRC/
cd $BUILDTREE && make VERBOSE=1

CFLAGS="-02 -g" \ <-- incorrect?

webthree-umbrella\CMakeLists.txt:
link_directories(${USRLIB_LINK_DIR})

[bobsummerwill]

Thanks, @mancoast - I have merged the convert_to reverts.

@gcolvin - Please could you do some diffs, and see how your content is now tracking against HEAD of develop for libethereum?

[chriseth]

Can this be closed?

[ghost]

everything functions on my end.

[bobsummerwill]

I've not closed it yet because it still needs documentation. On my plate! Will get to it soon.

[bobsummerwill]

Hey @mancoast!

Just doing release notes for cpp-ethereum-v1.2.6. Would you like to be credited with a real name?

Also, whether or not you stay pseudonymous, please can you add an e-mail address which you can be contacted at to http://piratepad.net/g9A0NTQjcI? Thanks! That is for #530

[ghost]

all updated, thanks

[bobsummerwill]

Hello ROBERT! Thanks :-)

[bobsummerwill]

Moved to ethereum/aleth#3219, but not all comments came across cleanly.

[SKlayer]

@bobsummerwill Hello ,i tried to do same thing to compilie to my Xeon Phi. but the intel compiler is not free to student (the XE not include ICC)..... so could you pelease give me a binary code to me? Thank you very much... My email :1919962626@qq.com 感激不尽