Net Exception: Address family not supported with clang

[Bjoe]

I write a simple HTTP client to download a file and I get following exception:

Net Exception: Address family not supported

I try to find out the root cause of these error because my code was working.
I found following weird bug:

I print following member stack addresses:

• _counter member in RefCountedObject -> poco/Foundation/src/RefCountedObject.cpp:26
• _addr member in IPv4SocketAddressImpl -> poco/Net/src/SocketAddressImpl.cpp:63
• _addr member in IPv4SocketAddressImpl addr function -> poco/Net/include/Poco/Net/SocketAddressImpl.h:103

(see also my https://github.com/Bjoe/poco.git Branch: stack_address_issue)

and I get following result:

RefCountedObject::RefCountedObject() Stack address from _counter: 0x2639bc8
IPv4SocketAddressImpl::IPv4SocketAddressImpl() Stack address from _addr: 0x2639bcc
IPv4SocketAddressImpl::addr() Stack address from _addr: 0x2639bc8

WTH... why is the stack address from _addr in function addr the same stack addr in _counter 0x2639bc8 ?

I try some compiler and it looks like it only happens with clang and -std=XXX argument without any optimization:

clang++-3.8 -std=c++11
clang++-4.0 -std=c++11
clang++-3.8 -std=c++14
clang++-4.0 -std=c++14

If I move the function

inline const struct sockaddr* IPv4SocketAddressImpl::addr() const

from the header in the source file it works.

Have somebody an idea?
How looks like on android? They use also clang there.
Is this really a compiler bug?
Looks like on FreeBSD, they use clang as compiler, they have also this issue, see #1684

Here my code to test:

#include <Poco/Net/SocketAddress.h>
int main()
{
  Poco::Net::SocketAddress sa("localhost", 80);
  const sockaddr* s = sa.addr();
  return 0;
}

with my branch:
https://github.com/Bjoe/poco.git Branch: stack_address_issue

[aleks-f]

@Bjoe that's with 1.7.9, right?

[aleks-f]

@Bjoe I can't reproduce it, tried with clang 3.8 debug build. Somehow, there seems to be a compile-time mixup and AutoPtr ends up being mixed with SOO mode, then you have class inheriting from RefCountedObject, and the first data member is _counter. That's the only explanation I have for that behavior.

There was an unrelated problem with MSVC precompiled headers and inlining of some IPAddress functions, fixed recently in develop. Can you try similar fix with SocketAddress and see if it fixes the problem (even better, try to understand why it happens, I can't reproduce and don't see it from reading code).

thanks for your help.

[Bjoe]

@aleks-f Sorry for the late answer. I try to create a bug report to clang. I try to create a small snippet for the bug report but it is not easy to reproduce this bug :-(.

that's with 1.7.9, right?

Yes, sorry I forgott some infos.

Somehow, there seems to be a compile-time mixup and AutoPtr ends up being mixed with SOO mode, then you have class inheriting from RefCountedObject, and the first data member is _counter.

I also thinking about this. When is the define POCO_HAVE_ALIGNMENT set?

There was an unrelated problem with MSVC precompiled headers and inlining of some IPAddress functions, fixed recently in develop. Can you try similar fix with SocketAddress and see if it fixes the problem

Yes, that is what I mean with

If I move the function ... from the header in the source file ...

See also my pull request.

I tested on:
Distributor ID: Ubuntu
Description: Ubuntu 17.04
Release: 17.04
Codename: zesty
poco 1.7.9

@aleks-f On which platftorm you test this? Can you try following:

Use poco from my github repo:
https://github.com/Bjoe/poco.git

Use branch: stack_address_issue
or
commit id: dcbc92e2e3d71a58180d24343023108a530aac44

git checkout dcbc92e2e3d71a58180d24343023108a530aac44

Build poco:

poco/configure --omit=Data,NetSSL,Zip,XML,JSON,CppParser,Crypto,MongoDB,PDF,Util --prefix=/opt/poco-1.7.9 --no-samples --no-tests

Edit config.make. Change POCO_CONFIG = Linux to POCO_CONFIG = Linux-clang

or use cmake build system:
cmake -DCMAKE_CXX_COMPILER=clang++ -DCMAKE_C_COMPILER=clang -DCMAKE_BUILD_TYPE=Debug -DCMAKE_INSTALL_PREFIX=/opt/poco-1.7.9 -DENABLE_XML=OFF -DENABLE_JSON=OFF -DENABLE_MONGODB=OFF -DENABLE_PDF=OFF -DENABBLE_UTIL=OFF -DENABLE_NETSSL=OFF -DENABLE_CRYPTO=OFF -DENABLE_DATA=OFF -DENABLE_ZIP=OFF -DENABLE_PAGECOMPILER=OFF -DENABLE_PAGECOMPILER_FILE2PAGE=OFF poco

execute: make install

Compile this code:

#include <Poco/Net/SocketAddress.h>
int main()
{
  Poco::Net::SocketAddress sa("localhost", 80);
  const sockaddr* s = sa.addr();
  return 0;
}

with:

clang++ -I/opt/poco-1.7.9/include -std=c++1y -MD -MT main.cpp.o -MF main.cpp.o.d -o main.cpp.o -c main.cpp
clang++ -std=c++1y main.cpp.o -o test-debug -rdynamic /opt/poco-1.7.9/lib/libPocoNetd.so /opt/poco-1.7.9/lib/libPocoFoundationd.so

execute test-debug:
RefCountedObject::RefCountedObject Stack address from _counter: 0x1ab0fc8
IPv4SocketAddressImpl::IPv4SocketAddressImpl Stack address from _addr: 0x1ab0fcc
IPv4SocketAddressImpl::addr Stack address from _addr: 0x1ab0fc8

Now you should get the weird bug.

Build with -Ox here with -O1
clang++ -I/opt/poco-1.7.9/include -std=c++1y -O1 -MD -MT main.cpp.o -MF main.cpp.o.d -o main.cpp.o -c /Development/poco-example/main.cpp
clang++ -std=c++1y -O1 main.cpp.o -o test-debug -rdynamic /opt/poco-1.7.9/lib/libPocoNetd.so /opt/poco-1.7.9/lib/libPocoFoundationd.so

RefCountedObject::RefCountedObject Stack address from _counter: 0xb86fc8
IPv4SocketAddressImpl::IPv4SocketAddressImpl Stack address from _addr: 0xb86fcc
IPv4SocketAddressImpl::addr Stack address from _addr: 0xb86fcc

all is fine.

Build without -std=c++XX and withouth -Ox:

clang++-3.8 -I/opt/poco-1.7.9/include -MD -MT main.cpp.o -MF main.cpp.o.d -o main.cpp.o -c main.cpp
clang++-3.8 main.cpp.o -o test-debug -rdynamic /opt/poco-1.7.9/lib/libPocoNetd.so /opt/poco-1.7.9/lib/libPocoFoundationd.so

is also fine.

For me it looks like it is a clang compiler bug. Should I post this to clang bug report? Maybe I meet somebody from clang project on the "Meeting C++" conference in Berlin :-)

[aleks-f]

When is the define POCO_HAVE_ALIGNMENT set?

It is set here. The intent was to use the custom alignment code on pre-c++11 compilers, but as it is now, it will not come automatically into effect unless c++11 mode is enabled (ie. default c++11 build will use standard alignment code).

Should I post this to clang bug report?

If you can create a simple example reproducing it, sure. One way or the other, I'd un-inline all the related functions in SocketAddress and IPAddress to prevent it from happening.

I'm busy with other things at the moment, may come to it later.

I could not reproduce with:

alex@ubuntu16:~$ uname -a 
Linux ubuntu16 4.4.0-97-generic #120-Ubuntu SMP Tue Sep 19 17:28:18 UTC 2017 x86_64 x86_64 x86_64 GNU/Linux
alex@ubuntu16:~$ c++ --version
clang version 3.8.0-2ubuntu4 (tags/RELEASE_380/final)
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin

[Bjoe]

It is set here. The intent was to use the custom alignment code on pre-c++11 compilers, but as it is now, it will not come automatically into effect unless c++11 mode is enabled (ie. default c++11 build will use standard alignment code).

Ok. I was expected that I can choose if I will have POCO_HAVE_ALIGNMENT via a parameter on configure or cmake.

I wonder what happens with following scenarios:

When POCO library is building with C++11 parameter. POCO_HAVE_ALIGNMENT is activated and after the preprocessor the header of SocketAddress.h and SocketAddressImpl.h looks like:

SocketAddress.h

namespace Poco {
namespace Net {

class IPAddress;

class __attribute__ ((visibility ("default"))) SocketAddress
{
public:
 SocketAddress();

 ....
 
 Ptr pImpl() const;

 void newIPv4();

 void newIPv4(const sockaddr_in*);

 void newIPv4(const IPAddress& hostAddress, Poco::UInt16 portNumber);

 ....
 
 char* storage();

  AlignedCharArrayUnion <Poco::Net::Impl::IPv6SocketAddressImpl>

  _memory;
};

....

inline void SocketAddress::newIPv4()
{
 new (storage()) Poco::Net::Impl::IPv4SocketAddressImpl;
}

inline void SocketAddress::newIPv4(const sockaddr_in* sockAddr)
{
 new (storage()) Poco::Net::Impl::IPv4SocketAddressImpl(sockAddr);
}

inline void SocketAddress::newIPv4(const IPAddress& hostAddress, Poco::UInt16 portNumber)
{
 new (storage()) Poco::Net::Impl::IPv4SocketAddressImpl(hostAddress.addr(), htons(portNumber));
}
...

SocketAddressImpl.h

class __attribute__ ((visibility ("default"))) SocketAddressImpl



{
public:
 virtual ~SocketAddressImpl();

 virtual IPAddress host() const = 0;
 virtual UInt16 port() const = 0;
 virtual socklen_t length() const = 0;
 virtual const struct sockaddr* addr() const = 0;
 virtual int af() const = 0;

protected:
 SocketAddressImpl();

private:
 SocketAddressImpl(const SocketAddressImpl&);
 SocketAddressImpl& operator = (const SocketAddressImpl&);
};


class __attribute__ ((visibility ("default"))) IPv4SocketAddressImpl: public SocketAddressImpl
{
public:
 IPv4SocketAddressImpl();
 IPv4SocketAddressImpl(const struct sockaddr_in* addr);
 IPv4SocketAddressImpl(const void* addr, UInt16 port);
 IPAddress host() const;
 UInt16 port() const;
 socklen_t length() const;
 const struct sockaddr* addr() const;
 int af() const;
 IPAddress::Family family() const;
 std::string toString() const;

private:
 struct sockaddr_in _addr;
};

....

inline const struct sockaddr* IPv4SocketAddressImpl::addr() const
{
 return reinterpret_cast<const struct sockaddr*>(&_addr);
}
....

Now I build my project that uses POCO withouth C++11 parameter. POCO_HAVE_ALIGNMENT is not activated and after the preprocessor the header of SocketAddress.h and SocketAddressImpl.h looks like:

SocketAddress.h

namespace Poco {
namespace Net {

class IPAddress;

class __attribute__ ((visibility ("default"))) SocketAddress

{
public:
 SocketAddress();

...

private:
 typedef Poco::Net::Impl::SocketAddressImpl Impl;

 typedef Poco::AutoPtr<Impl> Ptr;

 Ptr pImpl() const;

 void newIPv4();

 void newIPv4(const sockaddr_in*);

 void newIPv4(const IPAddress& hostAddress, Poco::UInt16 portNumber);
 
 Ptr _pImpl;

};

...

inline SocketAddress::Ptr SocketAddress::pImpl() const
{
 if (_pImpl) return _pImpl;
 throw Poco::NullPointerException("Pointer to SocketAddress implementation is NULL.");
}

inline void SocketAddress::newIPv4()
{
 _pImpl = new Poco::Net::Impl::IPv4SocketAddressImpl;
}

inline void SocketAddress::newIPv4(const sockaddr_in* sockAddr)
{
 _pImpl = new Poco::Net::Impl::IPv4SocketAddressImpl(sockAddr);
}

inline void SocketAddress::newIPv4(const IPAddress& hostAddress, Poco::UInt16 portNumber)
{
 _pImpl = new Poco::Net::Impl::IPv4SocketAddressImpl(hostAddress.addr(), htons(portNumber));
}

SocketAddressImpl.h

namespace Poco {
namespace Net {
namespace Impl {


class __attribute__ ((visibility ("default"))) SocketAddressImpl

 : public Poco::RefCountedObject

{
public:
 virtual ~SocketAddressImpl();

 virtual IPAddress host() const = 0;
 virtual UInt16 port() const = 0;
 virtual socklen_t length() const = 0;
 virtual const struct sockaddr* addr() const = 0;
 virtual int af() const = 0;

protected:
 SocketAddressImpl();

private:
 SocketAddressImpl(const SocketAddressImpl&);
 SocketAddressImpl& operator = (const SocketAddressImpl&);
};


class __attribute__ ((visibility ("default"))) IPv4SocketAddressImpl: public SocketAddressImpl
{
public:
 IPv4SocketAddressImpl();
 IPv4SocketAddressImpl(const struct sockaddr_in* addr);
 IPv4SocketAddressImpl(const void* addr, UInt16 port);
 IPAddress host() const;
 UInt16 port() const;
 socklen_t length() const;
 const struct sockaddr* addr() const;
 int af() const;
 IPAddress::Family family() const;
 std::string toString() const;

private:
 struct sockaddr_in _addr;
};

...

inline const struct sockaddr* IPv4SocketAddressImpl::addr() const
{
 return reinterpret_cast<const struct sockaddr*>(&_addr);
}

Ok, newIPv4() functions are private and is only accessible via SocketAddress. So this function is compiled into the library and should never compiled in my project (Hopefully nobody will create this function public).

But what is with SocketAddressImpl? With POCO_HAVE_ALIGNMENT, SocketAddressImpl is not inherited from RefCountedObject. Without POCO_HAVE_ALIGNMENT, SocketAddressImpl is inherited from RefCountedObject.

From my example project, the compiler thinks that in Poco library SocketAddressImpl is inherited from RefCountedObject, but it is not. This mismatch affects only IPv4SocketAddressImpl::addr() because this function is compiled in my project. The constructor of IPv4SocketAddressImpl is compiled in the library.

BtW, POCO_HAVE_ALIGNMENT is not only depended on the c++11 compiler parameter. In Alignment.h there check which compiler is used. For example, if clang is used as compiler and they have the cxx_alignas feature, POCO_HAVE_ALIGNMENT is also set.

I wonder, how I know as a "library user" which implementation/parameter is used when I install POCO via the package manager of the OS? For example we have this issue with #1684. I know that FreeBSD is using as default compiler, clang in the build system. I don't know if Dian8 is using gcc as a default compiler, because it is possible on FreeBSD to change this localy.

It looks like the ABI of POCO libray is depended on POCO_HAVE_ALIGNMENT.
Maybe we have following solutions:

1. Move all dependend POCO_HAVE_ALIGNMENT headers/inline ... .h and .ipp ... like boost.
2. Move all dependend POCO_HAVE_ALIGNMENT "in source" files to "hide the implementation".
3. Create a config.h in the build process to "save" how the library is build. Like openssl or pjproject. This can still make problems...

Some other ideas ? Comments ?

[aleks-f]

@Bjoe This feature has caused more trouble than benefit; I'd say, let's make things clean and simple - get rid of all SOO for IPAddress and SocketAddress altogether for 1.8.0. And un-inline anything that may cause problems.

2.0 is c++11 and alignment availability is guaranteed by standard, so the *Impl RefCountedObject inheritance can be removed and only SOO version used. I'll clean up develop for 2.0

@obiltschnig ?

[aleks-f]

an alternative for 1.8: have SOO and no-SOO, but for no-SOO version remove inheriting from RefCountedObject and move reference counting functionality directly into *Impl classes (not much work) and un-inline everything, so these classes look always the same to the users, regardless of an application compile-time flags

[Bjoe]

I'd say, let's make things clean and simple

Full Ack! 👍

get rid of all SOO for IPAddress and SocketAddress altogether for 1.8.0.

👍

an alternative for 1.8: have SOO and no-SOO, but for no-SOO version remove inheriting from RefCountedObject move reference counting functionality directly into *Impl classes

That's also an alternative.

un-inline everything, so these classes look always the same to the users, regardless of an application compile-time flags

That's what I mean with: Move all dependend POCO_HAVE_ALIGNMENT "in source" files to "hide the implementation".

Or, let developer decide if he will have POCO_HAVE_ALIGNMENT or not via a parameter at the build system. Default is off so that POCO is compiled on every Linux/Unix OS'es without POCO_HAVE_ALIGNMENT and POCO works with every compiler.
But, I vote for '''let's make things clean and simple''' .... Is mostly the better approach.

Let me know which solution we will choose. I will help you.

Btw. should I close my merge request #1959 ?

[Bjoe]

Sorry, by accident I pressed wrong button ...

[aleks-f]

Let's remove SOO for both IPAddress and SocketAddress in 1.8

should I close my merge request #1959 ?

Either (a) close and create a new one, or (b) modify it in accordance with the above.

I'll take care of develop.

[aleks-f]

the correct related issue mentioned in commit was #234 , not #264

[Bjoe]

Let's remove SOO for both IPAddress and SocketAddress in 1.8

@aleks-f There is a pull request #1971

[aleks-f]

@Bjoe I'm ok with it, requested review from @obiltschnig; he is working on 1.8 release and we did not yet hear from him about this issue

[plainwu]

I'm using 1.9.0 release and I still got this exception.

Using Clang 4.0, and build on Linux aarch64 platform.

nvidia@tegra-ubuntu:~$ uname -a
Linux tegra-ubuntu 4.4.38-tegra #1 SMP PREEMPT Thu May 17 00:15:19 PDT 2018 aarch64 aarch64 aarch64 GNU/Linux

http://192.168.0.21:8800/media/memberFace/plain_Pig7hP1.jpg
Net Exception: Address family not supported

I tried to apply patch of #1950, but it didn't work.
Is there anyone seeing this issue?