Tips for using Ninja to build NanoSDK/NanoMQ with msquic on MacOS

[alvin1221]

Describe the bug
Failed to use Ninja to build NanoMQ with msquic on MacOS.

../src/supplemental/quic/quic_api.c:18:10: fatal error: 'openssl/pem.h' file not found
#include "openssl/pem.h"
         ^~~~~~~~~~~~~~~
2 warnings and 1 error generated.

To Reproduce

  mkdir build 
  cd build
  cmake -G Ninja -DNNG_ENABLE_QUIC=ON ..

** Environment Details **

• NanoSDK/NNG version : latest
• Operating system and version: MacOS
• Compiler and language used: C/C++
• Shared or static library
• Remote MQTT Broker info
• Messaging scenario

Additional context

FAILED: CMakeFiles/nng_testing.dir/src/supplemental/quic/quic_api.c.o 
/Library/Developer/CommandLineTools/usr/bin/cc -DNNG_ENABLE_STATS -DNNG_HAVE_ARC4RANDOM=1 -DNNG_HAVE_BACKTRACE=1 -DNNG_HAVE_BUS0 -DNNG_HAVE_FLOCK=1 -DNNG_HAVE_GETPEEREID=1 -DNNG_HAVE_KQUEUE=1 -DNNG_HAVE_LANGINFO=1 -DNNG_HAVE_LOCALPEERCRED=1 -DNNG_HAVE_LOCALPEERPID=1 -DNNG_HAVE_LOCKF=1 -DNNG_HAVE_MQTT_CLIENT -DNNG_HAVE_MQTT_QUIC_CLIENT -DNNG_HAVE_MSG_CONTROL=1 -DNNG_HAVE_PAIR0 -DNNG_HAVE_PAIR1 -DNNG_HAVE_PTHREAD_ATFORK_PTHREAD=1 -DNNG_HAVE_PTHREAD_SETNAME_NP=1 -DNNG_HAVE_PUB0 -DNNG_HAVE_PULL0 -DNNG_HAVE_PUSH0 -DNNG_HAVE_REP0 -DNNG_HAVE_REQ0 -DNNG_HAVE_RESPONDENT0 -DNNG_HAVE_SEMAPHORE_PTHREAD=1 -DNNG_HAVE_STDATOMIC=1 -DNNG_HAVE_STRCASECMP=1 -DNNG_HAVE_STRLCPY=1 -DNNG_HAVE_STRNCASECMP=1 -DNNG_HAVE_STRNLEN=1 -DNNG_HAVE_SUB0 -DNNG_HAVE_SURVEYOR0 -DNNG_HAVE_UNIX_SOCKETS=1 -DNNG_HIDDEN_VISIBILITY -DNNG_MAX_TASKQ_THREADS=16 -DNNG_PLATFORM_DARWIN -DNNG_PLATFORM_POSIX -DNNG_PRIVATE -DNNG_STATIC_LIB -DNNG_SUPP_HTTP -DNNG_SUPP_QUIC -DNNG_TEST_LIB -DNNG_TRANSPORT_INPROC -DNNG_TRANSPORT_IPC -DNNG_TRANSPORT_MQTT_TCP -DNNG_TRANSPORT_MQTT_TLS -DNNG_TRANSPORT_TCP -DNNG_TRANSPORT_TLS -DNNG_TRANSPORT_WS -DNNG_TRANSPORT_WSS -DQUIC_TLS_ENGINE_FINI=nng_quic_libs_fini_msquic -DQUIC_TLS_ENGINE_INIT=nng_quic_libs_init_msquic -DSUPP_QUIC -D_GNU_SOURCE -D_POSIX_PTHREAD_SEMANTICS -D_REENTRANT -D_THREAD_SAFE -I../src -I_deps/opensslquic-build/openssl/include -I../extern/msquic/src/inc -I../src/testing -I../include -Wall -Wextra -fno-omit-frame-pointer   -O3 -DNDEBUG -isysroot /Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk -std=gnu99 -MD -MT CMakeFiles/nng_testing.dir/src/supplemental/quic/quic_api.c.o -MF CMakeFiles/nng_testing.dir/src/supplemental/quic/quic_api.c.o.d -o CMakeFiles/nng_testing.dir/src/supplemental/quic/quic_api.c.o -c ../src/supplemental/quic/quic_api.c
In file included from ../src/supplemental/quic/quic_api.c:11:
../extern/msquic/src/inc/msquic.h:28:9: warning: unknown pragma ignored [-Wunknown-pragmas]
#pragma warning(disable:4201)  // nonstandard extension used: nameless struct/union
        ^
../extern/msquic/src/inc/msquic.h:29:9: warning: unknown pragma ignored [-Wunknown-pragmas]
#pragma warning(disable:4214)  // nonstandard extension used: bit field types other than int
        ^
../src/supplemental/quic/quic_api.c:18:10: fatal error: 'openssl/pem.h' file not found
#include "openssl/pem.h"
         ^~~~~~~~~~~~~~~
2 warnings and 1 error generated.
[215/469] OpenSSL configure
Operating system: x86_64-apple-darwinDarwin Kernel Version 22.3.0: Mon Jan 30 20:42:11 PST 2023; root:xnu-8792.81.3~2/RELEASE_X86_64
WARNING! If you wish to build 32-bit libraries, then you have to
         invoke 'KERNEL_BITS=32 /Users/huanghuanhua/Projects/github/NanoSDK/extern/msquic/submodules/openssl/config  'enable-tls1_3' 'no-makedepend' 'no-dgram' 'no-ssl3' 'no-psk' 'no-srp' 'no-zlib' 'no-egd' 'no-idea' 'no-rc5' 'no-rc4' 'no-afalgeng' 'no-comp' 'no-cms' 'no-ct' 'no-srp' 'no-srtp' 'no-ts' 'no-gost' 'no-dso' 'no-ec2m' 'no-tls1' 'no-tls1_1' 'no-tls1_2' 'no-dtls' 'no-dtls1' 'no-dtls1_2' 'no-ssl' 'no-ssl3-method' 'no-tls1-method' 'no-tls1_1-method' 'no-tls1_2-method' 'no-dtls1-method' 'no-dtls1_2-method' 'no-siphash' 'no-whirlpool' 'no-aria' 'no-bf' 'no-blake2' 'no-sm2' 'no-sm3' 'no-sm4' 'no-camellia' 'no-cast' 'no-md4' 'no-mdc2' 'no-ocb' 'no-rc2' 'no-rmd160' 'no-scrypt' 'no-seed' 'no-weak-ssl-ciphers' 'no-shared' 'no-tests' '--openssldir=/private/etc/ssl' '--prefix=/Users/huanghuanhua/Projects/github/NanoSDK/build/_deps/opensslquic-build/openssl' '-isysroot' '/Library/Developer/CommandLineTools/SDKs/MacOSX13.1.sdk''.
Configuring OpenSSL version 1.1.1t+quic (0x1010114fL) for darwin64-x86_64-cc

[wanghaEMQ]

Hi @alvin1221

NanoMQ relies on MsQuic and the fork OpenSSL in MsQuic. The building order and strategy of Ninja on MacOS are different from those on Linux. On Linux. Ninja will build the fork OpenSSL before building MsQuic. But on MacOS it doesn't.

A temporary solution is using make -j1 instead of Ninja.

mkdir build 
cd build
cmake  -DNNG_ENABLE_QUIC=ON ..
make -j1

[alvin1221]

Hi @alvin1221

NanoMQ relies on MsQuic and the fork OpenSSL in MsQuic. The building order and strategy of Ninja on MacOS are different from those on Linux. On Linux. Ninja will build the fork OpenSSL before building MsQuic. But on MacOS it doesn't.

A temporary solution is using make -j1 instead of Ninja.

mkdir build 
cd build
cmake  -DNNG_ENABLE_QUIC=ON ..
make -j1

It works, thanks @wanghaEMQ