pthread_atfork for android API < 21

[petrovr]

Functions pthread_atfork is not defined if Android API < 12(a) and is not declared if API < 21(b).
If first case (a) build(link) fail.

What about to exclude code that use pthread_atfork "#if ANDROID_API < 21" ?

[richsalz]

Please merge this into your other Android 21 issue.

[petrovr]

Rich Salz wrote:

Please merge this into your other Android 21 issue.

Actually for [12-21) function is not declared only. Roumen

[misery]

Same problem here with openssl-1.1.1-pre6 .... is it safe to use "no-threads" if we do not use any API of https://www.openssl.org/docs/man1.0.2/crypto/threads.html or is it still a problem if we use multiple threads in our application?

crypto/threads_pthread.c: In function 'fork_once_func':
crypto/threads_pthread.c:221:5: warning: implicit declaration of function 'pthread_atfork' [-Wimplicit-function-declaration]
     pthread_atfork(OPENSSL_fork_prepare,
     ^
arm-linux-androideabi-ar: creating libssl.a
arm-linux-androideabi-ar: creating libcrypto.a
./libcrypto.so: error: undefined reference to 'pthread_atfork'

$ perl configdata.pm --dump

Command line (with current working directory = .):

    /usr/bin/perl ./Configure --prefix=/home/bos/tmp/ak/build/dist --debug no-tls1_3 no-ssl3 no-ssl3-method no-dtls no-camellia no-aria no-seed no-poly1305 no-srp no-gost no-ocsp no-idea no-mdc2 no-rc5 no-srtp no-hw no-engine no-async no-dso no-comp no-ts no-makedepend no-tests shared -D__ANDROID_API__=18 no-stdio android-arm -fstack-protector-strong -mfloat-abi=softfp

Perl information:

    /usr/bin/perl
    5.26.1 for x86_64-linux-thread-multi

Enabled features:

    asm
    autoalginit
    autoerrinit
    autoload-config
    bf
    blake2
    cast
    chacha
    cmac
    cms
    ct
    deprecated
    des
    dgram
    dh
    dsa
    ec
    ec2m
    ecdh
    ecdsa
    err
    filenames
    hw(-.+)?
    md4
    multiblock
    nextprotoneg
    ocb
    pic
    posix-io
    psk
    rc2
    rc4
    rdrand
    rfc3779
    rmd160
    scrypt
    shared
    siphash
    sm2
    sm3
    sm4
    sock
    sse2
    ssl
    static-engine
    threads
    tls
    ui-console
    whirlpool
    tls1
    tls1-method
    tls1_1
    tls1_1-method
    tls1_2
    tls1_2-method
    dtls1-method
    dtls1_2-method

Disabled features:

    afalgeng                [forced]              OPENSSL_NO_AFALGENG
    aria                    [option]              OPENSSL_NO_ARIA (skip crypto/aria)
    asan                    [default]             OPENSSL_NO_ASAN
    async                   [option]              OPENSSL_NO_ASYNC
    camellia                [option]              OPENSSL_NO_CAMELLIA (skip crypto/camellia)
    capieng                 [forced]              OPENSSL_NO_CAPIENG
    comp                    [option]              OPENSSL_NO_COMP (skip crypto/comp)
    crypto-mdebug           [default]             OPENSSL_NO_CRYPTO_MDEBUG
    crypto-mdebug-backtrace [default]             OPENSSL_NO_CRYPTO_MDEBUG_BACKTRACE
    devcryptoeng            [default]             OPENSSL_NO_DEVCRYPTOENG
    dso                     [option]              
    dtls                    [option(dtls)]        OPENSSL_NO_DTLS
    dynamic-engine          [forced]              
    ec_nistp_64_gcc_128     [default]             OPENSSL_NO_EC_NISTP_64_GCC_128
    egd                     [default]             OPENSSL_NO_EGD
    engine                  [option]              OPENSSL_NO_ENGINE (skip crypto/engine, engines)
    external-tests          [default]             OPENSSL_NO_EXTERNAL_TESTS
    fuzz-libfuzzer          [default]             OPENSSL_NO_FUZZ_LIBFUZZER
    fuzz-afl                [default]             OPENSSL_NO_FUZZ_AFL
    gost                    [option]              OPENSSL_NO_GOST
    heartbeats              [default]             OPENSSL_NO_HEARTBEATS
    idea                    [option]              OPENSSL_NO_IDEA (skip crypto/idea)
    makedepend              [option]              
    md2                     [default]             OPENSSL_NO_MD2 (skip crypto/md2)
    mdc2                    [option]              OPENSSL_NO_MDC2 (skip crypto/mdc2)
    msan                    [default]             OPENSSL_NO_MSAN
    ocsp                    [option]              OPENSSL_NO_OCSP (skip crypto/ocsp)
    poly1305                [option]              OPENSSL_NO_POLY1305 (skip crypto/poly1305)
    rc5                     [option]              OPENSSL_NO_RC5 (skip crypto/rc5)
    sctp                    [default]             OPENSSL_NO_SCTP
    seed                    [option]              OPENSSL_NO_SEED (skip crypto/seed)
    srp                     [option]              OPENSSL_NO_SRP (skip crypto/srp)
    srtp                    [option]              OPENSSL_NO_SRTP
    ssl-trace               [default]             OPENSSL_NO_SSL_TRACE
    stdio                   [option]              OPENSSL_NO_STDIO
    tests                   [option]              OPENSSL_NO_TESTS
    tls13downgrade          [default]             OPENSSL_NO_TLS13DOWNGRADE
    ts                      [option]              OPENSSL_NO_TS (skip crypto/ts)
    ubsan                   [default]             OPENSSL_NO_UBSAN
    unit-test               [default]             OPENSSL_NO_UNIT_TEST
    weak-ssl-ciphers        [default]             OPENSSL_NO_WEAK_SSL_CIPHERS
    zlib                    [default]             
    zlib-dynamic            [default]             
    ssl3                    [option(ssl3-method)] OPENSSL_NO_SSL3
    ssl3-method             [option]              OPENSSL_NO_SSL3_METHOD
    tls1_3                  [option]              OPENSSL_NO_TLS1_3
    dtls1                   [option(dtls)]        OPENSSL_NO_DTLS1
    dtls1_2                 [option(dtls)]        OPENSSL_NO_DTLS1_2

Config target attributes:

    AR => "ar",
    ARFLAGS => "r",
    CC => "gcc",
    CFLAGS => "-Wall -O0 -g",
    CXX => "g++",
    CXXFLAGS => "-Wall -O0 -g",
    HASHBANGPERL => "/usr/bin/env perl",
    RANLIB => "ranlib",
    RC => "windres",
    aes_asm_src => "aes_cbc.c aes-armv4.S bsaes-armv7.S aesv8-armx.S",
    aes_obj => "aes_cbc.o aes-armv4.o bsaes-armv7.o aesv8-armx.o",
    apps_aux_src => "",
    apps_init_src => "",
    apps_obj => "",
    bf_asm_src => "bf_enc.c",
    bf_obj => "bf_enc.o",
    bin_cflags => "-pie",
    bn_asm_src => "bn_asm.c armv4-mont.S armv4-gf2m.S",
    bn_obj => "bn_asm.o armv4-mont.o armv4-gf2m.o",
    bn_ops => "BN_LLONG RC4_CHAR",
    build_file => "Makefile",
    build_scheme => [ "unified", "unix" ],
    cast_asm_src => "c_enc.c",
    cast_obj => "c_enc.o",
    cflags => "-pthread -Wa,--noexecstack -mandroid --sysroot=\$(ANDROID_NDK)/platforms/android-18/arch-arm",
    chacha_asm_src => "chacha-armv4.S",
    chacha_obj => "chacha-armv4.o",
    cmll_asm_src => "camellia.c cmll_misc.c cmll_cbc.c",
    cmll_obj => "camellia.o cmll_misc.o cmll_cbc.o",
    cppflags => "",
    cpuid_asm_src => "armcap.c armv4cpuid.S",
    cpuid_obj => "armcap.o armv4cpuid.o",
    cxxflags => "-std=c++11 -pthread -Wa,--noexecstack -mandroid --sysroot=\$(ANDROID_NDK)/platforms/android-18/arch-arm",
    defines => [  ],
    des_asm_src => "des_enc.c fcrypt_b.c",
    des_obj => "des_enc.o fcrypt_b.o",
    disable => [  ],
    dso_extension => ".so",
    dso_scheme => "dlfcn",
    ec_asm_src => "ecp_nistz256.c ecp_nistz256-armv4.S",
    ec_obj => "ecp_nistz256.o ecp_nistz256-armv4.o",
    enable => [  ],
    ex_libs => "-ldl -pthread",
    exe_extension => "",
    includes => [  ],
    keccak1600_asm_src => "keccak1600-armv4.S",
    keccak1600_obj => "keccak1600-armv4.o",
    lflags => "",
    lib_cflags => "",
    lib_cppflags => "-DOPENSSL_USE_NODELETE",
    lib_defines => [  ],
    md5_asm_src => "",
    md5_obj => "",
    modes_asm_src => "ghash-armv4.S ghashv8-armx.S",
    modes_obj => "ghash-armv4.o ghashv8-armx.o",
    module_cflags => "-fPIC",
    module_cxxflags => "",
    module_ldflags => "-Wl,-znodelete -shared -Wl,-Bsymbolic",
    padlock_asm_src => "",
    padlock_obj => "",
    perlasm_scheme => "void",
    poly1305_asm_src => "poly1305-armv4.S",
    poly1305_obj => "poly1305-armv4.o",
    rc4_asm_src => "rc4_enc.c rc4_skey.c",
    rc4_obj => "rc4_enc.o rc4_skey.o",
    rc5_asm_src => "rc5_enc.c",
    rc5_obj => "rc5_enc.o",
    rmd160_asm_src => "",
    rmd160_obj => "",
    sha1_asm_src => "sha1-armv4-large.S sha256-armv4.S sha512-armv4.S",
    sha1_obj => "sha1-armv4-large.o sha256-armv4.o sha512-armv4.o",
    shared_cflag => "-fPIC",
    shared_defflag => "-Wl,--version-script=",
    shared_defines => [  ],
    shared_extension => ".so.\$(SHLIB_VERSION_NUMBER)",
    shared_extension_simple => ".so",
    shared_ldflag => "-Wl,-znodelete -shared -Wl,-Bsymbolic",
    shared_rcflag => "",
    shared_sonameflag => "-Wl,-soname=",
    shared_target => "linux-shared",
    thread_defines => [  ],
    thread_scheme => "pthreads",
    unistd => "<unistd.h>",
    uplink_aux_src => "",
    uplink_obj => "",
    wp_asm_src => "wp_block.c",
    wp_obj => "wp_block.o",

Recorded environment:

    AR = 
    BUILDFILE = 
    CC = 
    CROSS_COMPILE = 
    CXX = 
    HASHBANGPERL = 
    OPENSSL_LOCAL_CONFIG_DIR = 
    PERL = 
    RANLIB = 
    RC = 
    WINDRES = 
    __CNF_CFLAGS = 
    __CNF_CPPDEFINES = 
    __CNF_CPPFLAGS = 
    __CNF_CPPINCLUDES = 
    __CNF_CXXFLAGS = 
    __CNF_LDFLAGS = 
    __CNF_LDLIBS = 

Makevars:

    AR              = arm-linux-androideabi-ar
    ARFLAGS         = r
    CC              = arm-linux-androideabi-gcc
    CFLAGS          = -Wall -O0 -g -fstack-protector-strong -mfloat-abi=softfp
    CPPDEFINES      = __ANDROID_API__=18
    CPPFLAGS        = 
    CPPINCLUDES     = 
    CROSS_COMPILE   = arm-linux-androideabi-
    CXX             = arm-linux-androideabi-g++
    CXXFLAGS        = -Wall -O0 -g -fstack-protector-strong -mfloat-abi=softfp
    HASHBANGPERL    = /usr/bin/env perl
    LDFLAGS         = 
    LDLIBS          = 
    RANLIB          = arm-linux-androideabi-ranlib
    RC              = arm-linux-androideabi-windres

NOTE: These variables only represent the configuration view.  The build file
template may have processed these variables further, please have a look at the
build file for more exact data:
    Makefile

build file:

    Makefile

build file templates:

    Configurations/common0.tmpl
    Configurations/unix-Makefile.tmpl
    Configurations/common.tmpl

[gjasny]

Using Android NDK r16b with unified headers does not have that problem whereby r10e shows the error.

[mattcaswell]

What is to be done with this issue? This has the 1.1.1 milestone against it.

[petrovr]

Main question is how to correct. Another question is about minimum supported Android version.
Solutions depend from both.
I wonder if use of week alias is acceptable solution.

More up to date information . Android API 23 (Marshmallow v6.0) adds "__register_atfork" but still keep "pthread_atfork" visible for backward compatibility.

[mattcaswell]

This looks to me like an NDK bug. I've been testing various versions of the NDK and see different behaviour between them. The latest versions of the NDK do not have the problem at all - you can compile without warning or errors and target the oldest supported API level (14) without issue.

When I tested NDK 10e (the oldest version I could get my hands on), I found that at API level 12, I got the warning about the implicit declaration and it failed to link because of the missing definition.

I also tested NDK 14b. With this version API level 12 and 14 saw the implicit declaration warning, but linking succeeded. However at API level 9 (there is no 10 or 11 in the NDK...not sure why) linking failed.

NDK 15c behaves in the same way as 14b.

So, it seems to me that this isn't really an OpenSSL issue at all. The answer is to upgrade your NDK version. That does imply that we cannot support API versions lower than 12 because the bug is not fixed in API levels that old. If you want warning free OpenSSL builds then you have to use one of the latest NDK versions (which further restricts you to API version 14 or above). However since Google themselves do not support API versions this old, I'm not sure why we should have to.

[misery]

Problem seems to be in gcc compiler bugs in latest NDKs. Qt recommends to be compiled with NDK 10e (GCC 4.9).
We uses API 18 as minimum (Android 4.3) API. It results into this problem. So we are forced to Android 5.0 (API 21).

android/ndk#67
android/ndk#26
https://wiki.qt.io/Qt_for_Android_known_issues

[petrovr]

What about to resolve issue at run-time? Approach is not new to OpenSSL. It is used in CAPI engine - dialog functionality if I remember well.

For android to use:
(1) __register_atfork : should be available if runtime API is >=23 (Marshmallow/6.0)
(2) pthread_atfork : should be available if runtime API is >=12 (Honeycomb, but if version is 3.1)
(3) error case (old platform). Perhaps OpenSSL master (1.1.1) should exit with error is such case.

Any thoughts?

[mattcaswell]

@misery wrote:

Problem seems to be in gcc compiler bugs in latest NDKs. Qt recommends to be compiled with NDK 10e (GCC 4.9).

Any reason you can't use a different NDK version to compile OpenSSL vs Qt?

@petrovr wrote:

What about to resolve issue at run-time?

Why would we do that? It seems that is adding significant extra complexity into OpenSSL for something which is an NDK problem to solve (and has been in the latest NDK releases).

[misery]

We tried NDK r15 sometime ago and got crashes for arm64 builds. I think it will be possible to switch to clang of newest NDK someday. But this needs a little bit more testing in next release cycle. But since clang is supported by Qt for Android we need NDK r10e (gcc).

[mattcaswell]

I am closing this on the basis that I do not think this is an issue for OpenSSL to solve. The problem exists in NDK, and suitable workarounds exist.

[earonesty]

atexit/atfork ndk: support info

android/ndk#132

[mattcaswell]

I don't see anything in the linked issue referencing atfork?

[richsalz]

It is a shame Android removed this. What do you think openssl should do? A pull request would be helpful.