Fails to build on Apple Silicon M2

[mouse07410]

Problem:

Unable to generate bindings.: ClangDiagnostic("/Users/ur20980/.cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.13.0/aws-lc/include/openssl/err.h:112:10: fatal error: 'stdio.h' file not found\n")

It happens during build of https://github.com/rustls/rustls.git which tries to build dependencies aws-lc-rs -> aws-lc-sys.

The issue has been reported here and recommended to bring up in this repo, because it appears to be this crate's problem.
Steps to reproduce are shown there.

Relevant details

AWS-LC for Rust versions or commit: v1.6 (as far as I can tell)

System information: for linux, below info can be collected by running uname -srvmp

• CPU architecture: ARM8
• CPU name: Apple Silicon M2 Max
• OS: MacOS Sonoma 14.3.1
• Xcode: 15.2
• Rust: 1.76.0 (stable)

Build log:

• The log tells compiler and version.

$ git clone --recurse-submodules https://github.com/rustls/rustls.git
Cloning into 'rustls'...
remote: Enumerating objects: 26917, done.
remote: Counting objects: 100% (4049/4049), done.
remote: Compressing objects: 100% (1112/1112), done.
remote: Total 26917 (delta 3270), reused 3122 (delta 2912), pack-reused 22868
Receiving objects: 100% (26917/26917), 10.23 MiB | 2.53 MiB/s, done.
Resolving deltas: 100% (18838/18838), done.
$ cd rustls
$ cargo test --release; date                                         
   Compiling libc v0.2.153
   Compiling proc-macro2 v1.0.78
   Compiling unicode-ident v1.0.12
   Compiling glob v0.3.1
   Compiling rustix v0.38.31
   Compiling prettyplease v0.2.16
   Compiling memchr v2.7.1
   Compiling regex-syntax v0.8.2
   Compiling minimal-lexical v0.2.1
   Compiling either v1.10.0
   Compiling cfg-if v1.0.0
   Compiling bitflags v2.4.2
   Compiling libloading v0.8.1
   Compiling bindgen v0.69.4
   Compiling home v0.5.9
   Compiling itertools v0.12.1
   Compiling lazy_static v1.4.0
   Compiling rustc-hash v1.1.0
   Compiling nom v7.1.3
   Compiling clang-sys v1.7.0
   Compiling lazycell v1.3.0
   Compiling shlex v1.3.0
   Compiling log v0.4.20
   Compiling dunce v1.0.4
   Compiling fs_extra v1.3.0
   Compiling paste v1.0.14
   Compiling aho-corasick v1.1.2
   Compiling regex-automata v0.4.5
   Compiling aws-lc-rs v1.6.1
   Compiling untrusted v0.9.0
   Compiling cexpr v0.6.0
   Compiling rustls-pki-types v1.2.0
   Compiling mirai-annotations v1.12.0
   Compiling zeroize v1.7.0
   Compiling base64 v0.21.7
   Compiling rustls v0.23.0-alpha.0 (/Users/ur20980/src/rustls/rustls)
   Compiling once_cell v1.19.0
   Compiling regex v1.10.3
   Compiling is-terminal v0.4.12
   Compiling subtle v2.5.0
   Compiling termcolor v1.4.1
   Compiling humantime v2.1.0
   Compiling quote v1.0.35
   Compiling syn v2.0.48
   Compiling rustls-pemfile v2.0.0
   Compiling webpki-roots v0.26.1
   Compiling autocfg v1.1.0
   Compiling powerfmt v0.2.0
   Compiling bencher v0.1.5
   Compiling deranged v0.3.11
   Compiling num-traits v0.2.18
   Compiling num-bigint v0.4.4
   Compiling time-core v0.1.2
   Compiling cc v1.0.83
   Compiling errno v0.3.8
   Compiling num-conv v0.1.0
   Compiling serde v1.0.196
   Compiling cmake v0.1.50
   Compiling time v0.3.34
   Compiling which v4.4.2
   Compiling getrandom v0.2.12
   Compiling ring v0.17.7
   Compiling spin v0.9.8
   Compiling yasna v0.5.2
   Compiling pem v3.0.3
   Compiling socket2 v0.5.5
   Compiling env_logger v0.10.2
   Compiling num-integer v0.1.46
   Compiling mio v0.8.10
   Compiling pin-project-lite v0.2.13
   Compiling strsim v0.10.0
   Compiling bytes v1.5.0
   Compiling serde_derive v1.0.196
   Compiling tokio-macros v2.2.0
   Compiling tokio v1.36.0
   Compiling aws-lc-sys v0.13.0
   Compiling rcgen v0.12.1
error: failed to run custom build command for `aws-lc-sys v0.13.0`
note: To improve backtraces for build dependencies, set the CARGO_PROFILE_RELEASE_BUILD_OVERRIDE_DEBUG=true environment variable to enable debug information generation.

Caused by:
  process didn't exit successfully: `/Users/ur20980/src/rustls/target/release/build/aws-lc-sys-de27243413207beb/build-script-main` (exit status: 101)
  --- stdout
  cargo:rustc-cfg=use_bindgen_generated

  --- stderr
  /Users/ur20980/.cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.13.0/aws-lc/include/openssl/err.h:112:10: fatal error: 'stdio.h' file not found
  thread 'main' panicked at /Users/ur20980/.cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.13.0/builder/bindgen.rs:151:10:
  Unable to generate bindings.: ClangDiagnostic("/Users/ur20980/.cargo/registry/src/index.crates.io-6f17d22bba15001f/aws-lc-sys-0.13.0/aws-lc/include/openssl/err.h:112:10: fatal error: 'stdio.h' file not found\n")
  stack backtrace:
     0: rust_begin_unwind
               at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/std/src/panicking.rs:645:5
     1: core::panicking::panic_fmt
               at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/panicking.rs:72:14
     2: core::result::unwrap_failed
               at /rustc/07dca489ac2d933c78d3c5158e3f43beefeb02ce/library/core/src/result.rs:1649:5
     3: core::result::Result<T,E>::expect
     4: build_script_main::bindgen::generate_bindings
     5: build_script_main::generate_bindings
     6: build_script_main::main
     7: core::ops::function::FnOnce::call_once
  note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
warning: build failed, waiting for other jobs to finish...
Tue Feb 13 16:35:59 EST 2024
$

[justsmth]

I'm sorry about this. Because we bind to the AWS-LC (libcrypto) library, we must generate Rust bindings for its API. For Linux (x86, x86-64, aarch64) and Mac (x86-64), these bindings are pre-generated. However, other platforms need to enable our "bindgen" feature so that the binding are created at build-time.

One possible work-around is to take a direct dependency on aws-lc-rs and enable the "bindgen" feature. This may also require the installation of libclang/llvm. Our User Guide provides basic instructions for this.

[justsmth]

I'm having trouble reproducing this on an M2 Mac. I am curious whether you might be cross-compiling to a different platform?

Is your default toolchain stable-aarch64-apple-darwin?

% rustup toolchain list
stable-aarch64-apple-darwin (default)

[justsmth]

Another helpful data point might be your XCode active developer directory.

% xcode-select --print-path
/Library/Developer/CommandLineTools

[mouse07410]

For Linux (x86, x86-64, aarch64) and Mac (x86-64), these bindings are pre-generated. However, other platforms need to enable our "bindgen" feature so that the binding are created at build-time.

Since I'm not installing aws-lc-rs directly (and it's a library crate, so direct install doesn't seem possible anyway), I have no control over what parameters rustls passes to it when it builds it as a dependency. :-(

One possible work-around is to take a direct dependency on aws-lc-rs and enable the "bindgen" feature. This may also require the installation of libclang/llvm. Our User Guide provides basic instructions for this.

I already have LLVM and Clang installed via Macports, they appear to be fine (and live in /opt/local).

Is your default toolchain stable-aarch64-apple-darwin? Another helpful data point might be your XCode active developer directory.

$ rustup toolchain list
stable-aarch64-apple-darwin (default)
nightly-aarch64-apple-darwin
$ xcode-select --print-path
/Applications/Xcode.app/Contents/Developer
$ xcode-select  --version
xcode-select version 2405.
$ clang -v
Apple clang version 15.0.0 (clang-1500.1.0.2.5)
Target: arm64-apple-darwin23.3.0
Thread model: posix
InstalledDir: /Applications/Xcode.app/Contents/Developer/Toolchains/XcodeDefault.xctoolchain/usr/bin
$ 

[thynson]

Failed to build it too, but with a different log:

error[E0425]: cannot find function, tuple struct or tuple variant `BIO_ctrl` in this scope
  --> /PATH/TO/SOMEWHERE/aws-lc-sys-0.13.1/src/lib.rs:82:14
   |
82 |     unsafe { BIO_ctrl(b, BIO_CTRL_INFO, 0, pp.cast::<c_void>()) }
   |              ^^^^^^^^ not found in this scope

error[E0425]: cannot find function, tuple struct or tuple variant `CRYPTO_library_init` in this scope
  --> /PATH/TO/SOMEWHERE/aws-lc-sys-0.13.1/src/lib.rs:86:14
   |
86 |     unsafe { CRYPTO_library_init() }
   |              ^^^^^^^^^^^^^^^^^^^ not found in this scope

For more information about this error, try `rustc --explain E0425`.

[cpu]

Failed to build it too, but with a different log:

It looks like your log matches the one reported separately in #348

[yerke]

@justsmth is it possible to add macos aarch64 to the list of pre-generated targets? A lot of developers are using Apple M1/2/3 computers, and the number of affected people (who will have to build from scratch) will only be increasing. Thanks!

[justsmth]

@justsmth is it possible to add macos aarch64 to the list of pre-generated targets? A lot of developers are using Apple M1/2/3 computers, and the number of affected people (who will have to build from scratch) will only be increasing. Thanks!

Yes! :-)

This is something we've started working. I hope to have something ready to PR for it soon.

[justsmth]

Closing issue. For latest discussion of issue see: rust-lang/rust-bindgen#2401