unable to compile syn

[mobileink]

I'm new to Rust but not to Bazel. I've spent most of the past two days trying to compile syn. I've learned a lot, but I admit defeat.

MWE using unroll, which depends on syn:

~/tmp/unroll/WORKSPACE:

load("@bazel_tools//tools/build_defs/repo:http.bzl", "http_archive")
http_archive(name = "io_bazel_rules_rust",
             url = "https://github.com/bazelbuild/rules_rust/archive/57c6f4ec6b25b07958d91c658d0142dc0943e848.tar.gz",
             strip_prefix = "rules_rust-57c6f4ec6b25b07958d91c658d0142dc0943e848",
             sha256 = "8ceeeedcd55f444dce73494ad01f0d20551ff68d91ab655bf98043a672c65300"
)
http_archive(name = "bazel_skylib",
             sha256 = "9a737999532daca978a158f94e77e9af6a6a169709c0cee274f0a4c3359519bd",
             strip_prefix = "bazel-skylib-1.0.0",
             url = "https://github.com/bazelbuild/bazel-skylib/archive/1.0.0.tar.gz",
)
load("@io_bazel_rules_rust//rust:repositories.bzl", "rust_repositories")
rust_repositories(version = "1.43.1")
load("@io_bazel_rules_rust//:workspace.bzl", "bazel_version")
bazel_version(name = "bazel_version")

~/tmp/unroll/cargo/Cargo.toml:

[package]
name = "compile_with_bazel"
version = "0.1.0"
[dependencies]
unroll = "0.1.4"
[lib]
path = "fake_lib.rs"
[raze]
workspace_path = "//cargo"
target = "x86_64-unknown-linux-gnu"
output_buildfile_suffix = "BUILD.bazel"
[raze.crates.proc-macro2.'0.4.30']
gen_buildrs = true
[raze.crates.syn.'0.15.44']
gen_buildrs = true

Run cargo vendor --versioned-dirs && cargo raze from the cargo dir. Then cd cargo/vendor/syn-0.15.44 && cargo build. Builds ok. Then try building with Bazel, from ~/tmp/unroll: $ bazel build //cargo/vendor/syn-0.15.44:syn. Fails with

error[E0277]: the trait bound `proc_macro2::TokenStream: std::convert::From<proc_macro::TokenStream>` is not satisfied
   --> cargo/vendor/syn-0.15.44/src/buffer.rs:109:20
    |
109 |         Self::new2(stream.into())
    |                    ^^^^^^^^^^^^^ the trait `std::convert::From<proc_macro::TokenStream>` is not implemented for `proc_macro2::TokenStream`

Ditto if you try to build unroll. If you remove proc-macro from the syn build targets, it builds ok, but then unroll breaks with a different error.

I've tried a hundred variations on this, compared the compile commands, etc. and nothing works. I'm out of ideas, I hope somebody can help.

Thanks,

Gregg

[kornholi]

This smells like gen_buildrs not working correctly with proc-macro2 (could you check what cargo-raze generates?).

I'd try manually configuring the cfg flags:

[raze.crates.proc-macro2.'xxx']
additional_flags = [
    "--cfg=use_proc_macro",
    "--cfg=wrap_proc_macro",
]

[mobileink]

On Fri, May 29, 2020 at 5:20 PM Kornelijus Survila ***@***.***> wrote: This smells like gen_buildrs not working correctly with proc-macro2 (could you check what cargo-raze generates?). I'd try manually configuring the cfg flags: [raze.crates.proc-macro2.'xxx'] additional_flags = [ "--cfg=use_proc_macro", "--cfg=wrap_proc_macro", ]

Thanks, but I'm afraid that would not work; those flags must be passed as "--cfg foo" (no = sign), as I found by examining the output of `cargo build`. The only way I could get it working was by implementing a cfg_flags attribute in rules_rust - bazelbuild/rules_rust#321 The other thing that was giving me fits turned out to be the rust edition - rules_rust defaults to 2015; setting it to 2018 fixed a whole bunch of problems. Live and learn. Gregg

[kornholi]

I just tried your example and it builds fine after my suggested change:

$ bazel build //cargo/vendor/syn-0.15.44:syn
INFO: Invocation ID: 3c00aff8-7e90-4676-9324-f498474e64d4
INFO: Analyzed target //cargo/vendor/syn-0.15.44:syn (4 packages loaded, 95 targets configured).
INFO: Found 1 target...
Target //cargo/vendor/syn-0.15.44:syn up-to-date:
  bazel-out/k8-fastbuild-ST-5e74b77704d3a70b08875590eb0f067cbb9a6e09f41f090f307cf0d79d4b2461/bin/cargo/vendor/syn-0.15.44/libsyn-158068785.rlib
INFO: Elapsed time: 9.685s, Critical Path: 9.27s
INFO: 3 processes: 3 linux-sandbox.
INFO: Build completed successfully, 4 total actions

It did fail with the same error beforehand. We don't really ever use gen_buildrs but I don't think it can propagate compiler options due to Bazel's architecture.

1. Perhaps you forgot to run cargo raze after changing Cargo.toml?
2. Are you running latest/recent cargo-raze? Your comment about the editions makes me suspicious as edition related issues were fixed a long time ago.

those flags must be passed as
"--cfg foo" (no = sign), as I found by examining the output of cargo build.

--cfg foo and --cfg=foo are equivalent in the eyes of rustc

[UebelAndre]

Why is this necessary? It seems like there are a few issues about being unable to compile without needing to add some flags to specific dependencies. Is there an issue outlining why that is?

Maybe @acmcarther might know?

[GregBowyer]

Sorry I just noticed this, the issue with needed flags typically comes from build.rs files.

If you look at proc_macro2 you will find a build.rs like so https://github.com/alexcrichton/proc-macro2/blob/master/build.rs

This file is run by cargo to configure the build, run generators, do autoconf style workloads. Even if you were to run the build.rs (gen_buildrs = true) there is (to my knowledge) no good way of getting those flags into the bazel rules for the rust target (and even if there was, its kinda a way to kiss goodbye to being hermetic).

This ship sailed in cargo and is known to rust and us (https://internals.rust-lang.org/t/build-script-capabilities/8635).

Initially AFAIK the idea was to avoid C style #ifdef-o-rama, e.g. just how do you detect in a sane way you are x86 64bit solaris with say a none typically libc? However what I think happened was we wound up reimplementing it in many crates in subtle and different ways.

There is an RFC to work towards addressing this gap for the whole does my environment look like rustc using llvm on windows type deal (https://github.com/rust-lang/rfcs/blob/master/text/2523-cfg-path-version.md)

With that I would guess that 80% of build.rs work would vanish as people ould start using the following over feature flags and build.rs env sniffing.

#[cfg(accessible(::std::u128))]
impl From<u128> for MyFancyCryptoThing {}

For the record I thing the other 10% of build.rs work is typically generating source codes which is reasonably handled with gen_buildrs = True, 8% is native compilation (pkg-config, cc etc) and 2% is a hedge of mine for anything just .. well ... weird.

For a reference of all the things that bite people during a build #41

I have a crazy side project to infer the bulk of these flags using symbolic execution but I doubt I will complete it any time soon.

[dayfine]

Any update on this? I might have hit this when trying to use https://crates.io/crates/strum_macros which depends on syn.

ERROR: /private/var/tmp/_bazel_difan/f78d9d7b4392e86afba7ecb9acf5e2e3/external/raze__strum_macros__0_24_2/BUILD.bazel:34:16: Compiling Rust proc-macro strum_macros v0.24.2 (19 files) failed: (Exit 1): process_wrapper failed: error executing command
  (cd /private/var/tmp/_bazel_difan/f78d9d7b4392e86afba7ecb9acf5e2e3/sandbox/darwin-sandbox/219/execroot/xlab && \
  exec env - \
    APPLE_SDK_PLATFORM=MacOSX \
    APPLE_SDK_VERSION_OVERRIDE=12.3 \
    CARGO_CFG_TARGET_ARCH=x86_64 \
    CARGO_CFG_TARGET_OS=darwin \
    CARGO_CRATE_NAME=strum_macros \
    CARGO_MANIFEST_DIR='${pwd}/external/raze__strum_macros__0_24_2' \
    CARGO_PKG_AUTHORS='' \
    CARGO_PKG_DESCRIPTION='' \
    CARGO_PKG_HOMEPAGE='' \
    CARGO_PKG_NAME=strum_macros \
    CARGO_PKG_VERSION=0.24.2 \
    CARGO_PKG_VERSION_MAJOR=0 \
    CARGO_PKG_VERSION_MINOR=24 \
    CARGO_PKG_VERSION_PATCH=2 \
    CARGO_PKG_VERSION_PRE='' \
    SYSROOT=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/rust_darwin_x86_64/toolchain_for_x86_64-apple-darwin_impl \
    XCODE_VERSION_OVERRIDE=13.4.1.13F100 \
    ZERO_AR_DATE=1 \
  bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/rules_rust/util/process_wrapper/process_wrapper --arg-file bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__proc_macro2__1_0_40/proc_macro2_build_script.linksearchpaths --arg-file bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__quote__1_0_20/quote_build_script.linksearchpaths --arg-file bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__syn__1_0_98/syn_build_script.linksearchpaths --arg-file bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__rustversion__1_0_7/rustversion_build_script.linksearchpaths --subst 'pwd=${pwd}' -- bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/rust_darwin_x86_64/toolchain_for_x86_64-apple-darwin_impl/bin/rustc external/raze__strum_macros__0_24_2/src/lib.rs '--crate-name=strum_macros' '--crate-type=proc-macro' '--error-format=human' '--codegen=metadata=-225036374' '--codegen=extra-filename=-225036374' '--out-dir=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__strum_macros__0_24_2' '--codegen=opt-level=3' '--codegen=debuginfo=0' '--remap-path-prefix=${pwd}=.' '--emit=dep-info,link' '--color=always' '--target=x86_64-apple-darwin' -L bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/rust_darwin_x86_64/toolchain_for_x86_64-apple-darwin_impl/lib/rustlib/x86_64-apple-darwin/lib '--cap-lints=allow' '--edition=2018' '--codegen=linker=external/local_config_cc/cc_wrapper.sh' --codegen 'link-args=-lc++ -fobjc-link-runtime -headerpad_max_install_names -lSystem -lresolv -lSystem -lresolv -lSystem -lresolv -lSystem -lresolv -no-canonical-prefixes -target x86_64-apple-macosx -mmacosx-version-min=12.3 -lc++ -target x86_64-apple-macosx' '--extern=heck=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__heck__0_4_0/libheck-3309503150.rlib' '--extern=proc_macro2=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__proc_macro2__1_0_40/libproc_macro2-2982902350.rlib' '--extern=quote=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__quote__1_0_20/libquote-1640270559.rlib' '--extern=syn=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__syn__1_0_98/libsyn-487446987.rlib' '--extern=rustversion=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__rustversion__1_0_7/librustversion-1507692243.dylib' '-Ldependency=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__heck__0_4_0' '-Ldependency=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__unicode_ident__1_0_1' '-Ldependency=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__proc_macro2__1_0_40' '-Ldependency=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__quote__1_0_20' '-Ldependency=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__syn__1_0_98' '-Ldependency=bazel-out/darwin-opt-exec-2B5CBBC6/bin/external/raze__rustversion__1_0_7' --extern proc_macro)
# Configuration: 096ff4561e7e561745a0b7194853903b6ad0770355c59136f4208699a5df52ed
# Execution platform: @local_config_platform//:host

Use --sandbox_debug to see verbose messages from the sandbox
error[E0432]: unresolved import `syn::parse`
 --> external/raze__strum_macros__0_24_2/src/helpers/case_style.rs:6:5
  |
6 |     parse::{Parse, ParseStream},
  |     ^^^^^ could not find `parse` in `syn`

[bsilver8192]

Except for recent breakage fixed by #524, this works for me now (I know because I first noticed that problem with syn breaking strum_macros).

I think the work since this issue was filed around features fixed it, and @dayfine 's recent comment is due to the recent breakage.