Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Building for darwin on nix fails #516

Closed
nomeata opened this issue Oct 18, 2019 · 11 comments
Closed

Building for darwin on nix fails #516

nomeata opened this issue Oct 18, 2019 · 11 comments
Assignees
Labels
bug Incorrect behavior in the current implementation that needs fixing wasi:impl Issues pertaining to WASI implementation in Wasmtime

Comments

@nomeata
Copy link

nomeata commented Oct 18, 2019

Trying to build on nix for darwin, (see NixOS/nixpkgs#71316), I hit the following problem:

unpacking sources
unpacking source archive /nix/store/3ii33ri19pv2c092gzdr0vpjjqw1yrmj-source
source root is source
unpacking source archive /nix/store/g5zrf1aaw39w6cg7qfadfvrjy9cmky8a-wasmtime-20191018-vendor
patching sources
applying patch /nix/store/fh5c4jz5k0ny4fpmsiqkxk9qhqv2chh0-cargo-lock.patch
patching file Cargo.lock
configuring
building
++ env CC_x86_64-apple-darwin=/nix/store/hi40vi42w2rnf9pflyhmsalcj5jf2bsv-clang-wrapper-7.1.0/bin/cc CXX_x86_64-apple-darwin=/nix/store/hi40vi42w2rnf9pflyhmsalcj5jf2bsv-clang-wrapper-7.1.0/bin/c++ CC_x86_64-apple-darwin=/nix/store/hi40vi42w2rnf9pflyhmsalcj5jf2bsv-clang-wrapper-7.1.0/bin/cc CXX_x86_64-apple-darwin=/nix/store/hi40vi42w2rnf9pflyhmsalcj5jf2bsv-clang-wrapper-7.1.0/bin/c++ cargo build --release --target x86_64-apple-darwin --frozen
   Compiling log v0.4.8
   Compiling libc v0.2.64
   Compiling proc-macro2 v1.0.5
   Compiling unicode-xid v0.2.0
   Compiling cfg-if v0.1.10
   Compiling syn v1.0.5
   Compiling ryu v1.0.2
   Compiling serde v1.0.101
   Compiling semver-parser v0.7.0
   Compiling failure_derive v0.1.6
   Compiling proc-macro2 v0.4.30
   Compiling itoa v0.4.4
   Compiling unicode-xid v0.1.0
   Compiling cranelift-codegen-shared v0.46.1
   Compiling arrayvec v0.4.12
   Compiling memchr v2.2.1
   Compiling rustc-demangle v0.1.16
   Compiling syn v0.15.44
   Compiling lazy_static v1.4.0
   Compiling typenum v1.11.2
   Compiling glob v0.3.0
   Compiling nodrop v0.1.14
   Compiling byteorder v1.3.2
   Compiling scopeguard v1.0.0
   Compiling regex-syntax v0.6.12
   Compiling autocfg v0.1.6
   Compiling rayon-core v1.6.0
   Compiling smallvec v0.6.10
   Compiling quick-error v1.2.2
   Compiling byte-tools v0.3.1
   Compiling getrandom v0.1.12
   Compiling termcolor v1.0.5
   Compiling bitflags v1.2.1
   Compiling wasmtime-environ v0.2.0 (/private/tmp/nix-build-wasmtime-20191018.drv-0/source/wasmtime-environ)
   Compiling fake-simd v0.1.2
   Compiling plain v0.2.3
   Compiling ppv-lite86 v0.2.5
   Compiling unicode-segmentation v1.3.0
   Compiling wasmparser v0.39.2
   Compiling opaque-debug v0.2.3
   Compiling either v1.5.3
   Compiling fallible-iterator v0.2.0
   Compiling indexmap v1.2.0
   Compiling spin v0.5.2
   Compiling nix v0.15.0
   Compiling stable_deref_trait v1.1.1
   Compiling wasmparser v0.37.2
   Compiling void v1.0.2
   Compiling id-arena v2.2.1
   Compiling leb128 v0.2.4
   Compiling wasmtime v0.2.0 (/private/tmp/nix-build-wasmtime-20191018.drv-0/source)
   Compiling strsim v0.9.2
   Compiling wasi-common v0.1.0 (https://github.com/CraneStation/wasi-common?rev=c3bf040#c3bf0404)
   Compiling crossbeam-utils v0.6.6
   Compiling thread_local v0.3.6
   Compiling humantime v1.3.0
   Compiling block-padding v0.1.4
   Compiling semver v0.9.0
   Compiling bincode v1.2.0
   Compiling num-traits v0.2.8
   Compiling num-integer v0.1.41
   Compiling heck v0.3.1
   Compiling c2-chacha v0.2.2
   Compiling crossbeam-queue v0.1.2
   Compiling rustc_version v0.2.3
   Compiling base64 v0.10.1
   Compiling gimli v0.19.0
   Compiling aho-corasick v0.7.6
   Compiling generic-array v0.12.3
   Compiling num_cpus v1.10.1
   Compiling atty v0.2.13
   Compiling dirs-sys v0.3.4
   Compiling mach v0.2.3
   Compiling errno v0.2.4
   Compiling time v0.1.42
   Compiling filetime v0.2.7
   Compiling jobserver v0.1.17
   Compiling memoffset v0.5.1
   Compiling scroll v0.9.2
   Compiling directories v2.0.2
   Compiling digest v0.8.1
   Compiling block-buffer v0.7.3
   Compiling rand_core v0.5.1
   Compiling quote v0.6.13
   Compiling region v2.1.2
   Compiling sha2 v0.8.0
   Compiling quote v1.0.2
   Compiling rand_chacha v0.2.1
   Compiling cc v1.0.46
   Compiling chrono v0.4.9
   Compiling crossbeam-epoch v0.7.2
   Compiling rand v0.7.2
   Compiling crossbeam-deque v0.7.1
   Compiling rayon v1.2.0
   Compiling regex v1.3.1
   Compiling backtrace-sys v0.1.32
   Compiling zstd-sys v1.4.13+zstd.1.4.3
   Compiling cmake v0.1.42
   Compiling raw-cpuid v6.1.0
   Compiling wasmtime-runtime v0.2.0 (/private/tmp/nix-build-wasmtime-20191018.drv-0/source/wasmtime-runtime)
   Compiling scroll_derive v0.9.5
   Compiling wabt-sys v0.7.0
   Compiling backtrace v0.3.40
   Compiling goblin v0.0.24
   Compiling env_logger v0.6.2
   Compiling file-per-thread-logger v0.1.2
   Compiling pretty_env_logger v0.3.1
   Compiling serde_derive v1.0.101
   Compiling synstructure v0.12.1
   Compiling walrus-macro v0.12.0
   Compiling wasi-common-cbindgen v0.1.0 (https://github.com/CraneStation/wasi-common?rev=c3bf040#c3bf0404)
   Compiling failure v0.1.6
   Compiling zstd-safe v1.4.13+zstd.1.4.3
   Compiling zstd v0.4.28+zstd.1.4.3
error: linking with `/nix/store/hi40vi42w2rnf9pflyhmsalcj5jf2bsv-clang-wrapper-7.1.0/bin/cc` failed: exit code: 1
  |
  = note: "/nix/store/hi40vi42w2rnf9pflyhmsalcj5jf2bsv-clang-wrapper-7.1.0/bin/cc" "-m64" "-L" "/nix/store/r8ps3zwaxwam9bnw9i0n46z4qr2j7yzk-rustc-1.37.0/lib/rustlib/x86_64-apple-darwin/lib" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/wasi_common-cb01afa41255bfe8.wasi_common.2pp5p6tf-cgu.0.rcgu.o" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/wasi_common-cb01afa41255bfe8.wasi_common.2pp5p6tf-cgu.1.rcgu.o" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/wasi_common-cb01afa41255bfe8.wasi_common.2pp5p6tf-cgu.10.rcgu.o" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/wasi_common-cb01afa41255bfe8.wasi_common.2pp5p6tf-cgu.11.rcgu.o" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/wasi_common-cb01afa41255bfe8.wasi_common.2pp5p6tf-cgu.12.rcgu.o" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/wasi_common-cb01afa41255bfe8.wasi_common.2pp5p6tf-cgu.13.rcgu.o" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/wasi_common-cb01afa41255bfe8.wasi_common.2pp5p6tf-cgu.14.rcgu.o" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/wasi_common-cb01afa41255bfe8.wasi_common.2pp5p6tf-cgu.15.rcgu.o" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/wasi_common-cb01afa41255bfe8.wasi_common.2pp5p6tf-cgu.2.rcgu.o" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/wasi_common-cb01afa41255bfe8.wasi_common.2pp5p6tf-cgu.3.rcgu.o" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/wasi_common-cb01afa41255bfe8.wasi_common.2pp5p6tf-cgu.4.rcgu.o" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/wasi_common-cb01afa41255bfe8.wasi_common.2pp5p6tf-cgu.5.rcgu.o" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/wasi_common-cb01afa41255bfe8.wasi_common.2pp5p6tf-cgu.6.rcgu.o" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/wasi_common-cb01afa41255bfe8.wasi_common.2pp5p6tf-cgu.7.rcgu.o" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/wasi_common-cb01afa41255bfe8.wasi_common.2pp5p6tf-cgu.8.rcgu.o" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/wasi_common-cb01afa41255bfe8.wasi_common.2pp5p6tf-cgu.9.rcgu.o" "-o" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/libwasi_common-cb01afa41255bfe8.dylib" "-Wl,-exported_symbols_list,/private/tmp/nix-build-wasmtime-20191018.drv-0/rustcqP4NQ5/list" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/wasi_common-cb01afa41255bfe8.4mk5g2t6mk33db4q.rcgu.o" "-Wl,-dead_strip" "-nodefaultlibs" "-L" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps" "-L" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/release/deps" "-L" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/build/backtrace-sys-2d147214789255a0/out" "-L" "/nix/store/r8ps3zwaxwam9bnw9i0n46z4qr2j7yzk-rustc-1.37.0/lib/rustlib/x86_64-apple-darwin/lib" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/libnix-dfd91aca8c39020a.rlib" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/libvoid-9cdf6e4006aa038f.rlib" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/libbitflags-a7bbe403b9c1735b.rlib" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/librand-daaf851cf7868d45.rlib" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/librand_chacha-97fa38918d53e690.rlib" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/libc2_chacha-a0e1f0d831f8f9dd.rlib" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/libppv_lite86-2a1a38c81ee18cd3.rlib" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/liblazy_static-4cae960660a74250.rlib" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/librand_core-4e1a0c41489f9461.rlib" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/libgetrandom-28165c5b9affdbed.rlib" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/liblog-47c2bc685ba53e9c.rlib" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/libfiletime-e91ead56cad31a83.rlib" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/libfailure-7e4295e10a48e1af.rlib" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/libbacktrace-d1b25e4eda6cc091.rlib" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/libbacktrace_sys-38b2028395269248.rlib" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/liblibc-31f4ec99033d3060.rlib" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/libcfg_if-1ee9105a5976112b.rlib" "/private/tmp/nix-build-wasmtime-20191018.drv-0/source/target/x86_64-apple-darwin/release/deps/librustc_demangle-627c318ee7b04a5c.rlib" "/nix/store/r8ps3zwaxwam9bnw9i0n46z4qr2j7yzk-rustc-1.37.0/lib/rustlib/x86_64-apple-darwin/lib/libstd-c6ca9251a199aa27.rlib" "/nix/store/r8ps3zwaxwam9bnw9i0n46z4qr2j7yzk-rustc-1.37.0/lib/rustlib/x86_64-apple-darwin/lib/libpanic_unwind-39ec93fd4165c255.rlib" "/nix/store/r8ps3zwaxwam9bnw9i0n46z4qr2j7yzk-rustc-1.37.0/lib/rustlib/x86_64-apple-darwin/lib/libbacktrace-a948cb44f048597a.rlib" "/nix/store/r8ps3zwaxwam9bnw9i0n46z4qr2j7yzk-rustc-1.37.0/lib/rustlib/x86_64-apple-darwin/lib/libbacktrace_sys-9f041fb877b98c64.rlib" "/nix/store/r8ps3zwaxwam9bnw9i0n46z4qr2j7yzk-rustc-1.37.0/lib/rustlib/x86_64-apple-darwin/lib/librustc_demangle-3c4a5c551a922378.rlib" "/nix/store/r8ps3zwaxwam9bnw9i0n46z4qr2j7yzk-rustc-1.37.0/lib/rustlib/x86_64-apple-darwin/lib/libhashbrown-2c195470cc27ba70.rlib" "/nix/store/r8ps3zwaxwam9bnw9i0n46z4qr2j7yzk-rustc-1.37.0/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_alloc-38fc89345f49f6a1.rlib" "/nix/store/r8ps3zwaxwam9bnw9i0n46z4qr2j7yzk-rustc-1.37.0/lib/rustlib/x86_64-apple-darwin/lib/libunwind-e60112f814e80d3d.rlib" "/nix/store/r8ps3zwaxwam9bnw9i0n46z4qr2j7yzk-rustc-1.37.0/lib/rustlib/x86_64-apple-darwin/lib/libcfg_if-ebddb7552a11bbd5.rlib" "/nix/store/r8ps3zwaxwam9bnw9i0n46z4qr2j7yzk-rustc-1.37.0/lib/rustlib/x86_64-apple-darwin/lib/liblibc-46d8e16ab2813df4.rlib" "/nix/store/r8ps3zwaxwam9bnw9i0n46z4qr2j7yzk-rustc-1.37.0/lib/rustlib/x86_64-apple-darwin/lib/liballoc-349e6b1effd8559c.rlib" "/nix/store/r8ps3zwaxwam9bnw9i0n46z4qr2j7yzk-rustc-1.37.0/lib/rustlib/x86_64-apple-darwin/lib/librustc_std_workspace_core-1a46406c8cbb31ae.rlib" "/nix/store/r8ps3zwaxwam9bnw9i0n46z4qr2j7yzk-rustc-1.37.0/lib/rustlib/x86_64-apple-darwin/lib/libcore-2922fa7cbacf5093.rlib" "/nix/store/r8ps3zwaxwam9bnw9i0n46z4qr2j7yzk-rustc-1.37.0/lib/rustlib/x86_64-apple-darwin/lib/libcompiler_builtins-96d54cc5512fa6bf.rlib" "-lSystem" "-lresolv" "-lc" "-lm" "-dynamiclib" "-Wl,-dylib"
  = note: Undefined symbols for architecture x86_64:
            "_utimensat", referenced from:
                _$LT$$u5b$u8$u5d$$u20$as$u20$nix..NixPath$GT$::with_nix_path::h2c1b78ca714c817b in wasi_common-cb01afa41255bfe8.wasi_common.2pp5p6tf-cgu.12.rcgu.o
          ld: symbol(s) not found for architecture x86_64
          clang-7: error: linker command failed with exit code 1 (use -v to see invocation)


error: aborting due to previous error

error: Could not compile `wasi-common`.
warning: build failed, waiting for other jobs to finish...
error: build failed
builder for '/nix/store/6h89ffzlsxmxf8wp0abap155i9g8nw46-wasmtime-20191018.drv' failed with exit code 101

I read somewhere that on nix, certain symbols are removed from libsystem that are not available on all versions of MacOS. (See first two messages on https://logs.nix.samueldr.com/nix-darwin/2019-07-26)

@kubkon
Copy link
Member

kubkon commented Oct 18, 2019

Awesome, thanks for this! So, we do indeed use _utimensat in our codebase here. As far as I remember, _utimensat was added to darwin when macOS 13 was released, and before then, only _utimes was available. My proposition here is to try and use filetime crate which handles different darwin versions dynamically, which should hopefully fix the problem. I'd ask you to verify that it builds fine for you with this fix though. Would that be OK?

@nomeata
Copy link
Author

nomeata commented Oct 18, 2019

I have no idea about darwin and little idea about nix, but I am happy to test fixes.

@kubkon
Copy link
Member

kubkon commented Oct 18, 2019

OK, so I think there is a potential fix for this, but it requires some thought before committing to it. @alexcrichton @sunfishcode I'd very much appreciate your opinion on this. The potential fix, in principle, would require the same mechanism as the one used in @alexcrichton's filetime crate for checking whether a symbol exists at runtime:

static ADDR: AtomicUsize = AtomicUsize::new(0);
let func: Option<unsafe extern "C" fn(...)> = unsafe {
    fetch(&ADDR, b"utimensat\0").map(|sym| mem::transmute(sym))
};
if let Some(func) = func {
    // Success! "utimensat" symbol was found on our host
    // Call utimensat(...)
    func(fd, path, ...);
} else {
    // Sad days, "utimensat" is not available on this host
}

In case of the negative route, when utimensat symbol is not found on the host, filetime resorts back to utimes. However, our use case is slightly more constrained than filetime's in the sense that we always operate on a combination of a (pre)opened file descriptor and a relative path. Now, AFAIK there is no "at" version of utimes, i.e., something like utimesat, which makes things a little bit more complicated than I originally envisioned. We could improvise by calling openat on the provided pair (fd, rel_path), and then adjust filetimes using futimes syscall like so:

let fd = unsafe { libc::openat(dirfd, path, libc::O_NOFOLLOW | libc::O_RDWR) };
unsafe { libc::futimes(fd, times.as_ptr()) };

But this becomes somewhat messy, plus it adds up to two additional syscalls (one to get the file descriptor using openat, and second to read current values of atime and mtime -> no UTIME_OMIT or UTIME_NOW available if utimensat is not available :-().

Originally, I thought of extending filetime crate with this use case in mind, but I'm now I'm hesitant as it could potentially break the symmetry in implementation between Linux and BSD hosts.

Anyhow, the question here is should we support hosts without utimensat capabilities, or not?

@alexcrichton
Copy link
Member

I'm generally of the opinion that on recent (ish) versions of a platform is where a program should be the most optimized, and if a slower polyfill is required on older platforms that's better than not having the functionality at all. In that sense if the polyfill for older platforms requires some hoops and a few more syscalls I personally think that's ok.

Also FWIW if this is on Linux I'd recommend using libc::syscall since that is generally a bit more robust than dlopen in the face of static linking and musl. It's only available on Linux though!

@kubkon
Copy link
Member

kubkon commented Oct 18, 2019

I'm generally of the opinion that on recent (ish) versions of a platform is where a program should be the most optimized, and if a slower polyfill is required on older platforms that's better than not having the functionality at all. In that sense if the polyfill for older platforms requires some hoops and a few more syscalls I personally think that's ok.

Makes sense! :-)

Also FWIW if this is on Linux I'd recommend using libc::syscall since that is generally a bit more robust than dlopen in the face of static linking and musl. It's only available on Linux though!

Yep, absolutely. I was going to basically steal much of the "meat" from your lib, filetime and that would include both Linux and BSD implementations. I guess I'll prepare a draft PR with a potential resolution of the issue and cc all of you guys in, and coordinate with @nomeata for some testing, if that's OK of course! How does that sound for everyone?

@kubkon kubkon self-assigned this Oct 18, 2019
@sunfishcode
Copy link
Member

That sounds like a good approach to me!

@marmistrz
Copy link
Contributor

What's done in filetime for utimensat and futimens sounds like a quite repetitive piece of code, which may be useful in different crates.

How about writing some macro that would automatically generate the Option<unsafe extern "C" fn(...)> getters and publishing it as a crate? Alternatively, you could write a function, generic over the function type, which would accept the symbol name and return Option<unsafe extern "C" fn(...)>.

Idea:

maybe_function!("futimens", fn(c_int, *const timespec) -> c_int);

or

let futimens = maybe_function<fn(c_int, *const timespec) -> c_int>("futimens");

I'm not sure if we can create convert fn to unsafe extern "C" fn, though.

@nomeata
Copy link
Author

nomeata commented Nov 4, 2019

@kubkon, were you able to make progress here?

@kubkon
Copy link
Member

kubkon commented Nov 4, 2019

Hey @nomeata, apologies but I didn't have time to properly sit down to it yet. Having said that I've got some more time this week, so hopefully, I'll get most of it (if not all) done this week. The moment I get something working and reviewed, I shall let you know and kindly ask for your help in verifying that the changes are actually correct.

@kubkon kubkon transferred this issue from CraneStation/wasi-common Nov 8, 2019
@kubkon kubkon added bug Incorrect behavior in the current implementation that needs fixing wasi:impl Issues pertaining to WASI implementation in Wasmtime labels Nov 8, 2019
kubkon pushed a commit to kubkon/wasmtime that referenced this issue Nov 10, 2019
This commit introduces a change to file time management for *nix based
hosts in that it firstly tries to load `utimensat` symbol, and if it
doesn't exist, then falls back to `utimes` instead. This change is
borrowing very heavily from [filetime] crate, however, it introduces a
couple of helpers and methods specific to WASI use case (or more
generally, to a use case which requires modifying times of entities
specified by a pair `(DirFD, RelativePath)` rather than the typical
file time specification based only absolute path or raw file descriptor
as is the case with [filetime] crate. The trick here is, that on kernels
which do not have `utimensat` symbol, this implementation emulates this
behaviour by a combination of `openat` and `utimes`.

This commit also is meant to address bytecodealliance#516.

[filetime]: https://github.com/alexcrichton/filetime
kubkon pushed a commit to kubkon/wasmtime that referenced this issue Nov 10, 2019
This commit introduces a change to file time management for *nix based
hosts in that it firstly tries to load `utimensat` symbol, and if it
doesn't exist, then falls back to `utimes` instead. This change is
borrowing very heavily from [filetime] crate, however, it introduces a
couple of helpers and methods specific to WASI use case (or more
generally, to a use case which requires modifying times of entities
specified by a pair `(DirFD, RelativePath)` rather than the typical
file time specification based only absolute path or raw file descriptor
as is the case with [filetime] crate. The trick here is, that on kernels
which do not have `utimensat` symbol, this implementation emulates this
behaviour by a combination of `openat` and `utimes`.

This commit also is meant to address bytecodealliance#516.

[filetime]: https://github.com/alexcrichton/filetime
kubkon pushed a commit to kubkon/wasmtime that referenced this issue Nov 10, 2019
This commit introduces a change to file time management for *nix based
hosts in that it firstly tries to load `utimensat` symbol, and if it
doesn't exist, then falls back to `utimes` instead. This change is
borrowing very heavily from [filetime] crate, however, it introduces a
couple of helpers and methods specific to WASI use case (or more
generally, to a use case which requires modifying times of entities
specified by a pair `(DirFD, RelativePath)` rather than the typical
file time specification based only absolute path or raw file descriptor
as is the case with [filetime] crate. The trick here is, that on kernels
which do not have `utimensat` symbol, this implementation emulates this
behaviour by a combination of `openat` and `utimes`.

This commit also is meant to address bytecodealliance#516.

[filetime]: https://github.com/alexcrichton/filetime
@kubkon
Copy link
Member

kubkon commented Nov 10, 2019

Hi @nomeata, could you do me a favour and try building wasmtime using kubkon/wasmtime/tree/nixos_utimensat branch, and see if this fixes your issue?

kubkon pushed a commit to kubkon/wasmtime that referenced this issue Nov 11, 2019
This commit introduces a change to file time management for *nix based
hosts in that it firstly tries to load `utimensat` symbol, and if it
doesn't exist, then falls back to `utimes` instead. This change is
borrowing very heavily from [filetime] crate, however, it introduces a
couple of helpers and methods specific to WASI use case (or more
generally, to a use case which requires modifying times of entities
specified by a pair `(DirFD, RelativePath)` rather than the typical
file time specification based only absolute path or raw file descriptor
as is the case with [filetime] crate. The trick here is, that on kernels
which do not have `utimensat` symbol, this implementation emulates this
behaviour by a combination of `openat` and `utimes`.

This commit also is meant to address bytecodealliance#516.

[filetime]: https://github.com/alexcrichton/filetime
alexcrichton pushed a commit that referenced this issue Nov 11, 2019
* Dynamically load utimensat if exists on the host

This commit introduces a change to file time management for *nix based
hosts in that it firstly tries to load `utimensat` symbol, and if it
doesn't exist, then falls back to `utimes` instead. This change is
borrowing very heavily from [filetime] crate, however, it introduces a
couple of helpers and methods specific to WASI use case (or more
generally, to a use case which requires modifying times of entities
specified by a pair `(DirFD, RelativePath)` rather than the typical
file time specification based only absolute path or raw file descriptor
as is the case with [filetime] crate. The trick here is, that on kernels
which do not have `utimensat` symbol, this implementation emulates this
behaviour by a combination of `openat` and `utimes`.

This commit also is meant to address #516.

[filetime]: https://github.com/alexcrichton/filetime

* Fix symlink NOFOLLOW flag setting

* Add docs and specify UTIME_NOW/OMIT on Linux

Previously, we relied on [libc] crate for `UTIME_NOW` and `UTIME_OMIT`
constants on Linux. However, following the convention assumed in
[filetime] crate, this is now changed to directly specified by us
in our crate.

[libc]: https://github.com/rust-lang/libc
[filetime]: https://github.com/alexcrichton/filetime

* Refactor UTIME_NOW/OMIT for BSD

* Address final discussion points
@kubkon
Copy link
Member

kubkon commented Nov 11, 2019

I believe #535 fixed this issue as confirmed by @nomeata in #535, so closing :-)

@kubkon kubkon closed this as completed Nov 11, 2019
matthewbauer pushed a commit to NixOS/nixpkgs that referenced this issue Nov 26, 2019
With this bump, we pull in support for building on darwin, i.e.
bytecodealliance/wasmtime#516
dtzWill pushed a commit to dtzWill/nixpkgs that referenced this issue Jan 7, 2020
With this bump, we pull in support for building on darwin, i.e.
bytecodealliance/wasmtime#516

(cherry picked from commit 3ce8c48)
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Incorrect behavior in the current implementation that needs fixing wasi:impl Issues pertaining to WASI implementation in Wasmtime
Projects
None yet
Development

No branches or pull requests

5 participants