package: update vdirsyncer to account for the new rust dependancy #33050

teto · 2017-12-25T13:56:52Z

Issue description

I am having some issues with vdirsyncer pimutils/vdirsyncer#691, the author asked me to test the latest version so I looked into upgrading the package but the setup.py seems to compile a rust shared library so I am not sure how to tackle that (and I don't know rust):
https://github.com/pimutils/vdirsyncer/blob/master/setup.py#L41

I believe it would be best if nix could build the rust dependancy separately but I am not sure enough to actually ask vdirsyncer author to change that just for nixos.

@matthiasbeyer @jgeerds

matthiasbeyer · 2017-12-25T14:01:39Z

I'm on it.

matthiasbeyer · 2017-12-25T14:26:58Z

Ok. This will get messy and ugly... I doubt that I will succeed in this, especially as I do not have the resources to build rustc from source right now. Maybe @jgeerds can come up with a good idea.

pbogdan · 2017-12-25T17:56:14Z

It's super hacky but at least this manages to build - https://github.com/NixOS/nixpkgs/compare/master...pbogdan:pr_33052?expand=1 (based on the referenced PR).
I don't actually use the package myself so not quite sure how to test it doesn't fail at runtime. Also the current version of rustc from nixpkgs seems to panic a fair amount when building so the build may need to be restarted a few times before it succeeds :-(.
@teto perhaps at the very least that would let you test if the new version fixes your issue? (assuming it doesn't fail at runtime because of my hacks).

teto · 2017-12-26T02:56:29Z

Thanks for the help boys !

I tried the patch which fails on PATH="$HOME/.cargo/bin/:$PATH" . I guess this part could be removed or replaced with $out/bin


$ nix-shell -p vdirsyncer                                                                                                                                                                                                                               ~/nixpkgs
these derivations will be built:
  /nix/store/wg6laqc5zlxil69w1n8wlshbig1q22i3-vdirsyncer-0.17.0a1.drv
building path(s) ‘/nix/store/zay1w7lach4gznhnxca9ji7idsz9ih0q-vdirsyncer-0.17.0a1’
bash: warning: setlocale: LC_ALL: cannot change locale (en_US.utf8)
unpacking sources
unpacking source archive /nix/store/2krb64npcsaix238ki403sxd7vv2n032-vdirsyncer-0.17.0a1.tar.gz
source root is vdirsyncer-0.17.0a1
setting SOURCE_DATE_EPOCH to timestamp 1512415031 of file vdirsyncer-0.17.0a1/vdirsyncer/version.py
patching sources
configuring
building
running bdist_wheel
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/vdirsyncer
copying vdirsyncer/__main__.py -> build/lib.linux-x86_64-3.6/vdirsyncer
copying vdirsyncer/version.py -> build/lib.linux-x86_64-3.6/vdirsyncer
copying vdirsyncer/utils.py -> build/lib.linux-x86_64-3.6/vdirsyncer
copying vdirsyncer/native.py -> build/lib.linux-x86_64-3.6/vdirsyncer
copying vdirsyncer/vobject.py -> build/lib.linux-x86_64-3.6/vdirsyncer
copying vdirsyncer/exceptions.py -> build/lib.linux-x86_64-3.6/vdirsyncer
copying vdirsyncer/metasync.py -> build/lib.linux-x86_64-3.6/vdirsyncer
copying vdirsyncer/_native__ffi.py -> build/lib.linux-x86_64-3.6/vdirsyncer
copying vdirsyncer/repair.py -> build/lib.linux-x86_64-3.6/vdirsyncer
copying vdirsyncer/http.py -> build/lib.linux-x86_64-3.6/vdirsyncer
copying vdirsyncer/_native.py -> build/lib.linux-x86_64-3.6/vdirsyncer
copying vdirsyncer/__init__.py -> build/lib.linux-x86_64-3.6/vdirsyncer
creating build/lib.linux-x86_64-3.6/vdirsyncer/storage
copying vdirsyncer/storage/memory.py -> build/lib.linux-x86_64-3.6/vdirsyncer/storage
copying vdirsyncer/storage/dav.py -> build/lib.linux-x86_64-3.6/vdirsyncer/storage
copying vdirsyncer/storage/base.py -> build/lib.linux-x86_64-3.6/vdirsyncer/storage
copying vdirsyncer/storage/google.py -> build/lib.linux-x86_64-3.6/vdirsyncer/storage
copying vdirsyncer/storage/singlefile.py -> build/lib.linux-x86_64-3.6/vdirsyncer/storage
copying vdirsyncer/storage/filesystem.py -> build/lib.linux-x86_64-3.6/vdirsyncer/storage
copying vdirsyncer/storage/http.py -> build/lib.linux-x86_64-3.6/vdirsyncer/storage
copying vdirsyncer/storage/etesync.py -> build/lib.linux-x86_64-3.6/vdirsyncer/storage
copying vdirsyncer/storage/__init__.py -> build/lib.linux-x86_64-3.6/vdirsyncer/storage
creating build/lib.linux-x86_64-3.6/vdirsyncer/cli
copying vdirsyncer/cli/utils.py -> build/lib.linux-x86_64-3.6/vdirsyncer/cli
copying vdirsyncer/cli/tasks.py -> build/lib.linux-x86_64-3.6/vdirsyncer/cli
copying vdirsyncer/cli/discover.py -> build/lib.linux-x86_64-3.6/vdirsyncer/cli
copying vdirsyncer/cli/fetchparams.py -> build/lib.linux-x86_64-3.6/vdirsyncer/cli
copying vdirsyncer/cli/config.py -> build/lib.linux-x86_64-3.6/vdirsyncer/cli
copying vdirsyncer/cli/__init__.py -> build/lib.linux-x86_64-3.6/vdirsyncer/cli
creating build/lib.linux-x86_64-3.6/vdirsyncer/sync
copying vdirsyncer/sync/status.py -> build/lib.linux-x86_64-3.6/vdirsyncer/sync
copying vdirsyncer/sync/exceptions.py -> build/lib.linux-x86_64-3.6/vdirsyncer/sync
copying vdirsyncer/sync/__init__.py -> build/lib.linux-x86_64-3.6/vdirsyncer/sync
[ "$READTHEDOCS" != "True" ] || make install-rust
cd ./rust && PATH="$HOME/.cargo/bin/:$PATH" cargo build --release
error: failed to open: /homeless-shelter/.cargo/registry/index/github.com-1ecc6299db9ec823/.cargo-index-lock

Caused by:
  Permission denied (os error 13)
make: *** [Makefile:146: rust-ext] Error 101
builder for ‘/nix/store/wg6laqc5zlxil69w1n8wlshbig1q22i3-vdirsyncer-0.17.0a1.drv’ failed with exit code 2
error: build of ‘/nix/store/wg6laqc5zlxil69w1n8wlshbig1q22i3-vdirsyncer-0.17.0a1.drv’ failed

pbogdan · 2017-12-26T13:30:40Z

Was that from my branch? I thought https://github.com/NixOS/nixpkgs/compare/master...pbogdan:pr_33052?expand=1#diff-fa94f0db0527b004ae3809fa214083a0R40 ought to prevent that and the rust part would be built separately.

teto · 2017-12-26T14:37:14Z

Right my bad. I freaked out when I clicked on your link that proposed to create a new PR so I clicked away, and I ended up testing matthias PR :<

I tested yours which goes a bit farther. I ve tried twice with the same error. I am in sandboxed mode on ixos-unstable so that might be why ? I can try rerunning with RUST_BACKTRACE=1 if that helps. Thanks for your work.

installing ‘vdirsyncer-0.17.0a1’
these derivations will be built:
  /nix/store/alvzcahav063a8ivpy6w23pgp19qbssa-vdirsyncer-0.17.0a1-native.drv
  /nix/store/6xixijkqaqhpvn0nd5anmd56m2km7dsr-vdirsyncer-0.17.0a1.drv
building path(s) ‘/nix/store/c47mprb5gg0ln26k8ny08n94rggjjc22-vdirsyncer-0.17.0a1-native’
unpacking sources
unpacking source archive /nix/store/2krb64npcsaix238ki403sxd7vv2n032-vdirsyncer-0.17.0a1.tar.gz
source root is vdirsyncer-0.17.0a1/rust
unpacking source archive /nix/store/gv2hi9ig9ji7m4m7q8h86nh9z5ahlka9-vdirsyncer-0.17.0a1-native-vendor
setting SOURCE_DATE_EPOCH to timestamp 1511808735 of file vdirsyncer-0.17.0a1/rust/src/item.rs
patching sources
configuring
building
Running cargo build --release 
   Compiling cfg-if v0.1.2
   Compiling quote v0.3.15
   Compiling dtoa v0.4.2
   Compiling ansi_term v0.9.0
   Compiling gcc v0.3.54
   Compiling futures v0.1.17
   Compiling vec_map v0.8.0
   Compiling bitflags v0.9.1
   Compiling serde v0.9.15
   Compiling cc v1.0.3
   Compiling untrusted v0.5.1
   Compiling num-traits v0.1.40
   Compiling either v1.3.0
   Compiling scopeguard v0.3.3
   Compiling rayon-core v1.2.1
   Compiling itoa v0.3.4
   Compiling rustc-demangle v0.1.5
   Compiling log v0.3.8
   Compiling strsim v0.6.0
   Compiling unicode-width v0.1.4
   Compiling libc v0.2.33
   Compiling lazy_static v0.2.9
   Compiling unicode-xid v0.0.4
   Compiling coco v0.1.1
   Compiling textwrap v0.9.0
   Compiling backtrace-sys v0.1.16
   Compiling synom v0.11.3
   Compiling atty v0.2.3
   Compiling rand v0.3.17
   Compiling num_cpus v1.7.0
   Compiling syn v0.11.11
   Compiling toml v0.3.2
   Compiling serde_json v0.9.10
   Compiling clap v2.27.1
   Compiling tempdir v0.3.5
   Compiling rayon v0.8.2
thread '<unnamed>' panicked at 'failed to acquire jobserver token: Resource temporarily unavailable (os error 11)', src/librustc_trans/back/write.rs:1420:28
note: Run with `RUST_BACKTRACE=1` for a backtrace.
error: failed to acquire jobserver token: Resource temporarily unavailable (os error 11)

error: Could not compile `rayon`.
warning: build failed, waiting for other jobs to finish...
error: build failed
note: keeping build directory ‘/tmp/nix-build-vdirsyncer-0.17.0a1-native.drv-1’
builder for ‘/nix/store/alvzcahav063a8ivpy6w23pgp19qbssa-vdirsyncer-0.17.0a1-native.drv’ failed with exit code 101

pbogdan · 2017-12-26T17:51:08Z

Yes, unfortunately rustc seems to panic quite a lot in nondeterministic fashion :-(. I'm not sure why that is but I eventually succeeded building the package (with sandboxing enabled) after retrying a few times.
The panics happen with other packages too so I don't think it's something specific to vdirsyncer but other than retrying I don't know of any other work around.

teto · 2017-12-28T03:26:50Z

Indeed it crashes at different steps yet I could not reach the end of the build progress despite many attempts. I guess I need to pay attention for newer rust versions too.
I will try compiling several batches of fewer rust packages at at time to reduce the risk of failures, like first installing

 Compiling cfg-if v0.1.2
   Compiling quote v0.3.15
   Compiling dtoa v0.4.2
   Compiling ansi_term v0.9.0
   Compiling gcc v0.3.54

then

Compiling futures v0.1.17
   Compiling vec_map v0.8.0
   Compiling bitflags v0.9.1
   Compiling serde v0.9.15
   Compiling cc v1.0.3
   Compiling untrusted v0.5.1

etc so that once vdirsyncer installs dependancies they are already compiled.

Ralith · 2018-01-19T06:24:55Z

This is being widely reported, especially under Nix for some reason. See rust-lang/rust#46981

bennofs · 2018-01-31T09:07:33Z

I am working on this, it appears to be that our rust build is using an outdated version of the jobserver crate which had this bug.

teto · 2018-03-05T07:48:24Z

@pbogdan after successfully installing another rsut-based package, I tried your patch and it worked.
I hope it can make it into nixpkgs through either Mathias' or your PR 👍

teto · 2018-04-06T08:00:31Z

@bennofs any news :) ?

bennofs · 2018-04-06T09:29:30Z

@teto I think this should be fixed on master, since it uses a newer rustc release (1.24)

teto · 2018-04-25T06:03:31Z

PR merged. Thanks for your help

matthiasbeyer mentioned this issue Dec 25, 2017

Update vdirsyncer #33052

Closed

8 tasks

taku0 mentioned this issue Jan 20, 2018

firefox-bin: 57.0.4 -> 58.0, firefox-esr: 52.5.3esr -> 52.6.0esr #34073

Merged

8 tasks

teto mentioned this issue Apr 20, 2018

[RDY] Vdirsyncer update #39195

Merged

8 tasks

teto closed this as completed Apr 25, 2018

fritz-k mentioned this issue Apr 4, 2019

vdirsyncer fails to sync #58563

Closed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

package: update vdirsyncer to account for the new rust dependancy #33050

package: update vdirsyncer to account for the new rust dependancy #33050

teto commented Dec 25, 2017

matthiasbeyer commented Dec 25, 2017

matthiasbeyer commented Dec 25, 2017

pbogdan commented Dec 25, 2017

teto commented Dec 26, 2017

pbogdan commented Dec 26, 2017

teto commented Dec 26, 2017 •

edited

Loading

pbogdan commented Dec 26, 2017

teto commented Dec 28, 2017 •

edited

Loading

Ralith commented Jan 19, 2018

bennofs commented Jan 31, 2018

teto commented Mar 5, 2018

teto commented Apr 6, 2018

bennofs commented Apr 6, 2018

teto commented Apr 25, 2018

package: update vdirsyncer to account for the new rust dependancy #33050

package: update vdirsyncer to account for the new rust dependancy #33050

Comments

teto commented Dec 25, 2017

Issue description

matthiasbeyer commented Dec 25, 2017

matthiasbeyer commented Dec 25, 2017

pbogdan commented Dec 25, 2017

teto commented Dec 26, 2017

pbogdan commented Dec 26, 2017

teto commented Dec 26, 2017 • edited Loading

pbogdan commented Dec 26, 2017

teto commented Dec 28, 2017 • edited Loading

Ralith commented Jan 19, 2018

bennofs commented Jan 31, 2018

teto commented Mar 5, 2018

teto commented Apr 6, 2018

bennofs commented Apr 6, 2018

teto commented Apr 25, 2018

teto commented Dec 26, 2017 •

edited

Loading

teto commented Dec 28, 2017 •

edited

Loading