Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cargo on beta needs updating to build Rust on master #39504

Closed
alexcrichton opened this issue Feb 3, 2017 · 1 comment
Closed

Cargo on beta needs updating to build Rust on master #39504

alexcrichton opened this issue Feb 3, 2017 · 1 comment
Assignees
@alexcrichton
Labels
P-high High priority regression-from-stable-to-beta Performance or correctness regression from stable to beta.
Milestone
1.16.0

Comments

@alexcrichton
Copy link
Member

In #39463 I've opted due to not upgrade Cargo yet due to the errors preventing landing.

We need to track down what's causing this problem, land that change on Cargo master, then backport that change to rust-1.16.0 Cargo, and then update the Cargo used to bootstrap Rust on the master branch.

I'm tagging this as a regression as it's something we'll need to fix before the next release, and we shouldn't lose track of it!
@alexcrichton alexcrichton added the regression-from-stable-to-beta Performance or correctness regression from stable to beta. label Feb 3, 2017
@alexcrichton alexcrichton mentioned this issue Feb 3, 2017
Merged
@alexcrichton
Copy link
Member Author

So here's what I'm thinking is happening

  • Cargo emits $folder\deps\libcompiler_builtins-$hash.rlib
  • Cargo hard-links that to $folder\libcompiler_builtins.rlib
  • Rustbuild hard-links the former to $sysroot/lib/.../libcompiler_builtins-$hash.rlib
  • Later on we run cargo test
  • As part of cargo test's normal execution, it attempts to re-hard-link the $hash file to the non-deps file (the first two steps above)
  • Presumably some concurrent invocation of rustc has the library open (to read it)

This means that we're deleting a hard link where the actual file has a handle open to it elsewhere (opened through another link). Windows doesn't like this!

@alexcrichton alexcrichton mentioned this issue Feb 4, 2017
Merged
@brson brson added T-tools P-high High priority labels Feb 9, 2017
@brson brson modified the milestone: 1.16.0 Feb 11, 2017
@alexcrichton alexcrichton added this to the 1.16.0 milestone Feb 11, 2017
alexcrichton added a commit to alexcrichton/rust that referenced this issue Mar 6, 2017
@alexcrichton
rustbuild: Use copies instead of hard links
44bf3ec 
The original motivation for hard links was to speed up the various stages of
rustbuild, but in the end this is causing problems on Windows (rust-lang#39504).

This commit tweaks the build system to use copies instead of hard links
unconditionally to ensure that the files accessed by Windows are always
disjoint.

Locally this added .3s to a noop build, so it shouldn't be too much of a
regression hopefully!
bors added a commit that referenced this issue Mar 7, 2017
@bors
Auto merge of #39518 - alexcrichton:update-cargo, r=brson
d8bef59 
rustbuild: Use copies instead of hard links

The original motivation for hard links was to speed up the various stages of
rustbuild, but in the end this is causing problems on Windows (#39504).

This commit tweaks the build system to use copies instead of hard links
unconditionally to ensure that the files accessed by Windows are always
disjoint.

Locally this added .3s to a noop build, so it shouldn't be too much of a
regression hopefully!

Closes #39504
alexcrichton added a commit to alexcrichton/rust that referenced this issue Mar 9, 2017
@alexcrichton
rustbuild: Use copies instead of hard links
3be02fc 
The original motivation for hard links was to speed up the various stages of
rustbuild, but in the end this is causing problems on Windows (rust-lang#39504).

This commit tweaks the build system to use copies instead of hard links
unconditionally to ensure that the files accessed by Windows are always
disjoint.

Locally this added .3s to a noop build, so it shouldn't be too much of a
regression hopefully!
bors added a commit that referenced this issue Mar 10, 2017
@bors
Auto merge of #39518 - alexcrichton:update-cargo, r=arielb1
a625e03 
rustbuild: Use copies instead of hard links

The original motivation for hard links was to speed up the various stages of
rustbuild, but in the end this is causing problems on Windows (#39504).

This commit tweaks the build system to use copies instead of hard links
unconditionally to ensure that the files accessed by Windows are always
disjoint.

Locally this added .3s to a noop build, so it shouldn't be too much of a
regression hopefully!

Closes #39504
bors added a commit that referenced this issue Mar 10, 2017
@bors
Auto merge of #39518 - alexcrichton:update-cargo, r=arielb1
f573db4 
rustbuild: Use copies instead of hard links

The original motivation for hard links was to speed up the various stages of
rustbuild, but in the end this is causing problems on Windows (#39504).

This commit tweaks the build system to use copies instead of hard links
unconditionally to ensure that the files accessed by Windows are always
disjoint.

Locally this added .3s to a noop build, so it shouldn't be too much of a
regression hopefully!

Closes #39504
alexcrichton added a commit to alexcrichton/rust that referenced this issue Mar 10, 2017
@alexcrichton
Rollup merge of rust-lang#39518 - alexcrichton:update-cargo, r=arielb1
1c0daa5 
rustbuild: Use copies instead of hard links

The original motivation for hard links was to speed up the various stages of
rustbuild, but in the end this is causing problems on Windows (rust-lang#39504).

This commit tweaks the build system to use copies instead of hard links
unconditionally to ensure that the files accessed by Windows are always
disjoint.

Locally this added .3s to a noop build, so it shouldn't be too much of a
regression hopefully!

Closes rust-lang#39504
@alexcrichton alexcrichton mentioned this issue Sep 2, 2017
Merged
alexcrichton added a commit to alexcrichton/rust that referenced this issue Sep 10, 2017
@alexcrichton
rustbuild: Switch back to using hard links
38bedfa 
The `copy` function historically in rustbuild used hard links to speed up the
copy operations that it does. This logic was backed out, however, in rust-lang#39518 due
to a bug that only showed up on Windows, described in rust-lang#39504. The cause
described in rust-lang#39504 happened because Cargo, on a fresh build, would overwrite
the previous artifacts with new hard links that Cargo itself manages.

This behavior in Cargo was fixed in rust-lang/cargo#4390 where it no longer
should overwrite files on fresh builds, opportunistically leaving the filesystem
intact and not touching it.

Hopefully this can help speed up local builds by doing fewer copies all over the
place!
bors added a commit that referenced this issue Sep 10, 2017
@bors
Auto merge of #44260 - alexcrichton:hardlink-no-copy, r=Mark-Simulacrum
6d445e1 
rustbuild: Switch back to using hard links

The `copy` function historically in rustbuild used hard links to speed up the
copy operations that it does. This logic was backed out, however, in #39518 due
to a bug that only showed up on Windows, described in #39504. The cause
described in #39504 happened because Cargo, on a fresh build, would overwrite
the previous artifacts with new hard links that Cargo itself manages.

This behavior in Cargo was fixed in rust-lang/cargo#4390 where it no longer
should overwrite files on fresh builds, opportunistically leaving the filesystem
intact and not touching it.

Hopefully this can help speed up local builds by doing fewer copies all over the
place!
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@alexcrichton alexcrichton

Labels
P-high High priority regression-from-stable-to-beta Performance or correctness regression from stable to beta.
Projects
None yet
Milestone
1.16.0
Development

No branches or pull requests
2 participants
@alexcrichton @brson