Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Copy task can't copy symlink to directory #6821

Open
rolfbjarne opened this issue Sep 6, 2021 · 5 comments
Open

Copy task can't copy symlink to directory #6821

rolfbjarne opened this issue Sep 6, 2021 · 5 comments
Labels
bug needs-design Requires discussion with the dev team before attempting a fix. Partner request symlink triaged
Milestone

Comments

@rolfbjarne
Copy link
Member

Repro:

symlinks-9c50e79.zip

Download, extract and run "make sucess" or "make fail" depending on what you want to test.

I'm trying to copy a symlink that points to a directory. That fails with:

/Users/rolf/test/msbuild-tests/symlinks/test.proj(8,9): error MSB3025: The source file "source/symlink2" is actually a directory. The "Copy" task does not support copying directories.

Now I remove the directory the symlink points to (so that the symlink points to nowhere).

This rather strangely works just fine.

The problem seems to be that the Copy task doesn't consider symlinks as files when they point to a directory.

Binlogs:
binlogs.zip

@rolfbjarne rolfbjarne added bug needs-triage Have yet to determine what bucket this goes in. labels Sep 6, 2021
rolfbjarne added a commit to rolfbjarne/xamarin-macios that referenced this issue Sep 8, 2021
…eworks to the app bundle. Fixes xamarin#12369.

.NET/MSBuild don't handle symlinks properly [1], which means that we can't ask .NET
to copy frameworks to the app bundle, since frameworks may contain symlinks.

In our case, the symptom was that instead of copying symlinks, the file the symlink
pointed to was copied instead, and then codesign complained about invalid bundle
format when we tried to sign the framework.

We fix this by having our own target (_CopyFrameworksToBundle) to copy frameworks
to the app bundle (instead of adding all the files in the frameworks to the ResolvedFileToPublish
item group), and then using 'ditto' to copy the frameworks.

[1]: dotnet/msbuild#6821

Fixes xamarin#12369.
rolfbjarne added a commit to xamarin/xamarin-macios that referenced this issue Sep 9, 2021
…opy frameworks to the app bundle. Fixes #12369. (#12656)

.NET/MSBuild don't handle symlinks properly [1], which means that we can't ask
.NET to copy frameworks to the app bundle, since frameworks may contain
symlinks.

In our case, the symptom was that instead of copying symlinks, the file the
symlink pointed to was copied instead, and then codesign complained about
invalid bundle format when we tried to sign the framework.

We fix this by having our own target (_CopyFrameworksToBundle) to copy
frameworks to the app bundle (instead of adding all the files in the
frameworks to the ResolvedFileToPublish item group), and then using 'ditto' to
copy the frameworks.

In order to create a test case for this, I also made the macOS and Mac
Catalyst versions of the XTest framework use symlinks:

* Create a proper XTest framework bundle hierarchy for macOS and Mac Catalyst
  by using the typical symlink structure (actual files in the Versions/A
  subdirectory, and then symlinks pointing into that directory).
* Create a separate Info.plist for each platform for XTest.framework, since
  using an otherwise correct framework makes tooling (such as codesign)
  complain if the Info.plist isn't correct too.

This made our existing tests show the bug.

Finally I had to fix signing frameworks where the executable is a symlink.

We were first resolving symlinks for the input - say we had an
Example.framework/Example symlink to Example.framework/Versions/A/Example -
and then checking the parent directory if it's a framework. The parent
directory of 'Example.framework/Versions/A/Example' is 'A', which did not meet
our framewrok condition (if it ends with '.framework').

The fix is to adjust the logic to resolve symlinks after checking if the input
is a framework or not.

[1]: dotnet/msbuild#6821

Fixes #12369.
@rainersigwald
Copy link
Member

In general MSBuild is not very aware of symlinks; we tend to try to see through them to find "real files". I suspect this is #703.

Is this super painful for y'all? We can reevaluate symlinks as inputs to Copy but it'll take some design effort (since we'll need to preserve the "if you copy a linked file it copies the contents of the file, not the link itself" behavior we've had for a long time).

@rainersigwald rainersigwald added needs-more-info Issues that need more info to continue investigation. and removed needs-triage Have yet to determine what bucket this goes in. labels Sep 9, 2021
@rolfbjarne
Copy link
Member Author

Is this super painful for y'all?

I've been able to work around it so far by not using the Copy task when we might run into a symlink.

We can reevaluate symlinks as inputs to Copy but it'll take some design effort (since we'll need to preserve the "if you copy a linked file it copies the contents of the file, not the link itself" behavior we've had for a long time).

One thing that would be helpful would be to make Copy able to copy a directory (and in that case preserve all the contents as-is, including symlinks) - I believe this has been filed already though (#5881).

@ghost ghost added the stale For issues that haven't had activity in some time. label Oct 11, 2021
@ghost
Copy link

ghost commented Oct 11, 2021

This issue is marked as stale because feedback has been requested for 30 days with no response. Please respond within 14 days or this issue will be closed due to inactivity.

@rolfbjarne
Copy link
Member Author

Please respond within 14 days or this issue will be closed due to inactivity.

I did respond, see previous comment.

@ghost ghost added needs-attention Needs a member of the team to respond. and removed stale For issues that haven't had activity in some time. labels Oct 11, 2021
@benvillalobos benvillalobos added Partner request needs-design Requires discussion with the dev team before attempting a fix. and removed needs-attention Needs a member of the team to respond. labels Nov 4, 2021
@benvillalobos benvillalobos added this to the Backlog milestone Nov 4, 2021
@benvillalobos benvillalobos removed the needs-more-info Issues that need more info to continue investigation. label Nov 4, 2021
@benvillalobos
Copy link
Member

Team Triage: Since there's a workaround we're going to put this in the backlog, but we're not opposed to finding a solution for this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug needs-design Requires discussion with the dev team before attempting a fix. Partner request symlink triaged
Projects
None yet
Development

No branches or pull requests

5 participants