Source distribution cannot include symlinks

[BuildStream-Migration-Bot]

See original issue on GitLab
In GitLab by [Gitlab user @jmacarthur] on Mar 7, 2018, 18:43

Reporting as minor as I don't know if this is really a problem with BuildStream's CI setup or with setuptools.

!295 added a test which used an import element of a directory which contained a symlink. However, setup.py sdist will not capture that file, giving this warning, easily missed:

'tests/integration/project/files/compose-symlinks/base/sbin' not a regular file -- skipping

Since all our CI tests create a source distribution and test that we ended up with an unfortunate situation where tests passed locally but not on CI. Direct import of a directory (not a tarball or zip) containing a symlink seems like something we should test but we can't at the moment. There don't appear to be any options to sdist to include symlinks in the archive.

I can't think of a clean solution, but work-arounds are:

1. Regenerate the symlink in the compose.py test script
2. Add another stage to CI which runs tests from a git checkout instead of the source dist

[BuildStream-Migration-Bot]

In GitLab by [Gitlab user @tristanvb] on Mar 8, 2018, 07:52

I may be mistaken about the reason for our workaround which I mentioned in !295

i.e. https://gitlab.com/BuildStream/buildstream/blob/master/tests/sources/local.py#L58 documents this as a problem with datafiles, which it may very well also be, I'm no longer sure.

The datafiles upstream bug report is here.

I've commented on the pypa upstream bug report here, which is currently closed.

[BuildStream-Migration-Bot]

In GitLab by [Gitlab user @tristanvb] on Mar 8, 2018, 07:53

mentioned in merge request !295

[BuildStream-Migration-Bot]

In GitLab by [Gitlab user @cs-shadow] on Oct 10, 2019, 13:46

Since https://gitlab.com/BuildStream/buildstream/merge_requests/1629, symlinks are created on the fly and we shouldn't have any symlinks in BuildStream source and/or tests.

[BuildStream-Migration-Bot]

In GitLab by [Gitlab user @cs-shadow] on Oct 10, 2019, 13:46

closed