Ansible tarball duplicates symlinked modules

[felixfontein]

All modules in c.g and c.n are in subdirectories, with symlinks in plugins/modules/ to the correct copy. The collection artefact on galaxy contains these symlinks, but the Ansible tarball replaced the symlinks with copies.

[felixfontein]

The duplication seems to be done by the setup.py sdist call made by antsibull to build the tarball. This seems to be a known problem (ansible/ansible#27105, pypa/setuptools#415). It happens internally to setuptool's sdist implementation; before creating the tarball (which would keep symlinks), it copies all files into a new directory, and duplicates links at that step.

Maybe the best way around this would be to extract the resulting tarball, de-duplicate, and then re-tar.

[abadger]

We could do that. Of course, by then we've lost the symlink information. We could hash all of the files to decide what files are the same. Or we could override the setuptools sdist implementation with our own (which is what is done inside of ansible).

[felixfontein]

We still have the symlink information since we have the original version that was tared :) But yeah, replacing the sdist implementation sounds better, especially if ansible-core already does that.

[gundalow]

There maybe some about building on Python 3.9 and symlinks changing (at least using the process that builds ansible-core)

[felixfontein]

The main impact of this will go away with Ansible 6 since then community.general and community.network will drop the symlinks and use meta/runtime.yml for redirects (since they drop Ansible 2.9 support).

[webknjaz]

@felixfontein Is this still relevant? I thought people switched to the routing/redirect mechanism by now...

[felixfontein]

Some did, but not all. The bulk of it did though (which was community.network and community.general).

Since we don't really want to create (and maintain) our own package biulder which doesn't do this, I think we can close it though. Thanks for the reminder :)