option to use soft links instead of hard links #3308

davidslac · 2016-08-16T20:59:23Z

conda prefers hard links to soft links, and has an option to dissallow soft links. I prefer soft links - with a ls I can see what is what, and as I run into weird problems, I trust them more, can debug them easier. Are there technical reasons why you can't use soft links in site-packages or other places that conda uses them? I understand performance may be a reason to prefer hard links.

kalefranz · 2016-08-16T21:07:36Z

I'm hoping @ilanschnell can supplement here with the full history of why hard links are ultimately preferred over all else. There's a lot of engineering and experience behind it.

The short answer is that soft links are problematic when working with linked shared libraries. Hard links (or copies) more reliably keep the compiled-in relative paths intact.

@ilanschnell, mind adding some more history?

ilanschnell · 2016-08-16T21:24:27Z

Hard links are preferred over anything else because they are the cheapest, ie. no new file objects (inodes) are created, only directory name entries. Softlinks involve creating new inodes, and are also slower, because the OS has to follow them. Also hard-links work well across all OSes, including Windows. Moreover, a hard-linked file is just like the original file. There is no difference between the original file and the hard-linked file. In fact, any file is a hard-link already (with reference count 1). This also means that unexpected behavior is avoided, e.g. some programs detect soft likes and treat them special.

davidslac · 2016-08-16T21:37:21Z

Thanks! unexpected behavior is worrisome, I think the cost is affordable (for us at least). We haven't run into issues using soft links for binaries, shared object libraries, and python directories linked through a directory on the PYTHONPATH (on linux) but haven't thought about cross platform issues. I would still log a request for the feature - conda is very flexible, it will make soft links if it can't do hard links, but let you turn them off if you want (and turn off hard links too I think) so this would add to the flexibility - but no worries if this doesn't make sense for conda.

HugoTian · 2016-08-18T18:00:06Z

For now, I think user can use --copy to install all packages using copies instead of hard or soft-linking

ijstokes · 2016-09-02T12:50:57Z

From the Conda ML, possibly from @natefoo, and related to the request for a --softlink option to force soft/symbolic links:

I'm using Conda in CVMFS and am presented with the problem of rapidly growing space usage in the CVMFS repository (filesystem). This is in part because CVMFS does not support cross-directory hardlinks and so installing packages into environments results in multiple copies of the same package contents.

CVMFS is an HTTP-based FUSE-implemented read-only filesystem designed for software distribution which stores the "master" copy of a repository in content-addressable storage (CAS) (i.e. hashed file chunks on a local filesystem) on a host called a stratum 0. Changes are propagated out from stratum 0 servers and ultimately to clients via HTTP, which mount it via FUSE. When you make changes to the repository on the stratum 0 (i.e. beginning a transaction), CVMFS first read-only mounts the CAS using the CVMFS FUSE client. It then uses AUFS to mount a writable filesystem that is unioned with the read-only mount. You make your changes and when done, publish the changes. The difference between the read-write set and the read-only set becomes a snapshot.

This AUFS filesystem supports cross-directory hardlinks just fine. But when the changes are published, cross-directory hardlinks are broken and become multiple copies of the same file.

Conda falls back to symlinks if hardlinks are impossible, but unfortunately, this doesn't work in the CVMFS case, since conda is able to create hardlinks at runtime and doesn't know they will be broken later.

So, is it possible to force conda to use symlinks? The only config option I can see related to soft/hardlinks is to disable the symlink fallback. I need it to prefer symlinks to hardlinks, not the other way around.

natefoo · 2016-09-02T13:00:22Z

Thanks @ijstokes, that was indeed from me.

natefoo · 2016-10-04T20:35:19Z

I believe I've (re)discovered a serious limitation with the use of soft links, namely, they break conda's use of $ORIGIN RPATHs in executables. For example (different from the previous example where my entire conda install is in CVMFS), I have

vcflib-1.0.0_rc1 from bioconda installed in /cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0
an environment in a separate (NFS) filesystem in /galaxy/main/jobs/123456/conda-env

The environment has all of the appropriate symlinks, e.g.:

$ ls -lR conda-env
conda-env:
total 128
drwxr-xr-x 2 galaxy galaxygp 32768 Oct  4 13:35 bin
drwxr-xr-x 2 galaxy galaxygp 32768 Oct  4 11:32 conda-meta
drwxr-xr-x 2 galaxy galaxygp 32768 Oct  4 11:32 include
drwxr-xr-x 3 galaxy galaxygp 32768 Oct  4 11:32 lib

conda-env/bin:
total 2976
...
lrwxrwxrwx 1 galaxy galaxygp 65 Oct  4 11:32 vcfintersect -> /cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0/bin/vcfintersect
...

conda-env/lib:
total 608
lrwxrwxrwx 1 galaxy galaxygp    13 Oct  4 11:32 libgcc_s.so -> libgcc_s.so.1
lrwxrwxrwx 1 galaxy galaxygp    62 Oct  4 11:32 libgcc_s.so.1 -> /cvmfs/main.galaxyproject.org/deps/_conda/pkgs/libgcc-5.2.0-0/lib/libgcc_s.so.1
lrwxrwxrwx 1 galaxy galaxygp    20 Oct  4 11:32 libgfortran.so -> libgfortran.so.3.0.0
lrwxrwxrwx 1 galaxy galaxygp    20 Oct  4 11:32 libgfortran.so.3 -> libgfortran.so.3.0.0
lrwxrwxrwx 1 galaxy galaxygp    69 Oct  4 11:32 libgfortran.so.3.0.0 -> /cvmfs/main.galaxyproject.org/deps/_conda/pkgs/libgcc-5.2.0-0/lib/libgfortran.so.3.0.0
lrwxrwxrwx 1 galaxy galaxygp    16 Oct  4 11:32 libgomp.so -> libgomp.so.1.0.0
lrwxrwxrwx 1 galaxy galaxygp    16 Oct  4 11:32 libgomp.so.1 -> libgomp.so.1.0.0
lrwxrwxrwx 1 galaxy galaxygp    65 Oct  4 11:32 libgomp.so.1.0.0 -> /cvmfs/main.galaxyproject.org/deps/_conda/pkgs/libgcc-5.2.0-0/lib/libgomp.so.1.0.0
lrwxrwxrwx 1 galaxy galaxygp    20 Oct  4 11:32 libquadmath.so -> libquadmath.so.0.0.0
lrwxrwxrwx 1 galaxy galaxygp    20 Oct  4 11:32 libquadmath.so.0 -> libquadmath.so.0.0.0
lrwxrwxrwx 1 galaxy galaxygp    69 Oct  4 11:32 libquadmath.so.0.0.0 -> /cvmfs/main.galaxyproject.org/deps/_conda/pkgs/libgcc-5.2.0-0/lib/libquadmath.so.0.0.0
lrwxrwxrwx 1 galaxy galaxygp    19 Oct  4 11:32 libstdc++.so -> libstdc++.so.6.0.21
lrwxrwxrwx 1 galaxy galaxygp    19 Oct  4 11:32 libstdc++.so.6 -> libstdc++.so.6.0.21
lrwxrwxrwx 1 galaxy galaxygp    68 Oct  4 11:32 libstdc++.so.6.0.21 -> /cvmfs/main.galaxyproject.org/deps/_conda/pkgs/libgcc-5.2.0-0/lib/libstdc++.so.6.0.21
lrwxrwxrwx 1 galaxy galaxygp    53 Oct  4 11:32 libz.a -> /cvmfs/main.galaxyproject.org/deps/_conda/pkgs/zlib-1.2.8-3/lib/libz.a
lrwxrwxrwx 1 galaxy galaxygp    13 Oct  4 11:32 libz.so -> libz.so.1.2.8
lrwxrwxrwx 1 galaxy galaxygp    13 Oct  4 11:32 libz.so.1 -> libz.so.1.2.8
lrwxrwxrwx 1 galaxy galaxygp    60 Oct  4 11:32 libz.so.1.2.8 -> /cvmfs/main.galaxyproject.org/deps/_conda/pkgs/zlib-1.2.8-3/lib/libz.so.1.2.8

However, attempting to use vcfintersect will fail:

$ source activate /galaxy/main/jobs/123456/conda-env
discarding /cvmfs/main.galaxyproject.org/deps/_conda/bin from PATH
prepending /galaxy/main/jobs/123456/conda-env/bin to PATH
$ strace vcfintersect --help
execve("/galaxy-repl/main/jobdir/014/005/14005770/conda-env/bin/vcfintersect", ["vcfintersect", "--help"], [/* 24 vars */]) = 0
brk(0)                                  = 0x1d32000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f5f0349f000
readlink("/proc/self/exe", "/cvmfs/main.galaxyproject.org/de"..., 4096) = 82
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0/bin/../lib/tls/x86_64/libpthread.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0/bin/../lib/tls/x86_64", 0x7fff5f0b77d0) = -1 ENOENT (No such file or directory)
open("/cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0/bin/../lib/tls/libpthread.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0/bin/../lib/tls", 0x7fff5f0b77d0) = -1 ENOENT (No such file or directory)
open("/cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0/bin/../lib/x86_64/libpthread.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0/bin/../lib/x86_64", 0x7fff5f0b77d0) = -1 ENOENT (No such file or directory)
open("/cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0/bin/../lib/libpthread.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)
stat("/cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0/bin/../lib", 0x7fff5f0b77d0) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY)      = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=35290, ...}) = 0
mmap(NULL, 35290, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f5f03496000
close(3)                                = 0
...
writev(2, [{"vcfintersect", 12}, {": ", 2}, {"error while loading shared libra"..., 36}, {": ", 2}, {"libgomp.so.1", 12}, {": ", 2}, {"cannot open shared object file", 30}, {": ", 2}, {"No such file or directory", 25}, {"\n", 1}], 10) = 124
exit_group(127)                         = ?
+++ exited with 127 +++
vcfintersect: error while loading shared libraries: libgomp.so.1: cannot open shared object file: No such file or directory

This was discussed in #864, with the answer being i.e. "no_softlinks should be used more judiciously in package recipes." Unfortunately, the reality is that any executable that has dynamic library dependencies in conda is not symlink-suitable, and since symlinks really only make sense cross-filesystem, hard links are really the only suitable solution.

As a result, I suspect conda is not generally usable across filesystems for many packages unless the --copy argument to conda create is used (which is, unfortunately, slow). A (e.g.) --softlink-except-executables (that would be the default if the package and env filesystem are not the same) would make sense for this case and be slightly faster than --copy.

davidslac · 2016-10-07T05:03:34Z

That’s interesting, I guess soft links as a conda feature is a bad idea. I think LD_LIBRARY_PATH takes precedence over RPATH, if you want to try to work around it by managing environment variables.

From: Nate Coraor <notifications@github.com mailto:notifications@github.com>
Reply-To: conda/conda <reply@reply.github.com mailto:reply@reply.github.com>
Date: Tuesday, October 4, 2016 at 1:35 PM
To: conda/conda <conda@noreply.github.com mailto:conda@noreply.github.com>
Cc: David A Schneider <davidsch@slac.stanford.edu mailto:davidsch@slac.stanford.edu>, Author <author@noreply.github.com mailto:author@noreply.github.com>
Subject: Re: [conda/conda] option to use soft links instead of hard links (#3308)

I believe I've (re)discovered a serious limitation with the use of soft links, namely, they break conda's use of $ORIGIN RPATHs in executables. For example (different from the previous example where my entire conda install is in CVMFS), I have

vcflib-1.0.0_rc1 from bioconda installed in /cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0
an environment in a separate (NFS) filesystem in /galaxy/main/jobs/123456/conda-env

The environment has all of the appropriate symlinks, e.g.:

$ ls -lR conda-envconda-env:total 128drwxr-xr-x 2 galaxy galaxygp 32768 Oct 4 13:35 bindrwxr-xr-x 2 galaxy galaxygp 32768 Oct 4 11:32 conda-metadrwxr-xr-x 2 galaxy galaxygp 32768 Oct 4 11:32 includedrwxr-xr-x 3 galaxy galaxygp 32768 Oct 4 11:32 libconda-env/bin:total 2976...lrwxrwxrwx 1 galaxy galaxygp 65 Oct 4 11:32 vcfintersect -> /cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0/bin/vcfintersect...conda-env/lib:total 608lrwxrwxrwx 1 galaxy galaxygp 13 Oct 4 11:32 libgcc_s.so -> libgcc_s.so.1lrwxrwxrwx 1 galaxy galaxygp 62 Oct 4 11:32 libgcc_s.so.1 -> /cvmfs/main.galaxyproject.org/deps/_conda/pkgs/libgcc-5.2.0-0/lib/libgcc_s.so.1lrwxrwxrwx 1 galaxy galaxygp 20 Oct 4 11:32 libgfortran.so -> libgfortran.so.3.0.0lrwxrwxrwx 1 galaxy galaxygp 20 Oct 4 11:32 libgfortran.so.3 -> libgfortran.so.3.0.0lrwxrwxrwx 1 galaxy galaxygp 69 Oct 4 11:32 libgfortran.so.3.0.0 -> /cvmfs/main.galaxyproject.org/deps/_conda/pkgs/libgcc-5.2.0-0/lib/libgfortran.so.3.0.0lrwxrwxrwx 1 galaxy galaxygp 16 Oct 4 11:32 libgomp.so -> libgomp.so.1.0.0lrwxrwxrwx 1 galaxy galaxygp 16 Oct 4 11:32 libgomp.so.1 -> libgomp.so.1.0.0lrwxrwxrwx 1 galaxy galaxygp 65 Oct 4 11:32 libgomp.so.1.0.0 -> /cvmfs/main.galaxyproject.org/deps/_conda/pkgs/libgcc-5.2.0-0/lib/libgomp.so.1.0.0lrwxrwxrwx 1 galaxy galaxygp 20 Oct 4 11:32 libquadmath.so -> libquadmath.so.0.0.0lrwxrwxrwx 1 galaxy galaxygp 20 Oct 4 11:32 libquadmath.so.0 -> libquadmath.so.0.0.0lrwxrwxrwx 1 galaxy galaxygp 69 Oct 4 11:32 libquadmath.so.0.0.0 -> /cvmfs/main.galaxyproject.org/deps/_conda/pkgs/libgcc-5.2.0-0/lib/libquadmath.so.0.0.0lrwxrwxrwx 1 galaxy galaxygp 19 Oct 4 11:32 libstdc++.so -> libstdc++.so.6.0.21lrwxrwxrwx 1 galaxy galaxygp 19 Oct 4 11:32 libstdc++.so.6 -> libstdc++.so.6.0.21lrwxrwxrwx 1 galaxy galaxygp 68 Oct 4 11:32 libstdc++.so.6.0.21 -> /cvmfs/main.galaxyproject.org/deps/_conda/pkgs/libgcc-5.2.0-0/lib/libstdc++.so.6.0.21lrwxrwxrwx 1 galaxy galaxygp 53 Oct 4 11:32 libz.a -> /cvmfs/main.galaxyproject.org/deps/_conda/pkgs/zlib-1.2.8-3/lib/libz.alrwxrwxrwx 1 galaxy galaxygp 13 Oct 4 11:32 libz.so -> libz.so.1.2.8lrwxrwxrwx 1 galaxy galaxygp 13 Oct 4 11:32 libz.so.1 -> libz.so.1.2.8lrwxrwxrwx 1 galaxy galaxygp 60 Oct 4 11:32 libz.so.1.2.8 -> /cvmfs/main.galaxyproject.org/deps/_conda/pkgs/zlib-1.2.8-3/lib/libz.so.1.2.8

However, attempting to use vcfintersect will fail:

$ source activate /galaxy/main/jobs/123456/conda-envdiscarding /cvmfs/main.galaxyproject.org/deps/_conda/bin from PATHprepending /galaxy/main/jobs/123456/conda-env/bin to PATH
$ strace vcfintersect --helpexecve("/galaxy-repl/main/jobdir/014/005/14005770/conda-env/bin/vcfintersect", ["vcfintersect", "--help"], [/* 24 vars */]) = 0brk(0) = 0x1d32000mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7f5f0349f000readlink("/proc/self/exe", "/cvmfs/main.galaxyproject.org/de"..., 4096) = 82access("/etc/ld.so.preload", R_OK) = -1 ENOENT (No such file or directory)open("/cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0/bin/../lib/tls/x86_64/libpthread.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)stat("/cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0/bin/../lib/tls/x86_64", 0x7fff5f0b77d0) = -1 ENOENT (No such file or directory)open("/cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0/bin/../lib/tls/libpthread.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)stat("/cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0/bin/../lib/tls", 0x7fff5f0b77d0) = -1 ENOENT (No such file or directory)open("/cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0/bin/../lib/x86_64/libpthread.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)stat("/cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0/bin/../lib/x86_64", 0x7fff5f0b77d0) = -1 ENOENT (No such file or directory)open("/cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0/bin/../lib/libpthread.so.0", O_RDONLY) = -1 ENOENT (No such file or directory)stat("/cvmfs/main.galaxyproject.org/deps/_conda/pkgs/vcflib-1.0.0_rc1-0/bin/../lib", 0x7fff5f0b77d0) = -1 ENOENT (No such file or directory)open("/etc/ld.so.cache", O_RDONLY) = 3fstat(3, {st_mode=S_IFREG|0644, st_size=35290, ...}) = 0mmap(NULL, 35290, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7f5f03496000close(3) = 0...writev(2, [{"vcfintersect", 12}, {": ", 2}, {"error while loading shared libra"..., 36}, {": ", 2}, {"libgomp.so.1", 12}, {": ", 2}, {"cannot open shared object file", 30}, {": ", 2}, {"No such file or directory", 25}, {"\n", 1}], 10) = 124exit_group(127) = ?+++ exited with 127 +++vcfintersect: error while loading shared libraries: libgomp.so.1: cannot open shared object file: No such file or directory

This was discussed in #864 #864, with the answer being i.e. "no_softlinks should be used more judiciously in package recipes." Unfortunately, the reality is that any executable that has dynamic library dependencies in conda is not symlink-suitable, and since symlinks really only make sense cross-filesystem, hard links are really the only suitable solution.

As a result, I suspect conda is not generally usable across filesystems for many packages unless the --copy argument to conda create is used (which is, unfortunately, slow). A (e.g.) --softlink-except-executables (that would be the default if the package and env filesystem are not the same) would make sense for this case and be slightly faster than --copy.

—
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com//issues/3308#issuecomment-251505142, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AQAThRwAljhpossaybQvLYeVpUHgLe4sks5qwriMgaJpZM4Jl2BO.

abretaud · 2016-11-07T14:58:09Z

Hi,
I have the same kind of problems as @natefoo with perl: when the perl bin is installed by conda using a symlink, the search path for its modules (@inc) is relative to the original bin location (ie the pkg dir) => it cannot find the modules that are properly installed in the conda env.

Looking at this issue + #3373, I feel like having the following options would be ok for most use cases:

always_copy == --copy
always_softlinks == --softlinks

allow_softlinks (currently on by default) == --no-softlinks
allow_bin_softlinks (on by default?) == --no-bin-softlinks

What do you think?

msarahan · 2016-11-11T02:00:03Z

any executable that has dynamic library dependencies in conda is not symlink-suitable

This seems like a reasonable criterion to fix in conda-build. When we do the rpath replacement, shouldn't we then also be able to search for any symlinks and make them copies appropriately? Relative symlinks within a folder are probably fine, but anything cross-folder is a no-no, right?

natefoo · 2016-11-15T15:29:24Z

If I understand correctly the binary RPATH issue should all be fixed as of 4.3. Not sure about @abretaud's Perl @INC issue, though.

kalefranz · 2016-11-15T16:04:13Z

Relative symlinks within a folder are probably fine, but anything cross-folder is a no-no, right?

Pretty sure relative symlinks are generally fine even cross folder as long as they point to another file within the prefix that ultimately ends up in the conda package being built. Am I wrong?

abretaud · 2016-11-15T16:15:43Z

Nice! Looking at conda/conda-build/pull/1521 I think it should be ok for perl too, but I'll test it to confirm and let you know

abretaud · 2016-11-16T10:36:59Z

Wait, do I understand correctly that #1521 handles symlinks when building the packages?
I think the perl problem is when installing the package, ie perl will look for its module in a path relative to the real path of the bin (=after following the symlink, in pkgs dir).
Not sure if the RPATH problem is the same or not?

abretaud · 2016-11-17T13:40:43Z

Ok, I understand better now, the perl problem is indeed different

msarahan · 2016-11-17T13:52:06Z

So conda/conda-build#1501 does not address the problem?

That was a shot in the dark, more or less, so any feedback would be greatly appreciated.

davidslac · 2016-11-17T16:23:21Z

Sorry I haven't participated much in this thread - I'm not working with conda at the level of the code base where I can try out things in the master branch or evaluate. What I can say is that I have a beta version of centrally installed conda working here at LCLS/SLAC, things are pretty good - but it is awkward for users to create their own environments or clone centrally installed environments because each user gets a 3GB copy of the packages since they are on a different file system. If the soft link feature develops to the point where a user can add an option like --allow-soft-links when they clone a conda environment, or create an environment, or even better if I could put allow-soft-links option it in the centrally installed condarc so that users don't have to think about it, that would be neat. If the feature develops to the point where you can put it in a conda version in the conda-canary channel I could try it. Anyways, thanks for working on this issue!

best,

David Schneider
SLAC/LCLS

From: Mike Sarahan [notifications@github.com]
Sent: Thursday, November 17, 2016 5:52 AM
To: conda/conda
Cc: Schneider, David A.; Author
Subject: Re: [conda/conda] option to use soft links instead of hard links (#3308)

So conda/conda-build#1501 conda/conda-build#1501 does not address the problem?

That was a shot in the dark, more or less, so any feedback would be greatly appreciated.

�
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com//issues/3308#issuecomment-261251411, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AQAThau3Z6c8rKLH3VgJoXYx0afpds2Zks5q_FwIgaJpZM4Jl2BO.

kalefranz · 2016-11-17T16:29:02Z

David I think (hope) this part is done with the recent use-softlinks PR. There will be a canary release out soon if you want to test. Maybe a week or so.

On Nov 17, 2016, at 10:23 AM, David Schneider notifications@github.com wrote:

Sorry I haven't participated much in this thread - I'm not working with conda at the level of the code base where I can try out things in the master branch or evaluate. What I can say is that I have a beta version of centrally installed conda working here at LCLS/SLAC, things are pretty good - but it is awkward for users to create their own environments or clone centrally installed environments because each user gets a 3GB copy of the packages since they are on a different file system. If the soft link feature develops to the point where a user can add an option like --allow-soft-links when they clone a conda environment, or create an environment, or even better if I could put allow-soft-links option it in the centrally installed condarc so that users don't have to think about it, that would be neat. If the feature develops to the point where you can put it in a conda version in the conda-canary channel I could try it. Anyways, thanks for working on this issue!

best,

David Schneider
SLAC/LCLS

From: Mike Sarahan [notifications@github.com]
Sent: Thursday, November 17, 2016 5:52 AM
To: conda/conda
Cc: Schneider, David A.; Author
Subject: Re: [conda/conda] option to use soft links instead of hard links (#3308)

So conda/conda-build#1501 conda/conda-build#1501 does not address the problem?

That was a shot in the dark, more or less, so any feedback would be greatly appreciated.

�
You are receiving this because you authored the thread.
Reply to this email directly, view it on GitHubhttps://github.com//issues/3308#issuecomment-261251411, or mute the threadhttps://github.com/notifications/unsubscribe-auth/AQAThau3Z6c8rKLH3VgJoXYx0afpds2Zks5q_FwIgaJpZM4Jl2BO.

—
You are receiving this because you commented.
Reply to this email directly, view it on GitHub, or mute the thread.

kalefranz · 2017-03-30T15:47:13Z

An always_softlink configuration option was added as part of #3870 & #3876 in conda 4.3.0.

github-actions · 2021-10-20T06:01:43Z

Hi there, thank you for your contribution to Conda!

This issue has been automatically locked since it has not had recent activity after it was closed.

Please open a new issue if needed.

kalefranz added the type::feature request for a new feature or capability label Sep 1, 2016

natefoo mentioned this issue Sep 2, 2016

Add an always_softlink option to condarc to force softlinks even on filesystems that support hardlinks #3373

Closed

kalefranz added this to the 4.4 milestone Oct 20, 2016

mingwandroid mentioned this issue Oct 25, 2016

error loading shared object with conda-build 2 conda-forge/cmake-feedstock#21

Closed

natefoo mentioned this issue Oct 28, 2016

Add doc for conda_copy_dependencies option galaxyproject/galaxy#3105

Merged

msarahan mentioned this issue Nov 11, 2016

break symlink and copy file if symlink points to executable conda/conda-build#1521

Merged

natefoo mentioned this issue Nov 17, 2016

Autodetect the need for conda create --copy galaxyproject/galaxy#3193

Closed

bgruening mentioned this issue Dec 13, 2016

testing against latest conda version galaxyproject/tools-iuc#1071

Closed

kalefranz closed this as completed Mar 30, 2017

kalefranz mentioned this issue May 5, 2017

Conda does not fall back to using soft links #4253

Closed

github-actions bot added the locked [bot] locked due to inactivity label Oct 20, 2021

github-actions bot locked as resolved and limited conversation to collaborators Oct 20, 2021

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

option to use soft links instead of hard links #3308

option to use soft links instead of hard links #3308

davidslac commented Aug 16, 2016

kalefranz commented Aug 16, 2016

ilanschnell commented Aug 16, 2016

davidslac commented Aug 16, 2016

HugoTian commented Aug 18, 2016

ijstokes commented Sep 2, 2016

natefoo commented Sep 2, 2016

natefoo commented Oct 4, 2016

davidslac commented Oct 7, 2016

abretaud commented Nov 7, 2016

msarahan commented Nov 11, 2016

natefoo commented Nov 15, 2016

kalefranz commented Nov 15, 2016

abretaud commented Nov 15, 2016

abretaud commented Nov 16, 2016

abretaud commented Nov 17, 2016

msarahan commented Nov 17, 2016

davidslac commented Nov 17, 2016

kalefranz commented Nov 17, 2016

kalefranz commented Mar 30, 2017 •

edited

Loading

github-actions bot commented Oct 20, 2021

option to use soft links instead of hard links #3308

option to use soft links instead of hard links #3308

Comments

davidslac commented Aug 16, 2016

kalefranz commented Aug 16, 2016

ilanschnell commented Aug 16, 2016

davidslac commented Aug 16, 2016

HugoTian commented Aug 18, 2016

ijstokes commented Sep 2, 2016

natefoo commented Sep 2, 2016

natefoo commented Oct 4, 2016

davidslac commented Oct 7, 2016

abretaud commented Nov 7, 2016

msarahan commented Nov 11, 2016

natefoo commented Nov 15, 2016

kalefranz commented Nov 15, 2016

abretaud commented Nov 15, 2016

abretaud commented Nov 16, 2016

abretaud commented Nov 17, 2016

msarahan commented Nov 17, 2016

davidslac commented Nov 17, 2016

kalefranz commented Nov 17, 2016

kalefranz commented Mar 30, 2017 • edited Loading

github-actions bot commented Oct 20, 2021

kalefranz commented Mar 30, 2017 •

edited

Loading