Segfault on any global tool installed

[wli3]

@hmvs commented on Wed Feb 13 2019

I have an issue with global tools, all of them failing with segfault. I've tried dotnetsay, dmd5, Amazon.Lambda.Tools

cat /var/log/syslog
Feb 14 00:35:19 SPHERALPU kernel: [ 1711.376672] dotnetsay[6594]: segfault at 0 ip 0000000000000000 sp 00007ffef5bc26a8 error 14 in dotnetsay[3ff000+1000
Crash dump:
https://file.io/PlgY5w

Is there anything else I need to provide to understand what is the issue?

I've installed .net core 2.2 sdk from the snap package.
dotnet --info

.NET Core SDK (reflecting any global.json):
 Version:   2.2.104
 Commit:    73f036d4ac

Runtime Environment:
 OS Name:     ubuntu
 OS Version:  18.04
 OS Platform: Linux
 RID:         ubuntu.18.04-x64
 Base Path:   /snap/dotnet-sdk/29/sdk/2.2.104/

Host (useful for support):
  Version: 2.2.2
  Commit:  a4fd7b2c84

.NET Core SDKs installed:
  2.2.104 [/snap/dotnet-sdk/29/sdk]

.NET Core runtimes installed:
  Microsoft.AspNetCore.All 2.2.2 [/snap/dotnet-sdk/29/shared/Microsoft.AspNetCore.All]
  Microsoft.AspNetCore.App 2.2.2 [/snap/dotnet-sdk/29/shared/Microsoft.AspNetCore.App]
  Microsoft.NETCore.App 2.2.2 [/snap/dotnet-sdk/29/shared/Microsoft.NETCore.App]

dotnet tool list -g

Package Id               Version      Commands     
---------------------------------------------------
amazon.lambda.tools      3.1.2        dotnet-lambda
dmd5                     0.0.1        dmd5         
dotnetsay                2.1.4        dotnetsay  

dotnet run works file for the simple console app

Steps to reproduce:

Just tested on freshly started cloud instance:

1. Start a fresh Ubuntu 18.04
2. sudo snap install dotnet-sdk --classic
3. sudo snap alias dotnet-sdk.dotnet dotnet
4. dotnet --info

.NET Core SDK (reflecting any global.json):
 Version:   2.2.104
 Commit:    73f036d4ac
....

1. dotnet tool install -g dotnetsay
2. export PATH="$PATH:/home/ubuntu/.dotnet/tools"
3. dotnetsay
   Segmentation fault (core dumped)

---

@livarcocc commented on Wed Feb 13 2019

Have you tried running a regularly acquire application and seeing that works for you?

This seems more like an indicative of something native dependency missing in your box than an issue with global tools.

---

@livarcocc commented on Wed Feb 13 2019

@jkotas as someone who can identify the missing native dependency.

---

@jkotas commented on Wed Feb 13 2019

https://file.io/PlgY5w

This link is 404 Page Not Found for me.

Could you please run the command under strace? The strace log usually gives enough hints of what went wrong.

---

@hmvs commented on Thu Feb 14 2019

@jkotas
Here is the dump:
core.zip

strace dotnetsay

execve("/home/hmvs/.dotnet/tools/dotnetsay", ["dotnetsay"], 0x7fffee00f1d0 /* 63 vars */) = 0
brk(NULL)                               = 0x20e0000
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
readlink("/proc/self/exe", "/home/hmvs/.dotnet/tools/dotnets"..., 4096) = 34
access("/etc/ld.so.preload", R_OK)      = -1 ENOENT (No such file or directory)
open("/home/hmvs/.dotnet/tools/netcoredeps/tls/x86_64/libdl.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/home/hmvs/.dotnet/tools/netcoredeps/tls/x86_64", 0x7ffccce111e0) = -1 ENOENT (No such file or directory)
open("/home/hmvs/.dotnet/tools/netcoredeps/tls/libdl.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/home/hmvs/.dotnet/tools/netcoredeps/tls", 0x7ffccce111e0) = -1 ENOENT (No such file or directory)
open("/home/hmvs/.dotnet/tools/netcoredeps/x86_64/libdl.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/home/hmvs/.dotnet/tools/netcoredeps/x86_64", 0x7ffccce111e0) = -1 ENOENT (No such file or directory)
open("/home/hmvs/.dotnet/tools/netcoredeps/libdl.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/home/hmvs/.dotnet/tools/netcoredeps", 0x7ffccce111e0) = -1 ENOENT (No such file or directory)
open("/home/hmvs/.dotnet/tools/../../../lib/x86_64-linux-gnu/tls/x86_64/libdl.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/home/hmvs/.dotnet/tools/../../../lib/x86_64-linux-gnu/tls/x86_64", 0x7ffccce111e0) = -1 ENOENT (No such file or directory)
open("/home/hmvs/.dotnet/tools/../../../lib/x86_64-linux-gnu/tls/libdl.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/home/hmvs/.dotnet/tools/../../../lib/x86_64-linux-gnu/tls", 0x7ffccce111e0) = -1 ENOENT (No such file or directory)
open("/home/hmvs/.dotnet/tools/../../../lib/x86_64-linux-gnu/x86_64/libdl.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/home/hmvs/.dotnet/tools/../../../lib/x86_64-linux-gnu/x86_64", 0x7ffccce111e0) = -1 ENOENT (No such file or directory)
open("/home/hmvs/.dotnet/tools/../../../lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/home/hmvs/.dotnet/tools/../../../lib/x86_64-linux-gnu", 0x7ffccce111e0) = -1 ENOENT (No such file or directory)
open("/home/hmvs/.dotnet/tools/../../../usr/lib/x86_64-linux-gnu/tls/x86_64/libdl.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/home/hmvs/.dotnet/tools/../../../usr/lib/x86_64-linux-gnu/tls/x86_64", 0x7ffccce111e0) = -1 ENOENT (No such file or directory)
open("/home/hmvs/.dotnet/tools/../../../usr/lib/x86_64-linux-gnu/tls/libdl.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/home/hmvs/.dotnet/tools/../../../usr/lib/x86_64-linux-gnu/tls", 0x7ffccce111e0) = -1 ENOENT (No such file or directory)
open("/home/hmvs/.dotnet/tools/../../../usr/lib/x86_64-linux-gnu/x86_64/libdl.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/home/hmvs/.dotnet/tools/../../../usr/lib/x86_64-linux-gnu/x86_64", 0x7ffccce111e0) = -1 ENOENT (No such file or directory)
open("/home/hmvs/.dotnet/tools/../../../usr/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = -1 ENOENT (No such file or directory)
stat("/home/hmvs/.dotnet/tools/../../../usr/lib/x86_64-linux-gnu", 0x7ffccce111e0) = -1 ENOENT (No such file or directory)
open("/etc/ld.so.cache", O_RDONLY|O_CLOEXEC) = 3
fstat(3, {st_mode=S_IFREG|0644, st_size=161487, ...}) = 0
mmap(NULL, 161487, PROT_READ, MAP_PRIVATE, 3, 0) = 0x7efee62c6000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7efee62c5000
open("/lib/x86_64-linux-gnu/libdl.so.2", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0P\16\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=14560, ...}) = 0
mmap(NULL, 2109712, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7efee5ec5000
mprotect(0x7efee5ec8000, 2093056, PROT_NONE) = 0
mmap(0x7efee60c7000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x2000) = 0x7efee60c7000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libpthread.so.0", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0000b\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=144976, ...}) = 0
mmap(NULL, 2221184, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7efee5ca6000
mprotect(0x7efee5cc0000, 2093056, PROT_NONE) = 0
mmap(0x7efee5ebf000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x19000) = 0x7efee5ebf000
mmap(0x7efee5ec1000, 13440, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7efee5ec1000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/usr/lib/x86_64-linux-gnu/libstdc++.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\360\303\10\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=1594832, ...}) = 0
mmap(NULL, 3702816, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7efee591d000
mprotect(0x7efee5a96000, 2097152, PROT_NONE) = 0
mmap(0x7efee5c96000, 49152, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x179000) = 0x7efee5c96000
mmap(0x7efee5ca2000, 12320, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7efee5ca2000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libm.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\200\272\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=1700792, ...}) = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7efee62c4000
mmap(NULL, 3789144, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7efee557f000
mprotect(0x7efee571c000, 2093056, PROT_NONE) = 0
mmap(0x7efee591b000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x19c000) = 0x7efee591b000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libgcc_s.so.1", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\0\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\300*\0\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0644, st_size=96616, ...}) = 0
mmap(NULL, 2192432, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7efee5367000
mprotect(0x7efee537e000, 2093056, PROT_NONE) = 0
mmap(0x7efee557d000, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x16000) = 0x7efee557d000
close(3)                                = 0
access("/etc/ld.so.nohwcap", F_OK)      = -1 ENOENT (No such file or directory)
open("/lib/x86_64-linux-gnu/libc.so.6", O_RDONLY|O_CLOEXEC) = 3
read(3, "\177ELF\2\1\1\3\0\0\0\0\0\0\0\0\3\0>\0\1\0\0\0\260\34\2\0\0\0\0\0"..., 832) = 832
fstat(3, {st_mode=S_IFREG|0755, st_size=2030544, ...}) = 0
mmap(NULL, 4131552, PROT_READ|PROT_EXEC, MAP_PRIVATE|MAP_DENYWRITE, 3, 0) = 0x7efee4f76000
mprotect(0x7efee515d000, 2097152, PROT_NONE) = 0
mmap(0x7efee535d000, 24576, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_DENYWRITE, 3, 0x1e7000) = 0x7efee535d000
mmap(0x7efee5363000, 15072, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_FIXED|MAP_ANONYMOUS, -1, 0) = 0x7efee5363000
close(3)                                = 0
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7efee62c3000
mmap(NULL, 4096, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7efee62c2000
mmap(NULL, 8192, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_ANONYMOUS, -1, 0) = 0x7efee62c0000
arch_prctl(ARCH_SET_FS, 0x7efee62c0740) = 0
--- SIGSEGV {si_signo=SIGSEGV, si_code=SEGV_MAPERR, si_addr=NULL} ---
+++ killed by SIGSEGV (core dumped) +++
fish: “strace dotnetsay” terminated by signal SIGSEGV (Address boundary error)

---

@hmvs commented on Thu Feb 14 2019

@livarcocc

Have you tried running a regularly acquire application and seeing that works for you?

This seems more like an indicative of something native dependency missing in your box than an issue with global tools.

Pretty sure that something missing on my box. But as far as I understand the concept of the snap, every dependency should be packed and placed inside it. Correct me if am I wrong.

---

@jkotas commented on Thu Feb 14 2019

I think the problem is with mismatched binaries under /home/hmvs/.dotnet/tools/netcoredeps. Could you please delete this directory and see whether it will fix the problem?

I do not see .dotnet/tools/netcoredeps directory on my machine. We need to find out how it go there. @livarcocc Does the global tools installation ever create .dotnet/tools/netcoredeps ?

---

@wli3 commented on Thu Feb 14 2019

No, it should not create a folder called netcoredeps. It might be a tool called netcoredeps? One of the early preview does put assets under /tools. Today, all assets are under /tools/.store. Only shims are directly under /tools.

---

@jkotas commented on Thu Feb 14 2019

Actually, I have misread the log. There are no files under /home/hmvs/.dotnet/tools/netcoredeps - all attempts to access them fail with ENOENT.

---

@hmvs commented on Thu Feb 14 2019

And under the /home/hmvs/.dotnet/tools I have

-rwxrw-rw- 1 hmvs hmvs 109K Feb 13 23:09 dmd5*
-rwxrw-rw- 1 hmvs hmvs 109K Feb 13 22:05 dotnet-lambda*
-rwxrw-rw- 1 hmvs hmvs 109K Feb 14 00:35 dotnetsay*

---

@hmvs commented on Thu Feb 14 2019

All the dlls stored there /home/hmvs/.dotnet/tools/.store/dmd5/0.0.1/dmd5/0.0.1/tools/netcoreapp2.1/any
which is looks a bit strange

---

@wli3 commented on Thu Feb 14 2019

All the dlls stored there /home/hmvs/.dotnet/tools/.store/dmd5/0.0.1/dmd5/0.0.1/tools/netcoreapp2.1/any
which is looks a bit strange

This is expected layout. That folder is implementation detail. The layout is to separate different tools.

There is no /home/hmvs/.dotnet/tools/.store/dotnetsay folder?

---

@wli3 commented on Thu Feb 14 2019

And I note you are using snap. Maybe something to do with it? I hear it is using container to separate apps, maybe you need to also run dotnetsay in the same container?

---

@hmvs commented on Thu Feb 14 2019

All the dlls stored there /home/hmvs/.dotnet/tools/.store/dmd5/0.0.1/dmd5/0.0.1/tools/netcoreapp2.1/any
which is looks a bit strange

This is expected layout. That folder is implementation detail. The layout is to separate different tools.

There is no /home/hmvs/.dotnet/tools/.store/dotnetsay folder?

the is

[20:52:34] hmvs@SPHERALPU /home/hmvs/.dotnet/tools/.store (0) 
> ll
total 12K
drwxr-xr-x 3 hmvs hmvs 4.0K Feb 13 22:05 amazon.lambda.tools/
drwxr-xr-x 3 hmvs hmvs 4.0K Feb 13 23:09 dmd5/
drwxr-xr-x 3 hmvs hmvs 4.0K Feb 14 00:35 dotnetsay/

---

@hmvs commented on Thu Feb 14 2019

And I note you are using snap. Maybe something to do with it? I hear it is using container to separate apps, maybe you need to also run dotnetsay in the same container?

Not sure how to do it. I installed dotnet via snap. But I am using it just as usual dotnet installation.
Like dotnet run and this is it.

---

@jkotas commented on Thu Feb 14 2019

@hmvs Could you please open the core dump in the debugger on your system and get a stacktrace of the crash?

---

@hmvs commented on Thu Feb 14 2019

@hmvs Could you please open the core dump in the debugger on your system and get a stacktrace of the crash?

let me try :)

---

@hmvs commented on Thu Feb 14 2019

Here we are

(gdb) info stack
#0  0x0000000000000000 in ?? ()
dotnet/core-setup#1  0x00007f8c2a316414 in _dl_vdso_vsym (name=name@entry=0x7f8c2a3641d5 "__vdso_time", vers=vers@entry=0x7fffb2fde120)
    at ../sysdeps/unix/sysv/linux/dl-vdso.c:39
dotnet/core-setup#2  0x00007f8c2a2819f6 in time_ifunc () at ../sysdeps/unix/sysv/linux/x86/time.c:43
dotnet/core-setup#3  0x00007f8c2b30e530 in ?? () from /snap/dotnet-sdk/current/lib/x86_64-linux-gnu/ld-2.23.so
dotnet/core-setup#4  0x00007f8c2b3060db in ?? () from /snap/dotnet-sdk/current/lib/x86_64-linux-gnu/ld-2.23.so
dotnet/core-setup#5  0x00007f8c2b31b632 in ?? () from /snap/dotnet-sdk/current/lib/x86_64-linux-gnu/ld-2.23.so
dotnet/core-setup#6  0x00007f8c2b303c2a in ?? () from /snap/dotnet-sdk/current/lib/x86_64-linux-gnu/ld-2.23.so
dotnet/core-setup#7  0x00007f8c2b302c38 in ?? () from /snap/dotnet-sdk/current/lib/x86_64-linux-gnu/ld-2.23.so
dotnet/core-setup#8  0x0000000000000001 in ?? ()
dotnet/core-setup#9  0x00007fffb2fdf218 in ?? ()
dotnet/core-setup#10 0x0000000000000000 in ?? ()

---

@jkotas commented on Thu Feb 14 2019

This is crashing deep in the Linux system libraries. It is looks like a bad interaction with "snap".

---

@hmvs commented on Thu Feb 14 2019

Sooo, what is our next steps? 😳

---

@jkotas commented on Thu Feb 14 2019

1. Add exact steps for how you have installed this from snap so others can reproduce the issue
2. Ask somebody who understands how snap is supposed to work for help

---

@hmvs commented on Thu Feb 14 2019

@jkotas
Just tested on freshly started cloud instance:

1. Start a fresh Ubuntu 18.04
2. sudo snap install dotnet-sdk --classic
3. sudo snap alias dotnet-sdk.dotnet dotnet
4. dotnet --info

.NET Core SDK (reflecting any global.json):
 Version:   2.2.104
 Commit:    73f036d4ac
....

5. dotnet tool install -g dotnetsay
6. export PATH="$PATH:/home/ubuntu/.dotnet/tools"
7. dotnetsay
   Segmentation fault (core dumped)

---

@hmvs commented on Thu Feb 14 2019

@jkotas Do you know anyone whom we can add here ? :)

---

@jkotas commented on Thu Feb 14 2019

cc @janvorli @sergiy-k

---

@hmvs commented on Thu Feb 14 2019

I believe it is something related with dotnet uses some libc or another system lib instead one that comes with snap.
https://github.com/dotnet/core/blob/master/Documentation/self-contained-linux-apps.md

I've played around with LD_LIBRARY_PATH and I got more user friendly error.

Apparently this one (from strace) is wrong: /home/hmvs/.dotnet/tools/../../../usr/lib/x86_64-linux-gnu/tls/x86_64

I've tried to set DOTNET_ROOT variable to help dotnet to find path to correct libraries but it ignores this env variable.

---

@jkotas commented on Thu Feb 14 2019

wrong: /home/hmvs/.dotnet/tools/../../../usr/lib/x86_64-linux-gnu/tls/x86_64

And what this path should be instead? This path is computed by the Linux system libraries (like ld).

DOTNET_ROOT variable to help dotnet to find path to correct librarie

This crash is happening very early, before any of the .NET Core code had a chance to actually run (there is no .NET Core on the stack). So it won't recognize any DOTNET_XXX environment variables.

---

@hmvs commented on Thu Feb 14 2019

If I do something like this:
export LD_LIBRARY_PATH=/snap/dotnet-sdk/current/lib/x86_64-linux-gnu
I will get such behavior.

> dotnetsay
A fatal error occurred, the required library libhostfxr.so could not be found.
If this is a self-contained application, that library should exist in [/home/hmvs/.dotnet/tools/.store/dotnetsay/2.1.4/dotnetsay/2.1.4/tools/netcoreapp2.1/any/].
If this is a framework-dependent application, install the runtime in the default location [/usr/share/dotnet] or use the DOTNET_ROOT environment variable to specify the runtime location.

So apparently I need to hint somehow global-tools shim (don't know if this term is correct) where the all native libraries are placed. And I believe it is crashed because it loads those libraries from system instead of snap package folder.

---

@hmvs commented on Thu Feb 14 2019

Not sure. But might be related https://github.com/dotnet/cli/issues/9114

---

@livarcocc commented on Thu Feb 14 2019

@hmvs It seems setting the LD_LIBRARY_PATH has fixed the crash you are seeing. For the libhostfxr issue, yes, you need to set DOTNET_ROOT.

---

@hmvs commented on Thu Feb 14 2019

@livarcocc it didn't help. But I will try today evening.

Though, LD_LIBRARY_PATH it's not a solution, right?

---

@hmvs commented on Fri Feb 15 2019

Yup. This one works.
LD_LIBRARY_PATH=/snap/dotnet-sdk/current/lib/x86_64-linux-gnu DOTNET_ROOT=/snap/dotnet-sdk/current dotnetsay
But I can't leave LD_LIBRARY_PATH set to this directory, because all other software crashing.
So it only confirms my assumptions: that global-tools shim probing wrong path with standard linux libraries. We need to find a way how to make an installation with snap to find current path with dotnet.

ldd $(which dotnetsay)
	linux-vdso.so.1 (0x00007ffe0bf01000)
	libdl.so.2 => /lib/x86_64-linux-gnu/libdl.so.2 (0x00007f421c97b000)
	libpthread.so.0 => /lib/x86_64-linux-gnu/libpthread.so.0 (0x00007f421c75c000)
	libstdc++.so.6 => /usr/lib/x86_64-linux-gnu/libstdc++.so.6 (0x00007f421c3d3000)
	libm.so.6 => /lib/x86_64-linux-gnu/libm.so.6 (0x00007f421c035000)
	libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 (0x00007f421be1d000)
	libc.so.6 => /lib/x86_64-linux-gnu/libc.so.6 (0x00007f421ba2c000)
	/snap/dotnet-sdk/current/lib/x86_64-linux-gnu/ld-2.23.so => /lib64/ld-linux-x86-64.so.2 (0x00007f421cb7f000)

libdl.so.2, libpthread.so.0,libstdc++.so.6, libm.so.6,libgcc_s.so.1,libc.so.6 should be loaded from /snap/dotnet-sdk/current/lib/x86_64-linux-gnu instead of /lib/x86_64-linux-gnu/

Whom can I contact? Who are those brave guys who made snap package and global tools shim?

---

@wli3 commented on Fri Feb 15 2019

seems it is a problem with shim (app host). Moving to core-setup

[jkotas]

How snap works: https://github.com/canonical-docs/snappy-docs/blob/master/snaps/philosophy.md

Snap creates wrapper script that set environment variables, and then launch the actual application binaries. These wrappers are missing for global tools, the environment variables are not set, and it is what makes it crash.

What do other global tools (like Node global tools) do to make this work?

[hmvs]

Do't know about nodejs.
But I discovered that global tools shim trying to look libraries in the netcoredeps directory where the all global-tools installed.
So, the second workaround that could actually work is:
ln -s /snap/dotnet-sdk/current/lib/x86_64-linux-gnu /home/hmvs/.dotnet/tools/netcoredeps
And only thing is missing is DOTNET_ROOT

The question is: how we can automate it while installing a snap.
Even if we will be able to setup this link while installing snap: case when you install global tool in custom directory will be broken (dotnet tool install -g dotnetsay --tool-path /blabla)

Or another way it is to change the global tool shim to look into variable DOTNET_ROOT instead of looking into netcoredeps

[janvorli]

The netcoredeps is an optional directory that people can create and place dependencies into it. Shared library loader looks into this directory before looking into other regular system locations of shared libraries.
If that directory doesn't exist, then libraries are loaded from the usual locations. So it should not interfere with snap's default behavior in any way.

I have reproed the issue locally and the problem is caused by the way the snap patches the dotnetsay binary (the host ELF executable). It expects that the copies of the system libraries are relative to the dotnetsay location and sets the RPATH in the binary based on this assumption:

 0x000000000000000f (RPATH)              Library rpath: [$ORIGIN/netcoredeps:$ORIGIN/../../../lib/x86_64-linux-gnu:$ORIGIN/../../../usr/lib/x86_64-linux-gnu]

That would be correct if the dotnetsay was located in /home/hmvs/.dotnet, but not when it is located in /home/hmvs/.dotnet/tools. In such case, the relative path doesn't lead to the correct location of the copies of the libraries.

I am not familiar with how snap patches the dotnetsay ELF executable (I can only see it did patch the dotnetsay, because otherwise the RPATH would contain only $ORIGIN/netcoredeps). I suspect that what happens is that snap just patches the /snap/dotnet-sdk/29/sdk/2.2.104/AppHostTemplate/apphost and that is at the time of installation of the dotnetsay too copied into /home/hmvs/.dotnet/tools/dotnetsay (the apphost is the same for all published apps, it is just renamed to the name of the app).
Since the snap cannot know where the template will end up being placed, it can only work if the relative paths in the RPATH match the reality.

[janvorli]

The workaround for this issue is to patch the dotnetsay using the patchelf tool and fix the relative path:

patchelf --set-rpath '$ORIGIN/netcoredeps:$ORIGIN/../../../../lib/x86_64-linux-gnu:$ORIGIN/../../../../usr/lib/x86_64-linux-gnu' ~/.dotnet/tools/dotnetsay

[hmvs]

Maybe @leecow can help here? I see that it is him who created core-setup/src/pkg/packaging/snaps/dotnet-sdk/snap/snapcraft.yaml

[hmvs]

@janvorli And yeah, You are right.
Here it is https://github.com/snapcore/snapcraft/blob/master/snapcraft/internal/elf.py
Looking for way how we can hint it with our base path.

[leecow]

I hadn't done any investigation around this issue. The snapcraft team is very responsive so working with them through github should be fine. If needed, I can shoot a note to the snapcraft dev lead separately.

[hmvs]

Contacted them https://forum.snapcraft.io/t/rpath-in-elf-that-can-be-moved-across-system/10118

[guy-rouillier]

Found this issue by reading through core-setup for all issues containing "snap". I have Ubuntu 19.04, which has migrated to using snap for parts of the system. I decided to try out the dotnet snaps. After installing the dotnet-sdk-2.2, I am able to auto-generate a simple Console app and run it. But when I auto-generate a GtkSharp basic app, I can build it, but trying to run it produces:

Unhandled Exception: System.TypeInitializationException: The type initializer for 'Gtk.Application' threw an exception. ---> System.DllNotFoundException: Gtk
   at GLibrary.Load(Library library)
   at Gtk.Application..cctor()
   --- End of inner exception stack trace ---
   at Gtk.Application.Init()

Long story short, I contacted the GtkSharp people, they replied "Snap and flatpak are not supported in any shape or form, install dotnet sdk in a traditional manner and it will work fine." That is true; in a VM, I have another instance of Ubuntu 19.04, and I installed dotnet using the Microsoft packages; GtkSharp works fine from those.

[peterhuene]

We appear to be hitting this issue for any framework-dependent apphost (not just global tools), which is produced by default for 3.0 builds when using a snap-installed .NET Core SDK.

See dotnet/cli#12269.

[hmvs]

So we need either bump the thread that I created or help them implement it in the snap by ourselves.

My idea was to introduce an option in the snap that will allow ELF patch with an absolute path to the binaries.

See: Snapcraft forum thread

[NikolaMilosavljevic]

[Triage] Closing Snap issues to reflect current priorities.

[jkotas]

@NikolaMilosavljevic I see that you have closed all Snap issues as won't fix. Does it mean that we are no longer trying to support Snap?