Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

MSYS2-based programs don't output anything to console #262

Closed
timotei opened this issue Nov 29, 2016 · 31 comments
Closed

MSYS2-based programs don't output anything to console #262

timotei opened this issue Nov 29, 2016 · 31 comments

Comments

@timotei
Copy link

timotei commented Nov 29, 2016

Latest Git for Windows releases uses MSYS2-based utilities for things like ssh. However, those programs don't write anything to the console, even if there is actually text written to stdout. The problem itself seems to be actually about MSYS2-based programs, which I will detail below.

Expected behavior

Stdout from helloworld/ls should be written to the console:

C:\msys1>helloworld.exe
Hello World!

Actual behavior

Stdout is not written to the console. However it works if we redirect stdout to a file:

C:\msys2>helloworld.exe

C:\msys2>helloworld.exe > stdout

C:\msys2>type stdout
Hello World!
C:\msys2>helloworld.exe 2> stderr

C:\msys2>type stderr

C:\msys2>

Information

The hello world program with the following source, is compiled via gcc helloworld.c -o helloworld.exe:

#include <stdio.h>

int main()
{
    printf("Hello World!");
    return 0;
}

I did another test for ls (which can be also obtained via https://sourceforge.net/projects/msys2/files/Base/i686/msys2-base-i686-20161025.tar.xz/download) which yielded somehow a much better idea about the issue:

C:\msys2>ls

C:\msys2>ls 2>stderr

C:\msys2>type stderr
ls: write error: Bad file descriptor

C:\msys2>ls > stdout

C:\msys2>type stdout
helloworld.c
helloworld.exe
ls.exe
msys-2.0.dll
msys-gcc_s-1.dll
msys-iconv-2.dll
msys-intl-8.dll
stderr
stdout

As it can be seen, we get ls: write error: Bad file descriptor for the command line.

The current docker configuration:

C:\Users\X>docker version
Client:
 Version:      1.12.3
 API version:  1.24
 Go version:   go1.6.3
 Git commit:   6b644ec
 Built:        Thu Oct 27 00:09:21 2016
 OS/Arch:      windows/amd64
 Experimental: true

Server:
 Version:      1.12.2-cs2-ws-beta
 API version:  1.25
 Go version:   go1.7.1
 Git commit:   050b611
 Built:        Tue Oct 11 02:35:40 2016
 OS/Arch:      windows/amd64

C:\Users\X>docker info
Containers: 53
 Running: 7
 Paused: 0
 Stopped: 46
Images: 115
Server Version: 1.12.2-cs2-ws-beta
Storage Driver: windowsfilter
 Windows:
Logging Driver: json-file
Plugins:
 Volume: local
 Network: nat null overlay
Swarm: inactive
Security Options:
Kernel Version: 10.0 14393 (14393.447.amd64fre.rs1_release_inmarket.161102-0100)
Operating System: Windows 10 Pro
OSType: windows
Architecture: x86_64
CPUs: 8
Total Memory: 31.94 GiB
ID: XXXXXXXXXXXXXXXXXXXXXXXXXXXXXX
Docker Root Dir: C:\ProgramData\Docker
Debug Mode (client): false
Debug Mode (server): false
Http Proxy: http://proxy:9876
Https Proxy: http://proxy:9876
No Proxy: 127.0.0.1,.lan
Registry: https://index.docker.io/v1/
Insecure Registries:
 127.0.0.0/8

C:\Users\X>docker images
microsoft/windowsservercore   latest                 93a9c37b36d0        9 weeks ago         8.675 GB
microsoft/nanoserver          latest                 e14bc0ecea12        9 weeks ago         810.3 MB

Steps to reproduce the behavior

  1. Ensure you have Docker for Windows + Windows Containers enabled
  2. Download & extract https://github.com/timotei/public-files/raw/master/docker-msys.zip
  3. cd docker-msys
  4. docker build -t msys .
  5. docker run --rm -it msys cmd

Now, you can navigate into the msys/msys2 directories to test out the things. Optionally, you could download & install msys2 and mount it via the -v flag to enable the compile & test workflow.

Do you have any pointers towards how to further investigate this issue?

@peterbud
Copy link

peterbud commented Dec 6, 2016

@timotei : what is your docker host: Windows Server 2016 or Windows 10 Pro? I have found a similar issue but only on Windows 10 Pro host. Windows Server 2016 seemed to work OK. See #295

@timotei
Copy link
Author

timotei commented Dec 6, 2016

It's on Win 10 Pro. IIRC I tested on WS2016 in a VM and I had the same issue, but I will retest.

@peterbud
Copy link

peterbud commented Dec 7, 2016

Its worthwhile to read this

It seems that the --tty is the problem.
You can give a try running the helloworld.exe from outside of the container:
docker run [container name] c:\\msys2\\helloworld.exe

@timotei
Copy link
Author

timotei commented Dec 9, 2016

So, I have Windows Server 2016 Version 1607 (14393.447), in a Hyper-V VM, and I tried running the same scenario. It seems I have the same behaviour.

                                                                                    REPOSITORY                    TAG                    IMAGE ID            CREATED             SIZE
microsoft/dotnet-samples      dotnetapp-nanoserver   d022352ac999        11 days ago         1.02 GB
microsoft/windowsservercore   latest                 f49a4ea104f1        5 weeks ago         9.2 GB

C:\Users\Administrator>docker info
Containers: 1
 Running: 0
 Paused: 0
 Stopped: 1
Images: 3
Server Version: 1.12.2-cs2-ws-beta
Storage Driver: windowsfilter
 Windows:
Logging Driver: json-file
Plugins:
 Volume: local
 Network: nat null overlay
Swarm: inactive
Default Isolation: process
Kernel Version: 10.0 14393 (14393.447.amd64fre.rs1_release_inmarket.161102-0100)
Operating System: Windows Server 2016 Standard Evaluation
OSType: windows
Architecture: x86_64
CPUs: 1
Total Memory: 1.692 GiB
Name: WIN-5T4RKSB88AA
ID: 7KQP:22AR:NSGL:J2MZ:MQVX:Q7MN:MNLH:3NG6:EXJQ:HZJQ:ENCW:DGZL
Docker Root Dir: C:\ProgramData\docker
Debug Mode (client): false
Debug Mode (server): false
Http Proxy: proxy:9876
Https Proxy: proxy:9876
Registry: https://index.docker.io/v1/
Insecure Registries:
 127.0.0.0/8
Live Restore Enabled: false

C:\Users\Administrator>docker version
Client:
 Version:      1.12.2-cs2-ws-beta
 API version:  1.25
 Go version:   go1.7.1
 Git commit:   050b611
 Built:        Tue Oct 11 02:35:40 2016
 OS/Arch:      windows/amd64

Server:
 Version:      1.12.2-cs2-ws-beta
 API version:  1.25
 Go version:   go1.7.1
 Git commit:   050b611
 Built:        Tue Oct 11 02:35:40 2016
 OS/Arch:      windows/amd64

Your usecase works, but that's not a acceptable workaround as the issue is withing the container :(

@peterbud
Copy link

I agree that it should work also within the container. I believe we have more than just one issue, and I'm just trying to isolate.
For me it seems that there are at least two issues:

  • --tty is not working at all with MSYS2 programs stdout when you open an interactive session (docker run -i) to the container

  • There is a difference how WS2016 hosts and Win10 hosts are behaving as they run containers (docker run [container name] c:\\msys2\\helloworld.exe ). I got an intersting tip yesterday, that this is due to the default isolation: Windows 10 default isolation is hyperv, on WS2016 the default isolation is process. I could not test it as my WS2016 is a VM in Azure, and I was not able to turn on nested virtualization on that, but if you could give a try it could be useful (docker build --isolation hyperv ....).

@timotei
Copy link
Author

timotei commented Dec 10, 2016

Well, in WS2016 final release it doesn't work either (like I said above, a Hyper-V based WS2016, with nested virtualization). Last time it worked was in WS2016 TP5. But my TP5 VM is gone and I had trouble getting a new one in the exact state as the previous one was (via the old way of installing the Container Images - not through the Docker client + pull) So sadly I can't test if we change the isolation way it breaks.

I am trying to see how I can further investigate the issue from MSYS2's PoV. Something has changed between the releases with regards the way it prints output... maybe that will provide insight to a more lower-level issue (e.g., maybe a Win 32 API, or it may be within MSYS2's itself (using an unsupported call))

@peterbud
Copy link

I guess you are right. I have compiled the above helloworld.c using MINGW64 environment and compiler (that is gcc 6.2) , and put the exe into the docker container, and it was working in the container properly, printed to stdout without problem.

On the other side I have tried to compile it under MSYS2, which was working, I could test it at command prompt, but in the docker container even the redirection helloworld.exe > stdout was not working. I have seen that you have used msys-gcc_s-1.dll while the MSYS2 is using msys-gcc_s-seh-1.dll. Can I ask which MSYS2 environment you were using to compile the testapp?

@timotei
Copy link
Author

timotei commented Dec 11, 2016

@peterbud I used the latest version of MSYS2 32bit from https://msys2.github.io/

MSYS2 version: 20161025

t@10 MSYS ~
$ gcc --version
gcc (GCC) 5.3.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

t@10 MSYS ~
$ pacman --version

 .--.                  Pacman v5.0.1 - libalpm v10.0.1
/ _.-' .-.  .-.  .-.   Copyright (C) 2006-2016 Pacman Development Team
\  '-. '-'  '-'  '-'   Copyright (C) 2002-2006 Judd Vinet
 '--'
                       This program may be freely redistributed under
                       the terms of the GNU General Public License.

I copied just the .dlls required to run it fine, and the s one was required

@peterbud
Copy link

I see - I'm using the 64-bit version of MSYS2, but the same version. And on that it's a different file.
what is the result of pacman -Ql gcc-libs?

For me this is the result:

$ pacman -Ql gcc-libs
gcc-libs /usr/
gcc-libs /usr/bin/
gcc-libs /usr/bin/msys-atomic-1.dll
gcc-libs /usr/bin/msys-cilkrts-5.dll
gcc-libs /usr/bin/msys-gcc_s-seh-1.dll
gcc-libs /usr/bin/msys-gfortran-3.dll
gcc-libs /usr/bin/msys-gomp-1.dll
gcc-libs /usr/bin/msys-gomp-plugin-host_nonshm-1.dll
gcc-libs /usr/bin/msys-quadmath-0.dll
gcc-libs /usr/bin/msys-ssp-0.dll
gcc-libs /usr/bin/msys-stdc++-6.dll
gcc-libs /usr/bin/msys-vtv-0.dll
gcc-libs /usr/bin/msys-vtv_stubs-0.dll
gcc-libs /usr/share/
gcc-libs /usr/share/info/
gcc-libs /usr/share/info/libgomp.info.gz
gcc-libs /usr/share/info/libquadmath.info.gz
gcc-libs /usr/share/licenses/
gcc-libs /usr/share/licenses/gcc-libs/
gcc-libs /usr/share/licenses/gcc-libs/RUNTIME.LIBRARY.EXCEPTION

I have also checked with Dependency Walker what are the dependencies of the helloworld.exe which I have compiled under msys2 (which is not working in docker host), but it depends only on MSYS-2.0.DLL and does not depend on MSVCRT.DLL.

@timotei
Copy link
Author

timotei commented Dec 11, 2016

This is my result:

gcc-libs /usr/
gcc-libs /usr/bin/
gcc-libs /usr/bin/msys-atomic-1.dll
gcc-libs /usr/bin/msys-cilkrts-5.dll
gcc-libs /usr/bin/msys-gcc_s-1.dll
gcc-libs /usr/bin/msys-gfortran-3.dll
gcc-libs /usr/bin/msys-gomp-1.dll
gcc-libs /usr/bin/msys-gomp-plugin-host_nonshm-1.dll
gcc-libs /usr/bin/msys-quadmath-0.dll
gcc-libs /usr/bin/msys-ssp-0.dll
gcc-libs /usr/bin/msys-stdc++-6.dll
gcc-libs /usr/bin/msys-vtv-0.dll
gcc-libs /usr/bin/msys-vtv_stubs-0.dll
gcc-libs /usr/share/
gcc-libs /usr/share/info/
gcc-libs /usr/share/info/libgomp.info.gz
gcc-libs /usr/share/info/libquadmath.info.gz
gcc-libs /usr/share/licenses/
gcc-libs /usr/share/licenses/gcc-libs/
gcc-libs /usr/share/licenses/gcc-libs/RUNTIME.LIBRARY.EXCEPTION

Looks the same except the gcc "redist". It's good to know that the 32bit exposes the same issue as the 64bit version.

Yes, it's correct that it doesn't depend on MSVCRT, as it's using the GCC runtime from MSYS not Microsoft's C++ Redist. No MinGW/GCC-based binaries should have MSVCRT as dependency.

@timotei
Copy link
Author

timotei commented Dec 11, 2016

So, I did some more tests.

I created a MSYS2-based program, but instead of using print/iostream, I used directly the Win32 API.

Here's the helloworld-windows.c source code:

#include <windows.h>

int main()
{
    HANDLE std_out = GetStdHandle(STD_OUTPUT_HANDLE);
    if (std_out == INVALID_HANDLE_VALUE) {
        return 66;
    }
    //AttachConsole(ATTACH_PARENT_PROCESS);
    if (!WriteConsole(std_out, "Hello World!", 12, NULL, NULL)) {
        return 67;
    }
    return 0;
}
C:\msys2>helloworld.exe

C:\msys2>helloworld.exe > stdout

C:\msys2>type stdout
Hello World!
C:\msys2>helloworld-windows.exe

C:\msys2>helloworld-windows.exe > stdout

C:\msys2>type stdout

C:\msys2>helloworld-windows.exe > stdout 2>&1

C:\msys2>type stdout

C:\msys2>helloworld-windows.exe

C:\msys2>echo %ERRORLEVEL%
67

@timotei
Copy link
Author

timotei commented Dec 11, 2016

One more update. I printed the GetLastError to see if I can get a proper indication on why it fails to write to the stdout.

The program source code is now:

#include <stdio.h>
#include <windows.h>

int main()
{
    printf("Printf: starting\n");
    HANDLE std_out = GetStdHandle(STD_OUTPUT_HANDLE);
    if (std_out == INVALID_HANDLE_VALUE) {
        printf("Printf: invalid handle");
        return 66;
    }
    AttachConsole(ATTACH_PARENT_PROCESS);
    if (!WriteConsole(std_out, "Hello World!", 12, NULL, NULL)) {
        printf("Printf: cannot write :(\n");

        LPVOID lpMsgBuf;
        LPVOID lpDisplayBuf;
        DWORD dw = GetLastError();

        FormatMessage(
            FORMAT_MESSAGE_ALLOCATE_BUFFER |
            FORMAT_MESSAGE_FROM_SYSTEM |
            FORMAT_MESSAGE_IGNORE_INSERTS,
            NULL,
            dw,
            MAKELANGID(LANG_NEUTRAL, SUBLANG_DEFAULT),
            (LPTSTR) &lpMsgBuf,
            0, NULL );

        HRESULT hresult = HRESULT_FROM_WIN32(dw);
        printf("PRINTF: error %d: %s, hresult: %d\n", dw, lpMsgBuf, hresult);
        return 67;
    }
    return 0;
}

And the output from docker:

C:\msys2>helloworld-windows.exe

C:\msys2>helloworld-windows.exe > stdout

C:\msys2>type stdout
Printf: starting
Printf: cannot write :(
PRINTF: error 6: The handle is invalid.
, hresult: -2147024890

C:\msys2>helloworld-windows.exe > stdout 2>stderr

C:\msys2>type stderr

So, it seems... an invalid handle is handed out to the app... (which confirms the bad file descriptor from ls, in my initial report).

Now, what next? :) Since we're working directly with the Win32 API... maybe the MS guys could help?

@friism
Copy link

friism commented Dec 11, 2016

Checking with @jhowardmsft to see if he might know how to get to the bottom of this.

@timotei
Copy link
Author

timotei commented Dec 12, 2016

@peterbud So, it seems that we can use interactive input, but not -t/--ty:

Microsoft Windows [Version 10.0.14393]
(c) 2016 Microsoft Corporation. All rights reserved.

C:\>dir
dir
 Volume in drive C has no label.
 Volume Serial Number is 3CE1-F8DC

 Directory of C:\

11/28/2016  03:37 PM                49 Dockerfile
10/10/2016  08:20 AM             1,894 License.txt
11/29/2016  11:12 AM    <DIR>          msys1
12/11/2016  03:03 PM    <DIR>          msys2
07/16/2016  03:18 PM    <DIR>          PerfLogs
12/12/2016  10:19 AM    <DIR>          Program Files
07/16/2016  03:18 PM    <DIR>          Program Files (x86)
10/10/2016  08:29 AM    <DIR>          Users
10/10/2016  08:28 AM    <DIR>          Windows
               2 File(s)          1,943 bytes
               7 Dir(s)  21,184,655,360 bytes free

C:\>C:\msys2\helloworld.exe
C:\msys2\helloworld.exe
Hello World!

@peterbud
Copy link

That's what I have tried to describe in my previous comment

@peterbud
Copy link

@timotei Just to have the complete picture, I have run the same test with the 64-bit version of MSYS2.

The helloworld-windows.exe gives the same invalid handle.

Then I just run a few tes copying the whole msys64/usr/bin folder into the container, and run some test with 64-bit version of MSYS2 executables, and none of them was able to put anything on the console, except foy cygcheck.exe. I have tried to run docker run -i and docker run -it, no difference.

@rn
Copy link
Contributor

rn commented Dec 30, 2016

@timotei @peterbud thanks for the detailed instructions.

I've just tried with 1.13-rc4 which, IIRC, had some console related fixes but still no dice. Tried from powershell with cmd and powershell in the container as well as from git-bash on the host.

And for completeness (but I think you tried this already above):

PS> docker run --rm  msys c:\msys2\ls.exe
BOOTNXT
Boot
Dockerfile
Documents and Settings
License.txt
PerfLogs
Program Files
Program Files (x86)
ProgramData
Users
Windows
bootmgr
cygdrive
dev
msys1
msys2
proc
PS> docker run --rm -t msys c:\msys2\ls.exe
PS> docker run --rm -i msys c:\msys2\ls.exe
BOOTNXT
Boot
Dockerfile
Documents and Settings
License.txt
PerfLogs
Program Files
Program Files (x86)
ProgramData
Users
Windows
bootmgr
cygdrive
dev
msys1
msys2
proc
PS> docker run --rm -it msys c:\msys2\ls.exe

And:

PS> docker run --rm -i msys cmd
Microsoft Windows [Version 10.0.14393]
(c) 2016 Microsoft Corporation. All rights reserved.

C:\>\msys2\ls.exe
\msys2\ls.exe
BOOTNXT
Boot
Dockerfile
Documents and Settings
License.txt
PerfLogs
Program Files
Program Files (x86)
ProgramData
Users
Windows
bootmgr
cygdrive
dev
msys1
msys2
proc

C:\>

@dhirschfeld
Copy link

dhirschfeld commented Jun 29, 2017

I'm also running into this limitation when trying to setup a build server in a windowsservercore container.

@lowenna
Copy link

lowenna commented Jun 29, 2017

@dhirschfeld Do you need interactive? Can it be built from a dockerfile non-interactively executing the appropriate commands?

@dhirschfeld
Copy link

@jhowardmsft - I think I should be fine with non-interactive. It's just taken a while to find this thread and realise that! For those not in-the-know it would be nice if it Just Worked so I thought I'd add a +1 just to say that people are still coming across this problem and that not all of them may have made it here yet.

In my case I was trying to set things up interactively so I would know the correct commands to put in the dockerfile. I wasn't having much luck with that approach though because of this issue.

@lowenna
Copy link

lowenna commented Jun 29, 2017

Understood. I think it's going to have to be fixed upstream in msys2 to be container aware. I haven't myself looked at the code, but a colleague noted they were doing some odd things to the console stdio pipes which is making it incompatible. FWIW, the way I originally worked out dockerfile.Windows in moby/moby (it no longer uses msys) and the right set of commands was on a scratch VM doing it interactively, then dockerising it once I knew the steps were right. Maybe that helps?

@dhirschfeld
Copy link

Thanks @jhowardmsft - that's also a sensible way to do it. I opened an issue under the msys2 repo in the hope that it's fixed upstream.

@kode54
Copy link

kode54 commented Feb 18, 2018

This appears to be an upstream MSYS2 bug, which is still occurring in the version I am using, when compiling a completely unrelated console program for Win32, on a Windows VM, not using Docker. I only found this bug report when searching, and I have not seen any MSYS2 bug report related to this, maybe it's already fixed upstream?

@rom1v
Copy link

rom1v commented Mar 23, 2018

I confirm this is not related to docker, I have the same problem for an application I develop: it is compiled under MSYS2, but when run in cmd.exe, there is no output.

If you know a workaround, I'm very interested 😉

@nathanhi
Copy link

Interesting fact: If msys2 programs are run via a subprocess (i.e. spawning msys2 using powershell which has been started from cmd) they magically produce output:

$ docker run -i msys cmd
Microsoft Windows [Version 10.0.16299.431]
(c) 2017 Microsoft Corporation. All rights reserved.

C:\> C:\tools\msys2\2017-07-21\usr\bin\make.exe --version

C:\>

C:\> powershell
Windows PowerShell
Copyright (C) Microsoft Corporation. All rights reserved.

PS C:\> C:\tools\msys2\2017-07-21\usr\bin\make.exe --version
GNU Make 4.2.1
Built for x86_64-pc-msys
Copyright (C) 1988-2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

@docker-robott
Copy link
Collaborator

Issues go stale after 90d of inactivity.
Mark the issue as fresh with /remove-lifecycle stale comment.
Stale issues will be closed after an additional 30d of inactivity.

Prevent issues from auto-closing with an /lifecycle frozen comment.

If this issue is safe to close now please do so.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle stale

@kode54
Copy link

kode54 commented Aug 28, 2018

May as well close this, msys2 is probably never going to fix it anyway.

@mat007
Copy link
Member

mat007 commented Aug 28, 2018

Does MSYS2 have an issue open somewhere about this?

@filipesilva
Copy link

filipesilva commented Nov 15, 2018

I opened an issue in msys2/MSYS2-packages#1490.

In the reproduction I put there, using -i showed output but using -it did not.

@StefanScherer
Copy link
Member

Closing this. The issue to follow is msys2/MSYS2-packages#1490

@docker-robott
Copy link
Collaborator

Closed issues are locked after 30 days of inactivity.
This helps our team focus on active issues.

If you have found a problem that seems similar to this, please open a new issue.

Send feedback to Docker Community Slack channels #docker-for-mac or #docker-for-windows.
/lifecycle locked

@docker docker locked and limited conversation to collaborators Jun 19, 2020
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Projects
None yet
Development

No branches or pull requests