Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

python2/3, isatty and stdout buffering #2645

Closed
lazka opened this issue Jun 30, 2017 · 9 comments
Closed

python2/3, isatty and stdout buffering #2645

lazka opened this issue Jun 30, 2017 · 9 comments

Comments

@lazka
Copy link
Member

lazka commented Jun 30, 2017

Python2/3 stdout not being line buffered under minttty is annoying so I'm wondering what would be the best fix here:

  • On startup Python decides if stdout is linebuffered based on isatty, we could detect mintty there.
  • Intercept all C level isatty calls (there are 11 calls in Python 3) so that os.isatty also returns the right result.
  • Anything else?
@jtanx
Copy link
Contributor

jtanx commented Jun 30, 2017

@lazka
Copy link
Member Author

lazka commented Jun 30, 2017

winpty is indeed nice, thanks. But it's not that obvious and tedious to type before each command. Then my question is why isn't "winpty bash" not the default mingw shell?

@jtanx
Copy link
Contributor

jtanx commented Jun 30, 2017

Well, just guessing here, but there's a non-zero cost associated with using winpty, and most of the time, mintty works fine anyway. Your problem stems from trying to use mingw python which is a native windows build, and hence expects it to be used from a windows console. mintty uses pipes for input/output, so it doesn't look like a proper terminal to most programs.

On the other hand, if you use msys2 (or cygwin) python (.e.g /usr/local/bin/python2), everything just works.

Anyway, I'm pretty sure you can also just edit the default console to not be mintty, although I don't know the full details of this. Try looking/modifying the batch file that starts msys2.

Or you can just call bash.exe (c:\msys64\usr\bin\bash.exe -l) directly from cmd.exe.... that works too.

@lazka
Copy link
Member Author

lazka commented Jun 30, 2017

Hm, found a downside with winpty, it doesn't allow inputing umlauts (öäü..) in Python. A bit of a deal breaker I guess. It also seems a tiny bit slower, but I wouldn't mind that in exchange for proper program behavior.

@lazka
Copy link
Member Author

lazka commented Jun 30, 2017

If anyone thinks that adjusting isatty() in Python is a good idea I can provide patches. If not feel free to close this.

@mingwandroid
Copy link
Member

I'm fine with that personally. ag was recently patched in this way, maybe you can reuse the code? There was talk of trying to upstream to mingw-w64 a msys2/cygwin sympathetic isatty, does anyone know what happened with that effort?

@peterbud
Copy link
Contributor

peterbud commented Jun 30, 2017

Sorry for the potentially off-topic question: could it be a similar reason why msys2 is not working in docker and no stdout is written to the console?
docker/for-win#262
https://github.com/msys2/msys2/issues/46

lazka referenced this issue in lazka/MINGW-packages Jul 7, 2017
…a real Windows console

CPython uses isatty() to detect a terminal and change some settings
like line buffering and interactive mode. Use is_cygpty() to make
this also work under mintty.
See https://github.com/Alexpux/MINGW-packages/issues/2645

This also removes the bash script which forced the interactive mode
when python3 was started without arguments. This is no longer needed as
Python now detects the terminal output and does this automatically.

Also use is_cygpty() to detect when not under mintty and disable the readline
module there, as using it breaks input of certain characters and
leads to errors on shutdown when it tries to save the readline history.
(The readline module is not available in the official Python build)
See https://github.com/Alexpux/MINGW-packages/issues/2656
@lazka
Copy link
Member Author

lazka commented Jul 7, 2017

@peterbud no idea, sorry

I've opened a PR #2675 for Python 3 for starters. That fixes line buffering, interactive detection with mintty and unicode input with winpty. The interactive mode using readline is still weird (arrows keys do the wrong thing), maybe due to readline not detecting the terminal as well...

lazka referenced this issue in lazka/MINGW-packages Aug 14, 2017
…a real Windows console

CPython uses isatty() to detect a terminal and change some settings
like line buffering and interactive mode. Use is_cygpty() to make
this also work under mintty.
See https://github.com/Alexpux/MINGW-packages/issues/2645

This also removes the bash script which forced the interactive mode
when python3 was started without arguments. This is no longer needed as
Python now detects the terminal output and does this automatically.

Also use is_cygpty() to detect when not under mintty and disable the readline
module there, as using it breaks input of certain characters and
leads to errors on shutdown when it tries to save the readline history.
(The readline module is not available in the official Python build)
See https://github.com/Alexpux/MINGW-packages/issues/2656
Alexpux referenced this issue Aug 14, 2017
…a real Windows console (#2675)

CPython uses isatty() to detect a terminal and change some settings
like line buffering and interactive mode. Use is_cygpty() to make
this also work under mintty.
See https://github.com/Alexpux/MINGW-packages/issues/2645

This also removes the bash script which forced the interactive mode
when python3 was started without arguments. This is no longer needed as
Python now detects the terminal output and does this automatically.

Also use is_cygpty() to detect when not under mintty and disable the readline
module there, as using it breaks input of certain characters and
leads to errors on shutdown when it tries to save the readline history.
(The readline module is not available in the official Python build)
See https://github.com/Alexpux/MINGW-packages/issues/2656
@lazka
Copy link
Member Author

lazka commented Aug 19, 2017

Most things should be fixed now.

  • "python" now properly starts in interactive mode
  • python3 does automatic line buffering in a terminal
  • meson/pytest have color output now
  • the readline module is disabled in case no cygwin terminal is found

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants