bundle Julia with a better terminal on Windows?

[StefanKarpinski]

This seems like the nicest console emulator around on Windows:

http://bliker.github.io/cmder/
http://cmder.net
https://mintty.github.io

I think it would make sense, since Windows is not really a DIY platform, but more of a download-and-use-the-thing-that-you-can-click-on platform, that we include a really nice console emulator for people to use.

EDIT: fixed link
EDIT (@ViralBShah): mintty as the preferred choice

[lindahua]

+1

[tkelman]

That looks like a custom configuration wrapping up conemu, clink, and (optionally) msysgit. Clink is mostly readline, which we don't need any more. mintty might be an easier/smaller alternative to conemu.

[StefanKarpinski]

Yes, I think that's the deal. Since I don't use Windows myself, I'm somewhat uninformed about these tools, but I'd be more interested in making sure the experience is as nice as possible than minimizing the dependency size. Mintty just doesn't seem to look or feel as nice as cmder.

[tkelman]

I'm quite happy with mintty as used by Cygwin and MSYS2 (though there's still a bug making Julia not very usable inside mintty, I should check whether that's true in conemu), I don't see too much need for console tabs. There's some discussion of these and other alternatives in #6795

Any bugs that happen with whatever we choose would be harder to properly report or fix if we go with something that is a fork-of-fork-of-fork (or wrapper of wrapper of wrapper). Babun is another similar one of these projects that looks shiny, but is really just a custom configured version of Cygwin.

[quinnj]

Here's a crazy mastermind plan. Let's build a repl and/or IJulia notebook frontend for LightTable and ship with that!
But seriously, the more I've been poking around with LightTable, the more I can envision a proper "Julia IDE" being all baked in to a nice Julia plugin. It has all the extensibility to allow something like RStudio or MatLab IDE, but even better because you can do inline plots, or something notebook-ish where the program execution is "graph-based" instead of repl line-by-line. I think Mike's GSoC is a great start in getting the fundamentals in place, but there's a ton of potential there.

Anyway, I digress. I think mintty wounds like a good option here.

[tkelman]

I went through and tried a few of these options. cmder indeed looks decent.

With Console2 I was able to manually set it up to boot Julia (adding a Julia "tab" under the Settings - there's an xml config file so this could be automated) then running it as Console.exe -t Julia. Console2 didn't fix unicode display issues though. The default text select behavior is pretty awful, you hold left shift then click and drag. Holding a key to select can be omg-carpal-tunnel uncomfortable on a laptop.

I couldn't get cmder to start up Julia directly, but starting it manually worked okay, and displayed simple unicode properly. Cmder depends on the MSVC runtime dll's, was there a license concern in the particular terms under which you can redistribute those? The default selection behavior here was also left-shift (or right-clicking the icon, going to edit and switching to mark mode, same as in the default Windows console), but I found how to change it to not require simultaneous key+mouse.

ConEmu has some really annoying default behaviors, wanting to save its configuration in the registry by default, popping up a confirmation dialog every time I close it, etc. Was easy to start it directly in Julia, and simple unicode worked.

Mintty will take a little bit of work, but be the lightest and easiest option for users in the end. Some of the current issues with mintty are related to bigger libuv problems that will hopefully get fixed together. There's some code somewhere that tests specifically for Cygwin and MSYS2 named pipes, which I think would need to be checked and modified to work with the standalone download of mintty. We could maybe use MSYS2's version of mintty, but then we'd need msys-2.0.dll which is about 4x bigger than the msys-1.0.dll that we currently include as part of msysgit.

None of these options look to compile particularly easily with MinGW, and I wouldn't want to try cross-compiling. But just downloading the release binary should be good enough for these.

[StefanKarpinski]

@ihnorton, over here you said:

+1 for mintty for now. Based on my own experiences, I would be strongly opposed to packaging conemu.

can you elaborate on why over here?

[ihnorton]

@StefanKarpinski basically because every time I have tried it, too many things are just off. I don't remember all of the issues, but very clearly remember the worst one: it does bad things with subprocesses and makes spawning vastly slower than it already is on Windows.

noop.bat: @echo off

 julia> @time for i in 1:1000
       open(`noop.bat`,"r")
       end

time in seconds, two trials each:

    default console: 1.30,1.31
    cmder: 53.0, 61.6 (actually restarted twice more and got similar times)

In fact, my mouse starts freezing when running this test code under cmder - but there is no such problem under default Julia console.

[ihnorton]

See here for part of the reason why: https://code.google.com/p/conemu-maximus5/wiki/ConEmuHk
and from the FAQ:

Q. mingw/bash/git works slow in ConEmu.
A. In rare cases on some configurations users may notice lags while executing commands in ConEmu tab. This may be when many child processes are created while command execution. I you are so unlucky - just uncheck "Inject ConEmuHk" on Features page. I'm still looking for the cause of the problem.

This "feature" can be disabled, but bad spawn performance has been a known issue for several years. It is concerning that such a broken feature is enabled by default (and as I said, this is only the worst issue).

@tkelman:

Cmder depends on the MSVC runtime dll's, was there a license concern in the particular terms under which you can redistribute those?

Right, they can't be redistributed with GPL software. Now, technically, we wouldn't be linking it from Julia. I don't know if that changes anything.

[jhasse]

+1 for mintty. I'm using MSYS2 + Julia anyway and it works fine :)

[tkelman]

Using mintty from MSYS2 essentially works now, as long as you also include stty.exe, msys-2.0.dll, msys-intl-8.dll, and msys-iconv-2.dll. And you need to explicitly set HOME=%HOMEDRIVE%%HOMEPATH% before starting, otherwise mintty tries to set HOME to wherever it's located.

I also tried with MSYS1's version of mintty since we already have msys-1.0.dll with msysgit, but that didn't quite work. It seems in MSYS1 mintty, uv_pipe_getsockname(pipe, NULL, &len) in jl_uv.c returns 0.

[ihnorton]

mintty seems like the least-bad option. Just beware that support for shelling out to things using the Windows console API may be spotty. Pick your poison I guess, but this seems ok to me, in exchange for better "everything else".

(Have you tried something like git diff - does the pager work?)

[tkelman]

No, does not look like the pager works, but that might just be because we're using a version of less from msysgit, which doesn't work with mintty properly. Inside a full MSYS2 or Cygwin installation, less works fine from Julia (calling that environment's version of less). Tried just copying over MSYS2 less, and the dll's it depends on (ncurses and, hah, pcre), but that seemed to stack dump for some reason...

[pao]

@tkelman I believe I've had success with the version of less from gnuwin32, though it's not at all up to date.

[tkelman]

Will give it a try. Might not be that hard to just write a pager in pure Julia, would it?

[vtjnash]

i think lighttable/sublime/ijulia integration is going to be the way to go here. but that's a bigger question of how to bundle packages effectively (perhaps 8745 will help with that)

[tkelman]

I'm still intending to work on mintty for the purposes of the basic REPL. Juno is immature and not to my personal liking yet, IJulia depends on Python in an unsatisfying way. Improving the usability of the basic REPL is still something I plan on doing. That work is blocked on libgit2 however, since we'll need to make fairly substantial changes to the windows packaging when we remove command-line git.

[tkelman]

I just discovered the Powershell ISE. It's actually really nice, as long as you're using version 3.0 or newer. Julia doesn't start inside it at the moment, I get

julia.exe : ERROR: `convert` has no method matching convert(::Type{TTY}, ::Pipe)
At line:1 char:49

but that might be fixable. If so, maybe we should add a shortcut to start Julia inside it.

[tkelman]

Looks like similar to mintty, the Powershell ISE uses pipes for stdout and stderr, except stdin is a normal tty. The pipes for stdout and stderr do not have names as far as I can tell, both the uv_pipe_getsockname calls in jl_ispty return 0. If I skip the first early return and make jl_ispty return 1 when len == 0 after the second call to uv_pipe_getsockname, then Julia can start but it's completely unresponsive. And it doesn't seem to recognize color codes either. Judging by http://blogs.msdn.com/b/powershell/archive/2009/02/04/console-application-non-support-in-the-ise.aspx it might not be possible, but oh well, was worth trying.

[ncnc]

FWIW, I tried a fork of Console2 called ConsoleZ. Changing the defaults I get a reasonably sane copy-paste behavior (select with left mouse button), but similar to Console2 the unicode issues remain.

[StefanKarpinski]

We still need to ship with a better terminal emulator on Windows. Watching people struggle with cmd.exe is awful – Julia needs to provide a better default experience out of the box.

[IainNZ]

I agree, if the details can be hammered out. Although IJulia is certainly smoothing over this, at least in Julia classes I've been involved in over the last month.

(A higher priority would be anything to speed up Pkg operations, which seem extra slow on Windows.)

[tkelman]

As I said in December #7267 (comment), I will work on incorporating mintty. But not until we get rid of command-line git, since this will all require pretty significant rewriting of how the Windows binary builds get put together. Can we get moving on the libgit2 transition?

[StefanKarpinski]

Right, thanks @tkelman – I mainly wanted to reopen the issue since it's still unresolved.

[davidanthoff]

I like the plan with mintty, but I think it would be good if this was not the default if Julia is installed on a Win10 system. Win10 comes with a major overhaul over the console that probably solves all of the current issues, and I feel Julia should just adhere to the (then) good system default. Current Win10 RTM target seems to be June/summer, so certainly before Julia 0.5, and that kind of means that 0.4 will be the version that people will use on Win10.

[tkelman]

@davidanthoff yeah I have been meaning to try out the finally-not-crappy console in Win10 (and still confused about them skipping 9). The approach I think I'll take here is get mintty built and included in the binaries, and install a separate shortcut for starting Julia inside it. I don't want to do anything too complicated with detecting Windows versions when 7 and 8 are going to be around and what most Windows users will stay on for some time. Judging by traffic on the Cygwin mailing list, they're keeping up with changes in the prereleases and making sure mintty and other bits and pieces will continue to work.

[davidanthoff]

@tkelman That would certainly work. On the other hand, I like sensible defaults and am not a huge fan if new users have to make choices they don't understand when they first want to start julia. The vast majority of users will have no clue what mintty refers to, so when they see two shortcuts to start julia, and one is called something like JuliaMintty they will have no clue what that they should do. Maybe the installer could detect the version of windows, and then create an appropriate default shortcut? On the other hand, I agree, this shouldn't become a super complicated project, so if there is no easy way to achieve this, your suggestion might still be the best one.

[jebej]

For me ctrl-c does not kill the terminal, and ctrl-d works properly.
cd does not work and makes mintty/julia crash.

It seems that preferences also cannot be saved to disk, with mintty complaining about a non-existent .minttyrc file, even though I created it manually. Maybe this is a general filesystem issue.

FYI the mintty version is 3.0.2, which is not the latest one.

[jebej]

In addition, I do not get any issues when I use the version shipped with MSYS2, so these are probably compilation troubles.

[ViralBShah]

I am guessing the ctrl-c is going to mintty instead of to Julia. Perhaps there is a way to change these defaults. I had to use a mintty that @vtjnash provided me. The latest binaries did not seem to work for me.

[oif2003]

I noticed the mintty binary you provided leaves an extra blank "mintty.exe" window open when starting julia via the command line with "mintty %julia path%\julia.exe". If I send ctrl+c to that blank window, it interrupts code running in julia without closing out mintty.exe. This is not the behavior I get on mintty 2.9.5. At least as a crappy work around we can easily hide the extra window, intercept ctrl+c while julia REPL is the active window and redirect it to the hidden mintty.exe window.

[oif2003]

One catch though, ctrl+c sent while julia is not executing code will result in it exiting.

EDIT:
It appears that the byte at memory location 0x1004AA3C8 stores julia's execution state. 1 = ready, 0 = busy. This is for 64bit julia.exe.

EDIT2: The address is for Mintty.exe that is hosting julia.exe.
I have made two scripts based on my previous post. One using Mintty 3.0.2, which crashes on cd(). The other using Mintty 2.9.5. The latter does not crash on cd(), and instead of sending keystrokes, it sends SIGINT directly to julia.exe via Windows API.

[aminya]

I found this new cross-platform and customizable Terminal recently. It supports

• Hyperlinks
• Image
• Mouse support
• ...
  https://github.com/ismail-yilmaz/upp-components/tree/master/CtrlLib/Terminal

It supports Linux, Windows, macOS, and even a web browser!
Watch a demo here:

[davidanthoff]

Windows Terminal 1.0 shipped today.

[ViralBShah]

Windows Terminal is the thing to use. However, the issue is that it seems to require a much more recent version of Windows than what we support. Is it possible to ship Windows Terminal (no idea how easy it is) and fallback to the cmd if it doesn't start?

Does winget solve all these problems for us somehow?

[StefanKarpinski]

Can we just close this issue already? We’re clearly never going to do anything about it.

[aminya]

Windows Terminal is not a terminal itself. It is a terminal manager and it uses other terminals. If we want to use a Microsoft Terminal, we should use PowerShell Core.

What I suggested here is a real terminal with mouse and image support.
#7267 (comment)

[musm]

Can we just close this issue already? We’re clearly never going to do anything about it.

100% Please do so. Nothing productive has come out of this discussion.

[aminya]

Can we just close this issue already? We’re clearly never going to do anything about it.

100% Please do so. Nothing productive has come out of this discussion.

This issue is not solved yet. There is an unsubscribe button for opting out of the notifications.

[PetrKryslUCSD]

Most (all?) of the issues with the inadequate access to the operating system facilities through the shell mode can be solved by running the editor from within Git bash. For instance be VS code, Atom, Sublime Text. All would start Julia in a capable terminal emulator, with the expected Linux commands working in the shell mode. In my opinion, adding support for the Windows Terminal will add nothing but complexity.

[StefanKarpinski]

There is an unsubscribe button for opting out of the notifications.

Good point. This is officially the first Julia issue I’ve ever unsubscribed from. Someone send me a hologram when this issue gets resolved closed without doing anything in 2026 👋🏼

[ViralBShah]

I did install Git-Bash and simply use that.

[musm]

Windows Terminal is not a terminal itself. It is a terminal manager and it uses other terminals. If we want to use a Microsoft Terminal, we should use PowerShell Core.

What I suggested here is a real terminal with mouse and image support.
#7267 (comment)

This is not true. PowerShell Core, or now simply PowerShell (as of version 7), is a shell, just like cmd, pwsh, and bash are. On the other hand, Windows Terminal is a console or terminal, just like the myriad other terminals that are available, including MinTTY, ConEmu etc.

[aminya]

I am referring to the GUI of the terminal/console/shell. I am not concerned with the name. Julia already has a REPL with its rules, and using it from CMD is similar to using it from PowerShell. The good reason to use something new is to add some GUI features.

Windows Terminal does not add much GUI features, and it needs a shell (like CMD or PowerShell) to be called from inside it.

[stevengj]

Then can we stop worrying about Unicode rendering problems for people using the Windows Console?

For example, can we merge #33821 and tell Windows users who don't like seeing sparse matrices displayed as ?????? to download a better terminal?

Or do we continue to limit our display capabilities to the low bar set by the Windows console for the next 5-10 years, until the Windows Terminal is the default on all supported Windows systems? (We currently support the 11-year old Windows 7. How long will it be until we drop support for all Windows systems that don't ship with Windows Terminal?)

[musm]

For example, can we merge #33821 and tell Windows users who don't like seeing sparse matrices displayed as ?????? to download a better terminal?

IMO, some may disagree, but I think this is a perfectly reasonable approach, since it allows us to innovate. We can add Windows 7, to "supported", but not all functionality in terms of rendering fonts may be available by default without installing a better terminal, to the platform list.

While Windows 10 remains fully supported and we add a link the new Windows Terminal. For example, that PR renders perfectly with the new Terminal, since it also includes a decent font with ligatures.

[ViralBShah]

I think we should merge and move on. Given the ample number of options, we should not consider this our problem. I would also consider that a reasonable way to close this issue - in that we won't ship a terminal (we don't ship one on any OS).

[stevengj]

Okay.

(There have always been better terminals available for Windows, so I expect that the release of the Windows Terminal won't really resolve anything for several years, until it becomes the default — it won't stop the endless stream of complaints about Unicode rendering in Windows. Most Windows users still won't know how run Julia in a better terminal. But if we officially give up on out-of-the-box experience and say "get a better terminal," I guess that is one way to answer those questions.)

[ViralBShah]

Yes there have been better terminals available, but it somehow feels to me that it is much simpler today than even a year ago to install them.

To avoid the complaints - is there a way to test if a "good" terminal is being used so that a warning can be issued?

[ViralBShah]

A lot of windows users also use Juno/vs-code nowadays which probably do ok with Unicode display.

[ViralBShah]

Just like the other OSes, we will now expect users to "bring your own terminal". Once Windows Terminal is more widespread, hopefully we can detect it and start Julia using that, etc.

Documentation is updated about Terminals, and we have also updated the Downloads page to require Windows 10.

[davidanthoff]

Just a FYI on the latest developments, see here. TLDR: MS is slowly starting to make the new terminal the default for everything on Windows 11 now. I assume that once that has finished rolling out, the Julia start menu shortcut will just automatically start things in the new terminal.