Shell: Cat a large file (144MB, for example) and the shell app freezes while all the characters are being printed by JavaScript

[sync-by-unito]

xterm.js has a solution for this:

1. Use a circular list to manage buffer xtermjs/xterm.js#422
2. Improve xterm.js performance microsoft/vscode#17875

The easiest solution might be to move to using xterm.js instead of hterm.

In the short term we should consider documenting this limitation somewhere. This problem seems to make it difficult or impossible when working with the Apache Spark shell via hterm because commands will often print out a lot of data.

┆Issue is synchronized with this Asana task by Unito

[sync-by-unito]

➤ Jeremy Nicklas commented:

Ticket is here: INC0313174 ( https://oarnet.service-now.com/nav_to.do?uri=incident.do?sys_id=b7ecae1813517600804831f18144b071 )

The slowdown is caused by the repeated printing of data to the screen.

image ( https://cloud.githubusercontent.com/assets/4260509/23470289/05ef4182-fe74-11e6-8d90-9b4429366914.png )

One possible way to speed this up is to build an internal buffer before printing so that we call the print function less.

[sync-by-unito]

➤ Jeremy Nicklas commented:

Maybe look into throttling or debouncing.

[sync-by-unito]

➤ Eric Franz commented:

The two main problems are:

1. you don't see text (it is broken to the user -- except in IE you see the text)
2. you see the text, but can't type ctrl-c to stop the flow

Options for further exploration.

1. I think if we isolate the code hterm.Terminal.print, and manually call that with example arguments to produce output in the viewport, then given a local file (HTML5 local file system reading a file) we read 200MB of data and call this over and over again to reproduce the problem we see. If we can, we've just created a local example that we could experiment with different solutions.
2. lib.encodeUTF8, lib.wc.strWidth, lib.UTF8Decoder.decode etc. from the names all look like candidates for methods that could optimized (i.e. written in C complied using http://kripken.github.io/emscripten-site/) but that could be a big time sink and makes it more difficult to update hterm dependency in future versions

FWIW we only have had one complaint about this since the inception of OnDemand, and it was with using app (was it Apache Spark?) where the commands resulted in outputting large amounts of data.

[sync-by-unito]

➤ Jeremy Nicklas commented:

Possibly look into an ack workflow to slow down the flood of data. Where we send an ack after we render the current data in order to receive more data.