kitty graphics protocol support for NCBLIT_PIXEL

[joseluis]

What's your opinion about supporting the kitty graphics protocol? May it be worth it?

https://sw.kovidgoyal.net/kitty/graphics-protocol.html

[dankamongmen]

Good god that's aggressive and a mess and aggressively messy.

I'd be interested in supporting it but probably (at least initially) only through what it can give us that has some kind of commonality with other terminals. Perhaps NCBLIT_SIXEL ought instead be NCBLIT_PIXEL, and use whatever pixel-blitting method is available. Yes, under that umbrella I'd definitely want to support this.

kovidgoyal/kitty#33 has some....interesting statements.

<ESC>_Gi=31,s=10,v=2,t=s;<encoded /some-shared-memory-name><ESC>\
to which the terminal emulator will reply (after trying to load the data):

<ESC>_Gi=31;error message or OK<ESC>\

[vomit]

Man, if @kovidgoyal is feeling so froggy about fucking hard with the terminal paradigm, can we not just get a second communications channel on which to do these complicated, bidirectional escapes? kitty and friends could listen on an AF_UNIX socket somewhere and you could just connect up and send structured data like gentlemen.

[kovidgoyal]

@dankamongmen why dont you go look up the long history of bi-directional terminal escape codes. Especially if you want to set yourself up to write a TUI library. A minimum level of intelligence is table stakes in this game.

[dankamongmen]

@dankamongmen why dont you go look up the long history of bi-directional terminal escape codes. Especially if you want to set yourself up to write a TUI library. A minimum level of intelligence is table stakes in this game.

ahhh, @kovidgoyal, you hear me incorrectly -- my apologies for speaking offhandedly. i'm a big fan of kitty and what you've done there. i'm of course familiar with such, and employ them, i'm just saying it would be nice to move on to something better (and that doesn't come with builtin race conditions), much as you've done with the graphics.

peace, brother. i mean no ill will. sorry if i sounded disapproving of the great work you've done.

[dankamongmen]

@dankamongmen why dont you go look up the long history of bi-directional terminal escape codes. Especially if you want to set yourself up to write a TUI library. A minimum level of intelligence is table stakes in this game.

reading back over my comments, i understand why you felt insulted, and regret having tagged you among a bunch of flippancy without much context. be assured that i think kitty a very fine piece of work, regularly use it as my terminal emulator, and recommend it to many. sorry to have given offense -- all my fault, good sir!

[kovidgoyal]

No worries, internet communication is fraught with misunderstandings. And hey if you have ideas to improve the protocol I am all ears.

[dankamongmen]

No worries, internet communication is fraught with misunderstandings. And hey if you have ideas to improve the protocol I am all ears.

thanks for the gentlemanly response, and i will. btw, i might be filing a bug for you later today (i first need verify that it's not my own bug). every other terminal i've tested with is working properly with the xray demo from notcurses-demo, but kitty is drawing a transparent cell where i'm expecting cells with black background. like i said, it might very well be my bug in terms of overlooking some detail of ECMA-48 and friends. i wouldn't spend any time on this until i've (a) satisfied at least myself that it's an actual bug in kitty and (b) written up the bug properly, but see e.g. kitty on the left, alacritty on the right:

anyway, hack on man! much respet.

[kovidgoyal]

With regards to bug reports, best way to do it, is use the kitty --dump-bytes option, and generate a minimal case that shows behavior that is not as expected. Attach the dump file so I can repro easily.

[dankamongmen]

With regards to bug reports, best way to do it, is use the kitty --dump-bytes option, and generate a minimal case that shows behavior that is not as expected. Attach the dump file so I can repro easily.

will do, thanks kovid!

[dankamongmen]

Time to do this! I'd like 2.2.3 to have support for both sixel and the kitty protocol. We're just about complete with Sixel (see #1380).

[kovidgoyal]

cool feel free to ping me if you have questions/issues.

[dankamongmen]

Having read over the documentation, given that we already hold the data naturally as RGBA (coming out of ffmpeg or OIIO), we'll first go with that for all cases. I'll then want to run some experiments with deflate() (@kovidgoyal , do you have any data on whether the deflate step is worth it?). then, at some point, we might want to go for the shared memory, but that'll be a last step (especially since it's only applicable to local hookups).

The RGBA mode looks like simplicity itself, so I bet that can get done today.

@kovidgoyal , one thing I'm always interested in is the interplay of a graphic and other text. I'm guessing that "The other commands to erase text must have no effect on graphics. The dedicated delete graphics commands must be used for those." and the restriction on cursor placement mean that cupping into the middle of a graphic and printing are not supported? it'll be easy enough to test in a minute here.

[kovidgoyal]

On Mon, Mar 08, 2021 at 08:32:38AM -0800, Nick Black wrote: Having read over the documentation, given that we already hold the data naturally as RGBA (coming out of ffmpeg or OIIO), we'll first go with that for all cases. I'll then want to run some experiments with `deflate()` ***@***.*** , do you have any data on whether the `deflate` step is worth it?). then, at some point, we might want to go for the shared memory, but that'll be a last step (especially since it's only applicable to local hookups).

No, it very much depends on the image in question, and transmission medium and CPU speed. I didnt bother with it in icat, and I suggest you also skip it in your first pass.

The RGBA mode looks like simplicity itself, so I bet that can get done today. @kovidgoyal , one thing I'm always interested in is the interplay of a graphic and other text. I'm guessing that "The other commands to erase text must have no effect on graphics. The dedicated delete graphics commands must be used for those." and the restriction on cursor placement mean that `cup`ping into the middle of a graphic and printing are not supported? it'll be easy enough to test in a minute here.

Yes, you can. Once you put an image on the screen you can add text anywhere else and the image will not be affected. The only exception is scrolling which causes the image to also scroll. i.e. if you emit new lines the image will scroll off the screen just like text would. In fact by giving the image a -ve zindex you can ever write text on top of it. You can choose to have the image between the text and the background color, or even under both text and background color.

[dankamongmen]

we now have big, beautiful kitty graphics!