Allow draw png images in 8 bit character format

[carlos-montiers]

I want allow draw png images in 8bit character format in the console.
The idea is this next:

call @data sun.png
This will load the data of the png image, convert to 8bit and save in an special storage, in this case "sun" (extracted from the name)

Subsequent calls to call @data sun.png will no process again, if it found that the "sun" key is found in the storage it will no save anything new.

call @data /D sun.png
remove from the storage.

Then, a new extension called @draw will allow draw that images to the console.

call @draw sun [row] [column]

Also will be possible a extension called @canvas that will allow create an image (not properly an image, a strcuture that represent the data) transparent or filled with some color. Thus you can draw more than one image to the canvas, and after that call @draw to the canvas

call @canvas mycanvas 100 100
call @paint mycanvas sun [row] [coord]
call @draw mycanvas

I prefer the png because support alpha color, thus in the conversion is possible support transparent "pixels" that will use the current background of the structure.
I think the intermediate format structure should be something like an CHAR_INFO structure array with a COORD for indicate the size. For the transparent "pixels" the character 0 can be used as convention.

Here attached an example of draw a 24 bit image as 8 bit image.

A user called KKziomek write a tool similar to this called bit8img
BIT8IMG Tool v1.0 build 003.zip

But is uses C++, an uses bitmap, that not support transparency.
I prefer png because allow compression an support transparency.

I thinked in use a library called libspng for do the task.

This tool will allow draw nice menus, or backgrounds for the console.

[adoxa]

I think converting images to character data should be a separate tool, not an extension; the extension can just read the character data.

[lazna]

guys, this is great idea! A had searching for such tool in past, but found nothing.

Maybe dumb question, but (undestood 8bit colors for older systems) why not do 24bit colors too?

[adoxa]

It's not really 8-bit (256 colours), but 4-bit (16 colours; foreground and background combined make eight bits). The "terminal" has 256/24-bit colour, so porting catimg would be a better idea (it already supports transparency, too, and the higher resolution would work no problem). ANSICON reduces 24-bit to 4-bit, so that could be used/included for older systems (although perhaps bit8img with its stippling might be better).

[carlos-montiers]

I think is better have the resources as png files instead of non standard binary format. This allow more easy edit. Will be nice also allow save the processed (downsampled) image to png, for iterate the edition for get the proper result (somethink like pixel art edit).
Thus something like call @img4bit input.png output.png

About create a separate tool, it will allow use other languages like C++.
But I will prefer have the logic in EB.
What about use libspng? I like that because is focused on security (very important for handle binary formats) and ease of use. Here are easy examples of use: https://libspng.org/comparison/

About port catimg I'm not sure.
I will prefer something more specific, simple code like the used in bit8img.

[carlos-montiers]

ANSICON reduces 24-bit to 4-bit, so that could be used/included for older systems (although perhaps bit8img with its stippling might be better).

I prefer get the logic instead of add ansicon as a dependency to the project.

[adoxa]

Will be nice also allow save the processed (downsampled) image to png [...]

So not only do you want to read png, you want to write png, as well. Bit8img is 104k, which will double the size of EB. Don't know how big catimg is, but I can't imagine it being much different.

But I will prefer have the logic in EB.

Then you'll have to write it yourself or find someone else, because I'm not doing it. Having image processing in EB is going way beyond what it should be doing.

[carlos-montiers]

So not only do you want to read png, you want to write png, as well. Bit8img is 104k, which will double the size of EB. Don't know how big catimg is, but I can't imagine it being much different.

Is 104 KB because is C++ that uses a lot of crt. Also uses CImg.h (2,75 MB of code). On the other hand spng.c + spng.h are only (120 KB).

Then you'll have to write it yourself or find someone else, because I'm not doing it. Having image processing in EB is going way beyond what it should be doing.

I understand, not problem. I will try do it first before ask for help. I'm sure that doing it in C will result in a tiny size result.

About "a image processing" maybe for optimize the things, EB can only read images and the read and write can be a separate tool. I'm not sure.

[adoxa]

I am sure - EB should only read character images, a separate tool should create them.

[carlos-montiers]

I am sure - EB should only read character images, a separate tool should create them.

Use libspng requires zlib, that is other complexity for compile.
Thus, I will create the tool with golang https://golang.org/pkg/image/png/
and the output format (surely spr) will be readed by EB. This cancel my idea of read png direct. Spr is more simple format for read than png, and in this case is only needed spr.

[adoxa]

stb_image.h (used by catimg) supports PNG without zlib (as well as other formats). If you're willing to switch to Go, there's also the native GDI+ in C++.

Bear in mind spr does not support 8/24-bit colour, unless you're going to create another format. Hm, might not be a bad idea: use CHARINFO for pre-10 and control sequences for 10 (or ANSICON/ConEmu). Probably double the size, but some form of simple compression should help with that.

[carlos-montiers]

I write an initial png reader with Go and the size is 2.3 MB, too much for me.
About native GDI+ in C++, the only disadvantage that I see is that I read that the gdi dll is not installed on windows 2k (seems none uses that windows 2000 version) but we are targeting all the cmd versions since windows 2k, thus I think the image processing tools should also be supported on that.
About stb_image.h I think that is only needed support png images, no other formats. Thus I think we will have a overhead in the final size because the library includes support for other formats.

Finally I compiled the example of libspng with proper crt optimization and the size for x86 is 78 KB.
Thus, because this library is focused on security I will go with this.

[carlos-montiers]

About the 24 bit colour, the example of Microsoft show it using Bash. Is possible display more than 16 colors in cmd ?

[adoxa]

I believe GDI+ has a redistributable for 2K and stb_image has defines to remove unwanted formats.

The terminal escape sequences provide a fixed palette of 256 colours (16 ANSI colours, 216 6x6x6 RGB cube and 24 grey scale), as well as RGB colours; these are supported by ConHost since 10.0.15063.0, I think. CHARINFO is still only the 16 colours, though, so you could use sequences to draw in many colours, but you can't then copy the rectangle to keep them.

It took ages to get those percents right, I'll have to see if I can do something about it.

[carlos-montiers]

Jason the example is really nice. Thus print 24 bit color is only possible through escape sequences?.
Programatically using win32 api is not possible using WriteConsoleOutput because it uses CHAR_INFO ? CHARINFO Attributes field is WORD thus only 16 bit.
Using libspng with proper optimizations I get 49.5 KB.
About gdi, I'm not sure if that support really good all the png formats, I remember that old IE6 had a png issue, maybe because gdi? I think libspng can work better because is more modern.

Hm, might not be a bad idea: use CHARINFO for pre-10 and control sequences for 10 (or ANSICON/ConEmu). Probably double the size, but some form of simple compression should help with that.

I think that this is the approach. thus is needed defined a new format (not spr) that support correctly 24 bit. Maybe something like CHAR_INFO but with a Attributes field using DWORD.

[carlos-montiers]

I found this quote that will not be a new api for print using 24 bit colors:

We're not planning on expanding the Win32 API surface for this feature. Every function we add to the Console API just creates another point of incompatibility with linux terminal applications.
If you want to use the new color support in your application, you'll have to use VT sequences.

microsoft/WSL#345 (comment)

[adoxa]

And I found microsoft/terminal#292 suggesting that there's not going to be an API to read it, either, but, if we're lucky, possibly a file.

The new format would contain a literal string which is literally echoed - that's why catimg is good: tweak its output (replace newline with line down and left movements) and use it as-is. Make the string UTF-8 and apply some form of compression to that. There's 30 control characters (avoid NUL & ESC) probably not being used, so remap those to two characters (\377 is an invalid UTF-8 byte, so use that as a prefix) and use the single character to represent multiple characters. E.g. \e[38;2; would occur a lot, so it could be reduced to a single \1 character, saving six characters each time. Or just use a small compression library like LZO and compress the whole format.

Sixel graphics might appear one day, which would be nice.

I fixed the percent problem.

[carlos-montiers]

Maybe have a format that store the "pixel character representation" in CHAR_INFO and with a literal string for escape sequence ?
What about have a buffer of the console, this will avoid also call to ReadConsoleOutput. Accordingly to a comment found here: Commented here: https://github.com/misol1/cmdgfx/blob/master/cmdgfx.c

ReadConsoleOutput doesn't seem to read more than ~15680 chars, then it's all garbled characters!! Have to read in smaller blocks

Thus, I think will be very practical have a buffer with the data of the console in our own format (the most updated possible). This is also the key I think for support "transparent characters", we use the character from our buffer.

[adoxa]

From the docs: "The total size of the array must be less than 64K." How big an image do you want? 16K allows for 128x128 or about 160x100, which seems plenty big enough to me.

An opaque image could use WriteConsoleOutput, but a transparent image needs its own format, anyway (assuming transparent is really transparent and not just space). Bit8img writes four characters, transparency is effectively a fifth, so may as well as make it eight bits (although could probably remove dark shade and replace it with an inverted light shade, then it could be four bits, allowing two characters per byte), with the colour makes 16 bits. Hm, it might be better to separate character & colour, allowing RLE to be used on each (in which case character would probably be better as eight bits, again). If a rectangle isn't that much quicker than a buffer, keep it simple and just have a buffer even for opaque images. Actually, the buffer could know its not transparent so expands to the CHAR_INFO rectangle.

[adoxa]

Another approach is to draw the image directly.

It won't work pre-Vista (they lack the function to retrieve the current font and I want to keep it as character coordinates, not pixel); it should work on 10, after I've done some tweaking (yet to try); it won't work on Windows Terminal.

[carlos-montiers]

Really cool. It doesn't matter that it doesn't work on windows terminal. I really want this feature in EB.
If it uses gdi, maybe a trick or workaround for when the user minimizes the window and all gdi pixels are erased ?

[carlos-montiers]

About the character image mode, I not agree on create a external tool for convert to EB format, this are my reason. I want support only 24 bit png, using this: https://github.com/apankrat/lpng
What are the reasons for use only png 24 bit. A comparision, for example for this attached image sizes in different formats:

png (24 bit with pngout): 259 bytes
png (24 bit) 634 bytes
pcx: 2.33 KB
gif (color lost) 2921 bytes
bmp (24 bit) 20.7 KB
bmp (256 colors) 7.94 KB
bmp (16 colors) 3.72 KB
jpg (is not lossless) 2.78 KB

Is not need support other format than png 24 bit.

[carlos-montiers]

About the IMAGE extension: for Windows Terminal what about use a predefined or hardcoded font like "Consolas"?

[carlos-montiers]

I implemented a light version of read png using a library called upng (https://github.com/elanthis/upng). The extension using this increases the size of the dll 10 KB only. The branch is: feature-img4bit-upng-dev

[adoxa]

If it uses gdi, maybe a trick or workaround for when the user minimizes the window and all gdi pixels are erased ?

The trick is to read the documentation, where it will say if the window is minimized anything drawn by GDI is lost. Same if you scroll, or maximize, or change size.

What are the reasons for use only png 24 bit. [...]

What are the reasons for not using it? People have their images in other formats. So because you want to do it in EB, and you want to keep it small, you want every other batch file writer to convert their images to 24-bit PNG (and use pngout to make it smaller). If you must process images directly, use GDI+, it's hardly any size at all.

About the IMAGE extension: for Windows Terminal what about use a predefined or hardcoded font like "Consolas"?

Windows Terminal uses the font defined in its settings, totally overriding the console properties. Besides which, GDI simply doesn't work - I think Direct2D has precedence.

[carlos-montiers]

I tried it on Windows Terminal and seems when I run call @image the program never returns, thus maybe add a validation for disable execution if is runned from Windows Terminal ?

[adoxa]

It's not even supposed to run on WT at all (if ... onWindowsTerminal ...) return FALSE;).

[adoxa]

I had no problem using it on WT - just did nothing. Tried Direct2D, but that did nothing, too. Looks like WT will miss out - have to wait for sixel.

On 10 it looks like pressing Tab (file name completion) will also remove GDI.

[adoxa]

Did a force push of the feature-gdi branch (probably easiest to delete the branch and pull it in again). Almost merged it in, but still don't think it's quite ready (and if/when I do merge it'll be squashed). I've added vertical text, which was fine for the font I was testing (TerminalVector), but looked really awful with Lucida and Consolas (the characters all squished together). Don't know if I want to go to the trouble of making it work, or just remove it (maybe keep the escapement/orientation options and just say user beware). Pre-10 still leaves GDI remnants behind, but I was going to add call @clear to erase the window (or a specific region, with options for colour and character). I could add variable @drawn which keeps track of the region drawn by GDI, then you could just do call @clear %@drawn% to erase it.

[adoxa]

I've added @clear (and updated the demo to use it), but not @drawn (and probably won't - just clear the entire window). The demo also works properly in 10 (and explicitly refuses to run in WT). Carlos, if you're not bothered about the issues with /V I'm now content to merge this in; if you are bothered, maybe merge it in anyway and sort it out later...

[carlos-montiers]

Jason, I agree on merge it, and create a new issue for the /V. Maybe for solve that a option can be only choose the fonts that works properly? I not understand fully the issue.

[adoxa]

Here's my version of bit8img. It currently uses the "legacy" palette, but an extension would use the real palette (for proper dithering). The extension would also resize based on font dimensions, but this version needs manual resizing (in order to preserve proportions). The default is to use the "dark shade" (U+2593); the "light shade" (U+2591) can be used by specifying light before the file name. It also uses "full block" (U+2588), but it might be better to use space, as small Consolas sizes aren't drawn properly. Transparency is supported, but currently it must be total (i.e. an alpha of 0 is transparent, but 1 is not).

Here's Carlos reduced to 45x30 characters (8x12 font).

[carlos-montiers]

Jason, the result is awesome, much better than the original bit8img. Please we can merge it to EB?
Also maybe we can implement the 1x1 font size for more nicely draw

[adoxa]

I am not going to support a 1x1 font, use @image or a proper GUI for that. I will merge in feature-gdi, then add this as @img (in gdi.c, to keep all the GDI+ functions together). If it's feasible to change the palette the results are even better. I used IrfanView to resize to 45x30 and reduce color depth to 4 bits, setting the resulting palette in the console.

I don't think EB should change the palette automatically, though, so there's another extension.

I've also updated catimg, which has double the vertical resolution (two image lines per character line). Here's that version with the legacy palette and the same custom palette (color-matched by ANSICON).

With 10 we can forget about the palette and have proper RGB images.

[carlos-montiers]

Feature implemented nicely by @adoxa in the extension called @img