Implement urwid image widget

[AnonymouX47]

Resolves #71

• Adds term_image.widget subpackage and term_image.widget.urwid submodule.
• Adds the following widgets and related classes for the displaying images with the urwid TUI framework.
  • UrwidImage
  • UrwidImageCanvas
  • UrwidImageJanitor
  • UrwidImageScreen

One major feature left out in this PR is horizontal trimming of graphics-based images because it requires the implementation a non-trivial feature at the core of this library. This will happen in a few releases to come (tentatively planned for 0.8.0).

Credit: Major thanks to @danschwarz for his consistent suggestions and help with testing things out.

[AnonymouX47]

Hello @danschwarz! 😃

Please, I'll appreciate if you can help test this out.

[danschwarz]

Thanks! I pulled the urwid-widget branch and did a make install locally into the venv of my test app.

import term_image seems to work

mywidget = UrwidImage(img,format='RGB',upscale=False)

fails with NameError: name 'UrwidImage' is not defined.

I'm not a Python expert so this is probably an environment config issue on my side, but in case I'm just not importing the right library, let me know and that'll be an easy fix.

[danschwarz]

also tried from term_image import widget
then attempted to print out all functions in widget.. it prints []. So probably an environmental issue?

[AnonymouX47]

First of all, I apologize for not giving any directions earlier. 🤦🏾‍♂️

Did you install with make install?

If so, then term_image.widget shouldn't be empty, it should contain two classes, UrwidImage and UrwidImageCanvas. I just made sure of this by testing with a fresh virtual environment.

The key thing is the urwid package must also be installed (in the same environment) for the widgets to be available. You can do that by install urwid normally, though running make install does that.

As for importing the required parts of the library, this should be sufficient:

from term_image.image import AutoImage
from term_image.widget import UrwidImage

To use the widget, the docs should be of help.

You might need to familiarize yourself with the library first, the tutorial should be quick and sufficient.

Finally, I'll try to give a minimal working sample soon.

[AnonymouX47]

mywidget = UrwidImage(img,format='RGB',upscale=False)

This won't work even if UrwidImage were properly imported because the argument types/values are wrong. The pages I linked in my previous comment should help with that.

Basically,

• the first argument must be an instance of an image class defined in term_image.image.
• The second is an image format specifier, which is shown in use here in the tutorial. For now, you may omit this argument, the default should be fine.
• The third argument determines if the source image may be upscaled when rendering.

[AnonymouX47]

urwid_sample.py:

import sys

import urwid
from PIL import Image

from term_image.image import AutoImage
from term_image.widget import UrwidImage

class MyLoop(urwid.MainLoop):
    def process_input(self, keys):
        if "window resize" in keys:
            UrwidImage.clear()
        return super().process_input(keys)

def handle_input(key):
    if key == "q":
        UrwidImage.clear()
        raise urwid.ExitMainLoop
    return False

img = Image.open(sys.argv[1])
image = AutoImage(img)
image_w = UrwidImage(image)

base_widget = urwid.LineBox(
    urwid.Pile(
        [
            urwid.Columns(
                [urwid.LineBox(image_w, "Image")] * 2
            )
        ] * 2 
    ),
    "Images",
)

loop = MyLoop(base_widget, unhandled_input=handle_input)
loop.run()

python urwid_sample.py logo.png

produces

on Kitty and

on Terminator (VTE-based).

EDIT: Updated the sample code to clear graphics-based images properly.

[danschwarz]

Thanks, I'll try it when I have some free time. I read your comment in the code: ` If using an image widget with a :ref:`graphics-based <graphics-based>` render style as or within a **flow** widget, make sure to use a render method that splits images across lines such as the **LINES** render method for *kitty* and *iterm2* styles. ` This is an important point, and makes future sixel support in the widget seem unlikely. I have a widget that renders images using sixel but haven't figured out a way to get it to output a row at a time without creating a sixel image-per-row, which is likely to slow down or break Xterm.

…

On Sat, Feb 4, 2023 at 1:28 PM Toluwaleke Ogundipe ***@***.***> wrote: *urwid_sample.py*: import sys import urwidfrom PIL import Image from term_image.image import AutoImagefrom term_image.widget import UrwidImage def process_input(key): if key == "q": raise urwid.ExitMainLoop UrwidImage.clear() return True img = Image.open(sys.argv[1])image = AutoImage(img)image_w = UrwidImage(image) base_widget = urwid.LineBox( urwid.Pile( [ urwid.Columns( [urwid.LineBox(image_w, "Image")] * 2 ) ] * 2 ), "Images", ) loop = urwid.MainLoop(base_widget, unhandled_input=process_input)loop.run() python urwid_sample.py logo.png produces [image: Screenshot_2023-02-04_19-26-29] <https://user-images.githubusercontent.com/61663146/216783518-29ac8788-f69c-43ad-9e50-80642a600abf.png> on *Kitty* and [image: Screenshot_2023-02-04_19-27-44] <https://user-images.githubusercontent.com/61663146/216783545-ff083b1e-a8fb-46f2-8e22-7d9b524cb85b.png> on *Terminator* (VTE-based). — Reply to this email directly, view it on GitHub <#73 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAY4FJWLBMZVGQS5JF4CRKDWV2NVXANCNFSM6AAAAAAURC4P4I> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[AnonymouX47]

without creating a sixel image-per-row, which is likely to slow down or break Xterm.

I see. Unfortunately, that's the only way to achieve that :(

[AnonymouX47]

Please note that I just updated the sample code above to clear graphics-based images properly.

When images need to be cleared will majorly depend on your use case but one common case is a window resize.

[danschwarz]

Thanks for the extra documentation. With that, I was able to get the widget working properly in my test app.
ANSI and Kitty images look good.

One anomaly I've found with the ANSI images - I have one that entirely filled my screen down to the lower right corner. You can see the [ character printed over the image. I don't know why that is appearing. This is an UrwidImage enclosed in an urwid.Filler container, nothing more. Displaying on Windows Terminal/Ubuntu 22.04.

This is repeatable with any image displayed as ANSI that is as large, or larger than the terminal width/height.

Another example, this is a NASA image

[danschwarz]

I know there's some weirdness with VT100 terminal emulation and the last cell in the lower right position - urwid raw_display.py _last_row(..) method does some extra work to set the last row, rightmost cell. Perhaps something there is interacting with the ANSI graphics output? Just a guess.

[AnonymouX47]

I looked into it and as you mentioned, it's due to urwid.raw_display.Screen._last_row().

I have implemented a workaround in e8f89b2. Also, includes some comments explaining the problem.

The issue still affects, at least, the iterm2 style on Wezterm where the last line of the image is not drawn because technically, the image escape sequence is truncated/modified.
The workaround resulted in a different problem (scrolling the screen) which seems to be worse IMO. So, I have excluded this case from the workaround. The best option is to avoid having an image fall at the bottom right cell.

Please, let me know how this works on your end, particularly on Windows Terminal.

Thanks for all your feedback. 🙏🏾

[AnonymouX47]

The major thing keeping this from being merged is unit tests, such a necessary evil 😡 but I'm almost done with them 😃.

[danschwarz]

What is the minimum python version you’re targeting for the widget? The app I’m contributing to has a min version of 3.6 but that might get bumped up in future as 3.6 is EOL.

…

On Sun, Feb 5, 2023 at 8:44 AM Toluwaleke Ogundipe ***@***.***> wrote: The major thing keeping this from being merged is unit tests, such a necessary evil 😡 but I'm almost done with them 😃. — Reply to this email directly, view it on GitHub <#73 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAY4FJX4MXO3K5KDH26EMETWV6VC5ANCNFSM6AAAAAAURC4P4I> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[AnonymouX47]

Python 3.7 is the minimum supported version for the entire library.

[danschwarz]

Will do. I assume the widget won’t be available through pip until it lands in master branch and you create a release.

…

On Wed, Feb 22, 2023 at 3:02 AM Toluwaleke Ogundipe < ***@***.***> wrote: My image11 branch of toot <https://github.com/danschwarz/toot/tree/image11> integrates the latest term-image widget. As a reminder, when adding term_image as a dependency, please remember to pin it to a specific minor version, until version 1.0 is released. See here <https://github.com/AnonymouX47/term-image#usage>. — Reply to this email directly, view it on GitHub <#73 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAY4FJRH4W37C6GK5VPVW5TWYXBYFANCNFSM6AAAAAAURC4P4I> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[danschwarz]

I think it’d be best for an app to catch all unhandled exceptions and clear images in the exception handler before exiting. I haven’t tried this but it seems possible by using sys.excepthook. Responsibility of the app developer. As for clearing images on startup, option 1 seems like it’d work best, but it’s a lot to ask of an urwid developer who isn’t necessarily familiar with sending commands directly to the TTY. Sample code would help, or monkey patch urwid.Screen? #2 is more developer friendly, but if I read your note correctly, it depends on rendering an urwid image widget immediately on startup. Toot, for example, doesn’t do that (yet). It throws up a big text splash screen while it loads data. With #2 It would be necessary to render an image as part of the splash screen just to get the screen clearing benefits?

…

On Wed, Feb 22, 2023 at 2:52 AM Toluwaleke Ogundipe < ***@***.***> wrote: Another thing I should note at this point is that I haven't figured out a way to clear images upon a crash i.e when an unhandled exception is raised within MainLoop,run(). The closest solution to this is to clear images upon startup of the TUI i.e after urwid has switched to to alternate buffer. I've tried a few ideas: 1. The user overrides Screen.start() and clear all images after calling super().start(). - This requires either flushing the output buffer before clearing the images OR manually switching to the alternate buffer and clearing the images (and possibly switching back to the normal buffer) all by writing directly to the TTY. 2. Let it be handled by UrwidImagejanitor by clearing all images upon its first render. - I would prefer this in order not to bother users with No. 1 but there are a couple of issues that depend entirely on the user. - What if the first render of widget does not occur immediately after the urwid screen start up i.e before Screen.start() is called or later after urwid has switched to the alternate buffer and drawn the UI? - One way to solve the problem above is to make the clear at first render optional and disabled by default. What do you see to this? — Reply to this email directly, view it on GitHub <#73 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAY4FJQPZDTZWNIAUBKVZJ3WYXAUJANCNFSM6AAAAAAURC4P4I> . You are receiving this because you were mentioned.Message ID: ***@***.***>

[AnonymouX47]

I assume the widget won’t be available through pip until it lands
in master branch and you create a release.

True.

---

About clearing images at startup (and exit)...

Let it be handled by UrwidImageJanitor by clearing all images upon its first render.

I had actually implemented (and even pushed) this already but it smelt so much of "hackiness". So I reverted it.

I've just added a screen class to handle this in commit f0cb34b. This is definitely a better approach:

• By urwid.BaseScreen's design, it inherently takes care of unhandled exceptions via a context manager.
• It doesn't have any of the quirks associated with the previous approach (No. 2).
• It also handles clearing upon exit.
• It can be easily used with a custom screen via inheritance.

[danschwarz]

Wow. Do you sleep?

The latest changes are great. Added UrwidImageScreen support to toot, verified it clears images on startup/shutdown, even when ctrl-c exiting the app. (Yeah, subclassing Screen was a much better idea than monkeypatching the urwid Screen class 🙄)

Implemented a test for pixel vs block style rendering by creating an AutoImage and checking isinstance(ai, BlockImage). Works well. Now I display UrwidImage widgets using fewer rows when they're rendered using pixels, and more when they're rendered using blocks.

Can't think of anything more this needs, other than a release in pip. It's really working well.

My wishlist after that would be:

• Handle terminals like Apple Terminal that only support BlockImage @ 256 colors - no truecolor. (The code may do this already? I can't test until I get back to a Mac.)
• Sixel support 🙏

On my side, it's just polishing up the image display and refactoring some code. Oh, and I have to check with @ihabunek about dropping Python 3.6 support in toot, so we're at the same Python 3.7 baseline as term-image.

Thanks!

[AnonymouX47]

Finally, this PR seems to be coming to a wrap. I must say, we've really come a long way with this 🥲.

At this point, I think all initial features I had in mind and all that have come up along the way have been fully implemented (except for horizontal trim of images of graphics-based render styles) and I have written adequate tests for all.

Please help test it out and let me know of any issues you encounter or any other suggestions you have.

Thank you.

EDIT: Just saw your comment above after sending this.

[AnonymouX47]

Do you sleep?

🤔 Probably not :D

---

I'm really happy to hear that everything is working well.

Just two major things left to do before the next release, restructure the docs and tests (due to the fact that I've now separated the TUI). Not adding anything new, just re-organizing what's already there.

As for your wishlist, all that plus more will be coming in the release following this. See this milestone. I'll gladly appreciate your input in any of the issues listed.

[AnonymouX47]

I guess I'll wait till after I wake up before I merge... It's always best to go over one last time with a fresh brain :D

I can finally close all these terminal emulators and power off my PC in peace after days 😮‍💨.