Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

FEAT(client,images): Add animated gif support #6638

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

GeneralUser01
Copy link
Contributor

The way images are handled for the gif file format when pasting and receiving one in the chat was here changed to support playing the animation. By default they do not play but whenever an animation is not running this is indicated by a play-icon, thereby differentiating them from still images on a glance. Whether the animation is paused is toggled by left-clicking it and reset by middle-clicking it. Saving a gif image file from log is also supported just as it is for other image file formats.

Additionally, one can also toggle video controls for animations via the log context menu on them. This enables the following features:

  • Jumping to any point of the animation via the video bar
  • Viewing the current time and total time of the animation
  • Switching caching of all frames on or off
  • Switching loop mode between "Unchanged", "Loop" and "No loop"
  • Traversing frame-by-frame backwards or forward
  • Changing and resetting the playback speed

To turn on caching most notably boosts performance when jumping to frames that are sequentially far apart due to QMovie only being able to switch frame in order from start to end and around when caching is off, though it usually plays fine regardless except for when playing in reverse in an animation with many frames. Reverse playback is also implemented here, so when decreasing the speed to less than zero it will play at that speed in reverse as expected, taking a speed-step that's twice as big if the speed would otherwise become zero so that the animation only pauses when it's not playing. As for loop mode, "Unchanged" is to use the in-built setting in the gif image file for how many times it is to repeat, whereas "Loop" and "No loop" override this behavior accordingly.

The main limitation is the character limit on text messages for images. Currently this is usually set to 128 KB, which is very small for a gif image file and on top of this requires the data to be 33% smaller before being sent in base64 encoding. This limit should be at least ten times larger in order to fit many gif image files, or this should apply to another limit specifically for animations. Other than that, the functionality for pasting images from the clipboard itself, instead of by a file path from it, could not be implemented for gif image files due to not being able to get compatible formatted data from the MIME data received.

Here are workarounds that can be applied to these limitations:

  • Configure the server to increase or disable the limit on image message length
  • Save copied gif images to file and then copy and paste those files

Lastly, if settings were to be added to this feature, then here are some suggestions for those settings:

  • Play animations by default
  • Show video controls by default
  • Cache all frames by default
  • Specify the default loop mode
  • Specify the default playback speed

Checks

The way images are handled for the `gif` file format when pasting and receiving one in the chat
was here changed to support playing the animation. By default they do not play but whenever
an animation is not running this is indicated by a play-icon, thereby differentiating them
from still images on a glance. Whether the animation is paused is toggled by left-clicking it
and reset by middle-clicking it. Saving a `gif` image file from log is also supported
just as it is for other image file formats.

Additionally, one can also toggle video controls for animations via the log context menu on them.
This enables the following features:
- Jumping to any point of the animation via the video bar
- Viewing the current time and total time of the animation
- Switching caching of all frames on or off
- Switching loop mode between "Unchanged", "Loop" and "No loop"
- Traversing frame-by-frame backwards or forward
- Changing and resetting the playback speed

To turn on caching most notably boosts performance when jumping to frames that are sequentially
far apart due to `QMovie` only being able to switch frame in order from start to end and around
when caching is off, though it usually plays fine regardless except for when playing in reverse
in an animation with many frames. Reverse playback is also implemented here, so when decreasing
the speed to less than zero it will play at that speed in reverse as expected, taking a speed-step
that's twice as big if the speed would otherwise become zero so that the animation only pauses when
it's not playing. As for loop mode, "Unchanged" is to use the in-built setting in the `gif` image file
for how many times it is to repeat, whereas "Loop" and "No loop" override this behavior accordingly.

The main limitation is the character limit on text messages for images. Currently this is usually
set to 128 KB, which is very small for a `gif` image file and on top of this requires
the data to be 33% smaller before being sent in base64 encoding. This limit should be at least
ten times larger in order to fit many `gif` image files, or this should apply to another limit
specifically for animations. Other than that, the functionality for pasting images from
the clipboard itself, instead of by a file path from it, could not be implemented for `gif`
image files due to not being able to get compatible formatted data from the MIME data received.

Here are workarounds that can be applied to these limitations:
- Configure the server to increase or disable the limit on image message length
- Save copied `gif` images to file and then copy and paste those files

Lastly, if settings were to be added to this feature, then
here are some suggestions for those settings:
- Play animations by default
- Show video controls by default
- Cache all frames by default
- Specify the default loop mode
- Specify the default playback speed
@Hartmnt
Copy link
Member

Hartmnt commented Nov 26, 2024

Thank you very much for this contribution! Very cool!

Please note that this feature - with utmost certainty - will not land in the 1.5.x branch of Mumble, but in 1.6.x. With that being said, we are currently trying to get another 1.5.x release out and lack time to review this properly right now. There will be plenty of time for getting this merged, though!

I have not looked into the details of this PR yet, but since it is touching the user interface I would like you to take a look at the Mumble accessibility checklist and double check the application can still be used with accessibility in mind with your change applied.

Also take a look at the CI pipelines which are currently failing (you can disregard the Windows pipeline for now as we need to fix it in master right now).
The other thing is, we are currently requiring all PRs to be auto-formatted using clang-format in version 10. This is currently also failing for your PR. It will be enough to format everything once all the changes are reviewed and ready.

Again, please be patient for a proper review of this :)

@Hartmnt Hartmnt added client ui feature-request This issue or PR deals with a new feature labels Nov 26, 2024
@GeneralUser01
Copy link
Contributor Author

GeneralUser01 commented Nov 27, 2024

Thank you for being open to merge this PR and for the feedback!

Since the interactive aspects of this feature are part of custom text objects in the log, keyboard-related controls such as tabbing to navigate apply to user messages but it is unclear how this should be able to interact with items in messages if at all. As for contrasts which is another accessibility concern, is there any standard used in this project to test with to check if a contrast is high enough?

The only part of the CI pipelines that I'm unsure about how to resolve are the translations. There are more texts than what this feature brings that have no translation, so how is this to be handled?

@Hartmnt
Copy link
Member

Hartmnt commented Nov 27, 2024

Since the interactive aspects of this feature are part of custom text objects in the log, keyboard-related controls such as tabbing to navigate apply to user messages but it is unclear how this should be able to interact with items in messages if at all. As for contrasts which is another accessibility concern, is there any standard used in this project to test with to check if a contrast is high enough?

Currently, the individual text messages (objects) are not accessible. This is a TODO on our part. However, I am concerned that your new text objects containing a video player may create focus traps when tab-navigating. Another aspect that needs to be looked at is using a screen reader or text-to-speech for chat messages. It would be bad for example, if the TTS would read the entire base64 content of the video. Severe things like that. Don't worry too much about the contrast. When we look at this in detail, we can still talk about that.

The only part of the CI pipelines that I'm unsure about how to resolve are the translations. There are more texts than what this feature brings that have no translation, so how is this to be handled?

There is a script in the repository under the scripts folder called updatetranslations.py. If you run this from the repo root, it automatically creates a translation commit. But I would advice to do this at the very end, as your changed translation strings may need to be updated.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
client feature-request This issue or PR deals with a new feature ui
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants