Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Use in-memory thumbnail APIs when possible #3817

Merged

Conversation

jmartinesp
Copy link
Member

Content

  • Instead of using Client.getMediaFIle(...) to load thumbnails, use Client.getMediaThumbnail(...). This fixes a bug with upcoming new Rust APIs where thumbnails for videos weren't being displayed.
  • Add thumbnail dimensions info to TimelineItemImageContent and TimelineItemVideoContent.
  • For images in timeline items, try loading the thumbnail source if it exists, use the actual media source otherwise.
  • Add default values for thumbnail sizes if none is provided.
  • Removes redundant ByteArray.toUByteArray().toByteArray() transformation. This was needed before because uniffi returned something like Array<UByte> instead, but it now returns a proper ByteArray.

Motivation and context

Fix issue in upcoming SDK versions, use the right APIs when possible.

Tests

  • Clear the cache.
  • Open a room with media items.

Tested devices

  • Physical
  • Emulator
  • OS version(s): 14

Checklist

  • Changes have been tested on an Android device or Android emulator with API 23
  • UI change has been tested on both light and dark themes
  • Accessibility has been taken into account. See https://github.com/element-hq/element-x-android/blob/develop/CONTRIBUTING.md#accessibility
  • Pull request is based on the develop branch
  • Pull request title will be used in the release note, it clearly define what will change for the user
  • Pull request includes screenshots or videos if containing UI changes
  • Pull request includes a sign off
  • You've made a self review of your PR

@jmartinesp jmartinesp added the PR-Bugfix For bug fix label Nov 6, 2024
@jmartinesp jmartinesp requested a review from a team as a code owner November 6, 2024 11:36
@jmartinesp jmartinesp requested review from bmarty and removed request for a team November 6, 2024 11:36
Copy link
Contributor

github-actions bot commented Nov 6, 2024

📱 Scan the QR code below to install the build (arm64 only) for this PR.
QR code
If you can't scan the QR code you can install the build via this link: https://i.diawi.com/ziXR77

Copy link

codecov bot commented Nov 6, 2024

Codecov Report

Attention: Patch coverage is 48.27586% with 15 lines in your changes missing coverage. Please review.

Project coverage is 82.88%. Comparing base (98bf685) to head (daba49c).
Report is 56 commits behind head on develop.

Files with missing lines Patch % Lines
...l/timeline/model/event/TimelineItemImageContent.kt 35.71% 5 Missing and 4 partials ⚠️
...ctories/event/TimelineItemContentMessageFactory.kt 0.00% 0 Missing and 4 partials ⚠️
...timeline/components/event/TimelineItemVideoView.kt 33.33% 0 Missing and 2 partials ⚠️
Additional details and impacted files
@@             Coverage Diff             @@
##           develop    #3817      +/-   ##
===========================================
- Coverage    82.91%   82.88%   -0.03%     
===========================================
  Files         1772     1772              
  Lines        44534    44551      +17     
  Branches      5227     5235       +8     
===========================================
+ Hits         36924    36928       +4     
- Misses        5780     5784       +4     
- Partials      1830     1839       +9     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

)
} else {
MediaRequestData(
source = content.thumbnailSource,
Copy link
Member

@ganfra ganfra Nov 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we should keep content.preferredMediaSource and always use kind = MediaRequestData.Kind.Thumbnail

Copy link
Member

@ganfra ganfra Nov 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Except, the thumbnailAPI is probably not working with GIF?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Nope, it doesn't work properly... I guess we'll need to check the mime type here too.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is already done in preferredMediaSource, so maybe we could have a mediaRequestData extension function in Content, to pick the correct source and kind?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I tried something like that in 6da362e.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

But why do you want to use Kind.File when using mediaSource (except for isMimeTypeAnimatedImage I mean).
I'd have done something like that :

val thumbnailMediaRequest: MediaRequestData by lazy {
        if (mimeType.isMimeTypeAnimatedImage()) {
            MediaRequestData(
                source = mediaSource,
                kind = MediaRequestData.Kind.File(
                    fileName = filename,
                    mimeType = mimeType
                )
            )
        } else {
            MediaRequestData(
                source = thumbnailSource ?: mediaSource,
                kind = MediaRequestData.Kind.Thumbnail(
                    width = thumbnailWidth?.toLong() ?: MAX_THUMBNAIL_WIDTH,
                    height = thumbnailHeight?.toLong() ?: MAX_THUMBNAIL_HEIGHT
                ),
            )
        }
    }

and removed the preferredMediaSource?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Used this in daba49c.

Also add `TimelineItemImageContent.thumbnailMediaRequest` lazy property.
@jmartinesp jmartinesp force-pushed the fix/jme/use-specific-thumbnail-apis-for-loading-thumbnails branch from 8aaa467 to 6da362e Compare November 6, 2024 14:43
Rename `thumbnailMediaRequest` to `thumbnailMediaRequestData`.
Copy link

sonarcloud bot commented Nov 6, 2024

@jmartinesp
Copy link
Member Author

Just a heads up: this is needed to fix a bug with the media upload + send queue implementation, so it should be merged soon-ish.

Copy link
Member

@ganfra ganfra left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks for the changes, LGTM!

@ganfra ganfra enabled auto-merge November 7, 2024 19:46
@ganfra ganfra added the Run-Maestro Starts a Maestro Cloud session to run integration tests label Nov 7, 2024
@github-actions github-actions bot removed the Run-Maestro Starts a Maestro Cloud session to run integration tests label Nov 7, 2024
@jmartinesp jmartinesp merged commit 2873a68 into develop Nov 8, 2024
26 of 30 checks passed
@jmartinesp jmartinesp deleted the fix/jme/use-specific-thumbnail-apis-for-loading-thumbnails branch November 8, 2024 07:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
PR-Bugfix For bug fix
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants