Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support Image Resizing #52

Open
juhaodong opened this issue Aug 10, 2023 · 28 comments
Open

Support Image Resizing #52

juhaodong opened this issue Aug 10, 2023 · 28 comments

Comments

@juhaodong
Copy link

If we load a very large size picture the App will crash because oom problem, this is a IOS only issue. cause the IOS will draw the bmp in the memory first, if the picture size(4096*4096) not acutal file size is very large, then there will be a oom error when we use the picture.

@luca992
Copy link
Member

luca992 commented Aug 11, 2023

How big in MB? Any chance you tried lowering the filterQuality? not sure but that might help... Also, have you tried loading an image that big in a compose image without kamel? Maybe, it should be reported in the compose-multiplatform repo as well.

@luca992
Copy link
Member

luca992 commented Sep 7, 2023

I added an ios sample and tried modifying the gallery sample to use 5000X5000 images with https://picsum.photos/seed/1/5000/5000 and while it's slow to load in the ios simulator it's not crashing. If you can make a reproducible example by modifying the sample I can take a look.

@diegoberaldin
Copy link

I can confirm that on Android too, for large images, an exception is thrown:

java.lang.RuntimeException: Canvas: trying to draw too large(147424000bytes) bitmap.
	at android.graphics.RecordingCanvas.throwIfCannotDraw(RecordingCanvas.java:266)
	at android.graphics.BaseRecordingCanvas.drawBitmap(BaseRecordingCanvas.java:94)
	at androidx.compose.ui.graphics.AndroidCanvas.drawImageRect-HPBpro0(AndroidCanvas.android.kt:271)
	at androidx.compose.ui.graphics.drawscope.CanvasDrawScope.drawImage-AZ2fEMs(CanvasDrawScope.kt:263)
	at androidx.compose.ui.node.LayoutNodeDrawScope.drawImage-AZ2fEMs(Unknown Source:40)
	at androidx.compose.ui.graphics.drawscope.DrawScope.drawImage-AZ2fEMs$default(DrawScope.kt:510)
	at androidx.compose.ui.graphics.painter.BitmapPainter.onDraw(BitmapPainter.kt:93)
	at androidx.compose.ui.graphics.painter.Painter.draw-x_KDEd0(Painter.kt:212)
	at androidx.compose.ui.draw.PainterModifierNode.draw(PainterModifier.kt:347)
	at androidx.compose.ui.node.LayoutNodeDrawScope.draw-x_KDEd0$ui_release(LayoutNodeDrawScope.kt:92)
	at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:370)
	at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:359)
	at androidx.compose.ui.node.LayoutModifierNodeCoordinator.performDraw(LayoutModifierNodeCoordinator.kt:236)
	at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:367)
	at androidx.compose.ui.node.NodeCoordinator.access$drawContainedDrawModifiers(NodeCoordinator.kt:58)
	at androidx.compose.ui.node.NodeCoordinator$invoke$1.invoke(NodeCoordinator.kt:396)
	at androidx.compose.ui.node.NodeCoordinator$invoke$1.invoke(NodeCoordinator.kt:395)
	at androidx.compose.runtime.snapshots.Snapshot$Companion.observe(Snapshot.kt:2200)
	at androidx.compose.runtime.snapshots.SnapshotStateObserver$observeReads$1$1.invoke(SnapshotStateObserver.kt:234)
	at androidx.compose.runtime.snapshots.SnapshotStateObserver$observeReads$1$1.invoke(SnapshotStateObserver.kt:230)
	at androidx.compose.runtime.SnapshotStateKt__DerivedStateKt.observeDerivedStateRecalculations(DerivedState.kt:341)
	at androidx.compose.runtime.SnapshotStateKt.observeDerivedStateRecalculations(Unknown Source:1)
	at androidx.compose.runtime.snapshots.SnapshotStateObserver.observeReads(SnapshotStateObserver.kt:230)
	at androidx.compose.ui.node.OwnerSnapshotObserver.observeReads$ui_release(OwnerSnapshotObserver.kt:120)
	at androidx.compose.ui.node.NodeCoordinator.invoke(NodeCoordinator.kt:395)
	at androidx.compose.ui.node.NodeCoordinator.invoke(NodeCoordinator.kt:58)
	at androidx.compose.ui.platform.RenderNodeApi29.record(RenderNodeApi29.android.kt:209)
	at androidx.compose.ui.platform.RenderNodeLayer.updateDisplayList(RenderNodeLayer.android.kt:301)
	at androidx.compose.ui.platform.RenderNodeLayer.drawLayer(RenderNodeLayer.android.kt:242)
	at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:354)
	at androidx.compose.ui.node.LayoutModifierNodeCoordinator.performDraw(LayoutModifierNodeCoordinator.kt:236)
	at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:367)
	at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:359)
	at androidx.compose.ui.node.LayoutModifierNodeCoordinator.performDraw(LayoutModifierNodeCoordinator.kt:236)
	at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:367)
	at androidx.compose.ui.node.NodeCoordinator.draw(NodeCoordinator.kt:359)
	at androidx.compose.ui.node.LayoutNode.draw$ui_release(LayoutNode.kt:866)
	at androidx.compose.ui.node.InnerNodeCoordinator.performDraw(InnerNodeCoordinator.kt:151)
	at androidx.compose.ui.node.LayoutNodeDrawScope.drawContent(LayoutNodeDrawScope.kt:64)
	at androidx.compose.foundation.NoIndication$NoIndicationInstance.drawIndication(Indication.kt:136)
	at androidx.compose.foundation.IndicationModifier.draw(Indication.kt:183)
	at androidx.compose.ui.node.BackwardsCompatNode.draw(BackwardsCompatNode.kt:361)
	at androidx.compose.ui.node.LayoutNodeDrawScope.draw-x_KDEd0$ui_release(LayoutNodeDrawScope.kt:92)
	at androidx.compose.ui.node.NodeCoordinator.drawContainedDrawModifiers(NodeCoordinator.kt:370)
	at androidx.compose.ui.node.NodeCoordinator.access$drawContainedDrawModifiers(NodeCoordinator.kt:58)
	at androidx.compose.ui.node.NodeCoordinator$invoke$1.invoke(NodeCoordinator.kt:396)
	at androidx.compose.ui.node.NodeCoordinator$invoke$1.invoke(NodeCoordinator.kt:395)
	at androidx.compose.runtime.snapshots.Snapshot$Companion.observe(Snapshot.kt:2200)
	at androidx.compose.runtime.snapshots.SnapshotStateObserver$observeReads$1$1.invoke(SnapshotStateObserver.kt:234)
	at androidx.compose.runtime.snapshots.SnapshotStateObserver$observeReads$1$1.invoke(SnapshotStateObserver.kt:230)
	at androidx.compose.runtime.SnapshotStateKt__DerivedStateKt.observeDerivedStateRecalculations(DerivedState.kt:341)
	at androidx.compose.runtime.SnapshotStateKt.observeDerivedStateRecalculations(Unknown Source:1)
	at androidx.compose.runtime.snapshots.SnapshotStateObserver.observeReads(SnapshotStateObserver.kt:230)
	at androidx.compose.ui.node.OwnerSnapshotObserver.observeReads$ui_release(OwnerSnapshotObserver.kt:120)
	at androidx.compose.ui.node.NodeCoordinator.invoke(NodeCoordinator.kt:395)
	at androidx.compose.ui.node.NodeCoordinator.invoke(NodeCoordinator.kt:58)
	at androidx.compose.ui.platform.RenderNodeApi29.record(RenderNodeApi29.android.kt:209)
	at androidx.compose.ui.platform.RenderNodeLayer.updateDisplayList(RenderNodeLayer.android.kt:301)
	at androidx.compose.ui.platform.AndroidComposeView.dispatchDraw(AndroidComposeView.android.kt:1046)
	at android.view.View.draw(View.java:23197)
	at android.view.View.updateDisplayListIfDirty(View.java:22061)
	at android.view.ViewGroup.recreateChildDisplayList(ViewGroup.java:4513)
	at android.view.ViewGroup.dispatchGetDisplayList(ViewGroup.java:4486)
	at android.view.View.updateDisplayListIfDirty(View.java:22017)
	at android.view.ViewGroup.recreateChildDisplayList(ViewGroup.java:4513)
	at android.view.ViewGroup.dispatchGetDisplayList(ViewGroup.java:4486)
	at android.view.View.updateDisplayListIfDirty(View.java:22017)
	at android.view.ViewGroup.recreateChildDisplayList(ViewGroup.java:4513)
	at android.view.ViewGroup.dispatchGetDisplayList(ViewGroup.java:4486)
	at android.view.View.updateDisplayListIfDirty(View.java:22017)
	at android.view.ViewGroup.recreateChildDisplayList(ViewGroup.java:4513)
	at android.view.ViewGroup.dispatchGetDisplayList(ViewGroup.java:4486)
	at android.view.View.updateDisplayListIfDirty(View.java:22017)
	at android.view.ThreadedRenderer.updateViewTreeDisplayList(ThreadedRenderer.java:689)
	at android.view.ThreadedRenderer.updateRootDisplayList(ThreadedRenderer.java:695)
	at android.view.ThreadedRenderer.draw(ThreadedRenderer.java:793)
	at android.view.ViewRootImpl.draw(ViewRootImpl.java:4670)
	at android.view.ViewRootImpl.performDraw(ViewRootImpl.java:4381)
	at android.view.ViewRootImpl.performTraversals(ViewRootImpl.java:3600)
	at android.view.ViewRootImpl.doTraversal(ViewRootImpl.java:2328)
	at android.view.ViewRootImpl$TraversalRunnable.run(ViewRootImpl.java:9087)
	at android.view.Choreographer$CallbackRecord.run(Choreographer.java:1231)
	at android.view.Choreographer$CallbackRecord.run(Choreographer.java:1239)
	at android.view.Choreographer.doCallbacks(Choreographer.java:899)
	at android.view.Choreographer.doFrame(Choreographer.java:832)
	at android.view.Choreographer$FrameDisplayEventReceiver.run(Choreographer.java:1214)
	at android.os.Handler.handleCallback(Handler.java:942)
	at android.os.Handler.dispatchMessage(Handler.java:99)
	at android.os.Looper.loopOnce(Looper.java:201)
	at android.os.Looper.loop(Looper.java:288)
	at android.app.ActivityThread.main(ActivityThread.java:7872)
	at java.lang.reflect.Method.invoke(Native Method)
	at com.android.internal.os.RuntimeInit$MethodAndArgsCaller.run(RuntimeInit.java:548)
	at com.android.internal.os.ZygoteInit.main(ZygoteInit.java:936)

and using FilterQuality.Medium or FilterQuality.Low does not help. The crash happens both on emulators and on physical devices.
I'm stuck on version 0.7.1 because I have other libraries that do not allow me to update.

@luca992
Copy link
Member

luca992 commented Sep 23, 2023

Please provide a reproducible example. Or at least a link to load which fails.

@diegoberaldin
Copy link

Yes, this is the last one that gave me that issue:

https://lemmy.world/pictrs/image/922015bc-6491-4813-9308-da498e699873.jpeg

with a size of 4645439 bytes.

By the way, in the meantime, I managed to upgrade to version 0.7.3 and Compose 1.5.1 (I had to upgrade quite a few other dependencies so it took me some time).

Here is an example of how I'm using Kamel:

@Composable
fun PostCardImage(
    modifier: Modifier = Modifier,
    imageUrl: String,
    blurred: Boolean = false,
    onImageClick: ((String) -> Unit)? = null,
) {
    val painterResource = asyncPainterResource(
        data = imageUrl,
        filterQuality = FilterQuality.Medium,
    )
    KamelImage(
        modifier = modifier.fillMaxWidth()
            .heightIn(min = 200.dp)
            .blur(radius = if (blurred) 60.dp else 0.dp),
        resource = painterResource,
        contentDescription = null,
        contentScale = ContentScale.FillWidth,
        onFailure = {
            // ...
        },
        onLoading = {
           // ...
        },
    )
}

at this point I am starting to suspect the contentScale is to blame, or maybe the heightIn modifier.

@juhaodong
Copy link
Author

juhaodong commented Sep 25, 2023 via email

@luca992
Copy link
Member

luca992 commented Sep 26, 2023

It should relate to the pixel size crash merely only on real devices cause the limited RAM, you can try display 20 of 4k images on same screen, that should reproduce the crash Luca Spinazzola @.> 于 2023年8月11日周五 09:37写道:

How big in MB? Any chance you tried lowering the filterQuality? not sure but that might help... Also, have you tried loading an image that big in a compose image without kamel? Maybe, it should be reported in the compose-multiplatform https://github.com/JetBrains/compose-multiplatform repo as well. — Reply to this email directly, view it on GitHub <#52 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF73KQOUGMTUTDKK7OGUXZTXUWEGXANCNFSM6AAAAAA3LXGGBQ . You are receiving this because you authored the thread.Message ID: @.
>

@juhaodong got it. Unfortunately I don't have a real ios device to test on. I'll try soon on android though. Handling huge images may require some additional processing to resize before display, which I would have to do some research on how other libraries handle it. One thing you could try in the meantime as well is lowering the cache size: https://github.com/Kamel-Media/Kamel#cache-size-number-of-entries-to-cache

@juhaodong
Copy link
Author

It should relate to the pixel size crash merely only on real devices cause the limited RAM, you can try display 20 of 4k images on same screen, that should reproduce the crash Luca Spinazzola @.> 于 2023年8月11日周五 09:37写道:

How big in MB? Any chance you tried lowering the filterQuality? not sure but that might help... Also, have you tried loading an image that big in a compose image without kamel? Maybe, it should be reported in the compose-multiplatform https://github.com/JetBrains/compose-multiplatform repo as well. — Reply to this email directly, view it on GitHub <#52 (comment)>, or unsubscribe https://github.com/notifications/unsubscribe-auth/AF73KQOUGMTUTDKK7OGUXZTXUWEGXANCNFSM6AAAAAA3LXGGBQ . You are receiving this because you authored the thread.Message ID: @.
>

@juhaodong got it. Unfortunately I don't have a real ios device to test on. I'll try soon on android though. Handling huge images may require some additional processing to resize before display, which I would have to do some research on how other libraries handle it. One thing you could try in the meantime as well is lowering the cache size: https://github.com/Kamel-Media/Kamel#cache-size-number-of-entries-to-cache

Tried, but not working, this problem should be an iOS only issue, same code and image runs fine on Android.

@luca992
Copy link
Member

luca992 commented Sep 26, 2023

@juhaodong does it crash with a single image (like the one @diegoberaldin listed), or only after multiple? Also, a helpful test I'd be interested hearing about if you have time, is if you can download the images locally and display them in a normal non-kamel compose image without the app crashing on ios

@juhaodong
Copy link
Author

It should only crash the App if OOM happens on IOS, this is a IOS 'bug', can be avoid by resize the downloaded image before displaying them, otherwise it will try to draw a very large bitmap, then it will OOM. Android have GC, so normally it won't crash the App, but the Android have also a limit for the large image.

@juhaodong
Copy link
Author

@juhaodong does it crash with a single image (like the one @diegoberaldin listed), or only after multiple? Also, a helpful test I'd be interested hearing about if you have time, is if you can download the images locally and display them in a normal non-kamel compose image without the app crashing on ios

So it crashed as expected

@panpf
Copy link

panpf commented Oct 27, 2023

The image should be sampled according to the actual size of the Image component during decoding to ensure that the size of the image loaded into memory will not exceed the actual size of the Image component. The following APIs can be used:

  • BitmapFactory.Options.inSampleSize = 4 on Android
  • ImageReadParam.setSourceSubsampling(4, 4, 0, 0) on JVM

The image loading library on the Android platform does this, this prevents overly large images from being loaded into memory and causing the app to crash.

@rohitst
Copy link

rohitst commented Dec 27, 2023

Hitting this on iOS as well. I'm trying to display 6 images in a lazyRow and it will crash on scroll to the 6th image.

@luca992
Copy link
Member

luca992 commented Jan 15, 2024

Can someone check if the sample I just updated to include an xl image that causes a crash on android also causes a crash on ios? I don't have an iOS device and I can't reproduce this in simulator

@luca992 luca992 changed the title Very Large Size Picture crash IOS App Support Image Resizing Jan 15, 2024
@luca992 luca992 mentioned this issue Jan 15, 2024
@luca992
Copy link
Member

luca992 commented Jan 15, 2024

The image should be sampled according to the actual size of the Image component during decoding to ensure that the size of the image loaded into memory will not exceed the actual size of the Image component. The following APIs can be used:

* `BitmapFactory.Options.inSampleSize = 4` on Android

* `ImageReadParam.setSourceSubsampling(4, 4, 0, 0)` on JVM

The image loading library on the Android platform does this, this prevents overly large images from being loaded into memory and causing the app to crash.

So tried out using inSampleSize in the android decoder and it does work to prevent the error mentioned in #82

However, I'm trying to figure out the best way to implement image resizing in kamel and it's a bit tricky because asyncPainterResource is not a view, so there's not really a way to measure the view size without changing the api a bit. Started working on it here. Android works:

I modified asyncPainterResource to take a new maxBitmapDecodeSize param... Not sure, if I like that as it's only used for the bitmap decoder:

https://github.com/Kamel-Media/Kamel/blob/c7cf8c90a316128fecfb7691fcb014a541c30ab6/kamel-image/src/commonMain/kotlin/io/kamel/image/AsyncPainterResource.kt

Also I modified KamelImage to take a Resource<Painter> in a BoxWithConstraintsScope scope so that a new BoxWithConstraintsScope.asyncPainterResource scoped function can determine max size the image should be resized to.

And I use the determined size here to scale the image if needed:

I changed the behavior to always scale the image down if the display size is smaller than the image size:

https://github.com/Kamel-Media/Kamel/blob/c7cf8c90a316128fecfb7691fcb014a541c30ab6/kamel-image/src/androidMain/kotlin/io/kamel/image/decoder/AndroidImageBitmapDecoder.kt

If anyone has any opinions on possible better ways to do this lmk, it would be appreciated.

@amrfarid140
Copy link

amrfarid140 commented Feb 8, 2024

Can someone check if the sample I just updated to include an xl image that causes a crash on android also causes a crash on ios? I don't have an iOS device and I can't reproduce this in simulator

The iOS sample is missing the .xcodeproj file. Are you able to commit to the repo?

Without this file, cocoapods fails to install because there is no target project for it to configure.

@luca992
Copy link
Member

luca992 commented Feb 8, 2024

Can someone check if the sample I just updated to include an xl image that causes a crash on android also causes a crash on ios? I don't have an iOS device and I can't reproduce this in simulator

The iOS sample is missing the .xcodeproj file. Are you able to commit to the repo?

Without this file, cocoapods fails to install because there is no target project for it to configure.

Oh my bad, I checked it in 👍

@amrfarid140
Copy link

Thanks @luca992 ! just tested your "XL Bitmap" sample and it crashes on iPad Air (5th generation).

@luca992
Copy link
Member

luca992 commented Feb 8, 2024

@amrfarid140 perfect, haha. Thanks for letting me know. Just need to add a resizing implementation for iOS now... and come up with a strategy of when to resize as I was saying above

@amrfarid140
Copy link

amrfarid140 commented Feb 8, 2024

Yeah read your comment, looking at it as well wondering if delaying the Decoder.decode call and executing it in KamelImage composable would be viable.

Mostly internal APIs so no changes to asyncPainterResource and you are inside a composable where you can fetch its maximum size and scale the bitmap internally by default.

@luca992
Copy link
Member

luca992 commented Feb 11, 2024

Yeah read your comment, looking at it as well wondering if delaying the Decoder.decode call and executing it in KamelImage composable would be viable.

Mostly internal APIs so no changes to asyncPainterResource and you are inside a composable where you can fetch its maximum size and scale the bitmap internally by default.

Yeah, but I think then it would force people to use KamelImage and that's not required right now. Currently, you can just use asyncPainterResource with a standard Image if you want

@rohitst
Copy link

rohitst commented Mar 15, 2024

Any ideas on how we can pass the required image size in? This is a blocker for me unfortunately..

@luca992
Copy link
Member

luca992 commented Mar 15, 2024

@rohitst I could publish a special branch I started working on here to get feedback on the resizing if that helps. However, I only have only implemented resizing on android. I'd still need to find a replacement for Bitmap.createScaledBitmap to support non-android.

@FunkyMuse
Copy link

Hi, is there any update on this?

I'm getting OOM errors on Android, while it's not happening with the same images on iOS (perhaps the devices there are with better hardware) on Android it's happening even on higher end devices like S22 U.

@luca992
Copy link
Member

luca992 commented May 29, 2024

@FunkyMuse I could release this in the next 1.0.0 beta and add the decoder there as an optional decoder for android only while I figure out resizing on other platforms if that helps.

Help or research on how to handle other platforms would be appreciated, I only have so much free time.

@FunkyMuse
Copy link

@FunkyMuse I could release this in the next 1.0.0 beta and add the decoder there as an optional decoder for android only while I figure out resizing on other platforms if that helps.

Help or research on how to handle other platforms would be appreciated, I only have so much free time.

That'll be a start at least, I can't be of a good help since I've only Android knowledge.

Thanks.

@luca992
Copy link
Member

luca992 commented May 29, 2024

@FunkyMuse

you can try:

implementation("media.kamel:kamel-image-default:1.0.0-beta.6-SNAPSHOT")
implementation("media.kamel:kamel-decoder-image-bitmap-resizing:1.0.0-beta.6-SNAPSHOT") // android only

Working on it here:
#105

@luca992
Copy link
Member

luca992 commented May 31, 2024

released 1.0.0-beta.6 with #105 that includes kamel-decoder-image-bitmap-resizing with resizing support on android

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

7 participants