[Android] Image.Source = ImageSource.FromResource() & ImageSource.FromStream() are not reliably integrating back with the MainThread

[jonmdev]

Description

I am running the following code on a Behavior attached to an Image for managing the loading of photos to the Image. In Windows and iOS this can run 100+ times in a row with zero problems. I have never seen an error in them. But maybe 1 out of 5-7 times in Android fails with what appears to be a threading problem.

From what I can see in the Debugging, Image.Source = ImageSource.FromResource() runs an async process inside of it. This async process is not cooperating with the main thread in Android.

There seems to be no way to load an Image without using this async process inside it which is causing the problems.

Here is my overly cautious code to try applying an ImageSource to an Image:

private void applyPhoto() {
    MainThread.BeginInvokeOnMainThread(new Action(() => {
        if (resourceName != null && imageView != null) {

            Assembly assembly = GetType().GetTypeInfo().Assembly;

            Debug.WriteLine("IMAGE SOURCE FROM RESOURCE IS ON MAIN THREAD: " + MainThread.IsMainThread);

            imageView.Source = ImageSource.FromResource(resourceName, assembly);

            Debug.WriteLine("IMAGE SOURCE FINISHED");
        }
    }));

}

The Debug output confirms my command was run on the main thread as requested, and the command did not initially cause any errors. It debugs out the two Debug lines (one before and one after the ImageSource function) sequentially with nothing between and no problems.

Yet a few lines of debugging later it then spits out an endless stream of errors. This to me suggests the ImageSource command is running async and then failing to re-integrate with the MainThread once it gets done or is trying to mess with the Image when it finishes out of turn.

[0:] LOAD IMAGE FROM EMBEDDED RESOURCE IS ON MAIN THREAD: True
[0:] IMAGE SOURCE FROM RESOURCE IS ON MAIN THREAD: True
[0:] IMAGE SOURCE FINISHED

...//a few other random debugs from elsewhere in project (suggesting other things are still happening), then:

[EGL_emulation] app_time_stats: avg=2516.58ms min=389.68ms max=4643.47ms count=2
[0:] Microsoft.Maui.StreamImageSourceService: Warning: Unable to load image stream.

Android.Util.AndroidRuntimeException: Only the original thread that created a view hierarchy can touch its views.
   at Java.Interop.JniEnvironment.StaticMethods.CallStaticVoidMethod(JniObjectReference type, JniMethodInfo method, JniArgumentValue* args) in /Users/runner/work/1/s/xamarin-android/external/Java.Interop/src/Java.Interop/obj/Release/net7.0/JniEnvironment.g.cs:line 13250
   at Java.Interop.JniPeerMembers.JniStaticMethods.InvokeVoidMethod(String encodedMember, JniArgumentValue* parameters) in /Users/runner/work/1/s/xamarin-android/external/Java.Interop/src/Java.Interop/Java.Interop/JniPeerMembers.JniStaticMethods.cs:line 97
   at Microsoft.Maui.PlatformInterop.LoadImageFromStream(ImageView imageView, Stream inputStream, IImageLoaderCallback callback) in D:\a\_work\1\s\src\Core\src\obj\Release
et7.0-android\generated\src\Microsoft.Maui.PlatformInterop.cs:line 443
   at Microsoft.Maui.StreamImageSourceService.LoadDrawableAsync(IImageSource imageSource, ImageView imageView, CancellationToken cancellationToken) in D:\a\_work\1\s\src\Core\src\ImageSources\StreamImageSourceService\StreamImageSourceService.Android.cs:line 28
  --- End of managed Android.Util.AndroidRuntimeException stack trace ---
android.view.ViewRootImpl$CalledFromWrongThreadException: Only the original thread that created a view hierarchy can touch its views.
	at android.view.ViewRootImpl.checkThread(ViewRootImpl.java:9758)
	at android.view.ViewRootImpl.requestLayout(ViewRootImpl.java:1959)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.widget.ImageView.setImageDrawable(ImageView.java:602)
	at androidx.appcompat.widget.AppCompatImageView.setImageDrawable(AppCompatImageView.java:112)
	at com.microsoft.maui.glide.MauiCustomViewTarget.onResourceCleared(MauiCustomViewTarget.java:30)
	at com.bumptech.glide.request.target.CustomViewTarget.onLoadCleared(CustomViewTarget.java:210)
	at com.bumptech.glide.request.SingleRequest.clear(SingleRequest.java:337)
	at com.bumptech.glide.request.ErrorRequestCoordinator.clear(ErrorRequestCoordinator.java:48)
	at com.bumptech.glide.manager.RequestTracker.clearAndRemove(RequestTracker.java:70)
	at com.bumptech.glide.RequestManager.untrack(RequestManager.java:660)
	at com.bumptech.glide.Glide.removeFromManagers(Glide.java:913)
	at com.bumptech.glide.RequestManager.untrackOrDelegate(RequestManager.java:647)
	at com.bumptech.glide.RequestManager.clear(RequestManager.java:624)
	at com.bumptech.glide.RequestBuilder.into(RequestBuilder.java:811)
	at com.bumptech.glide.RequestBuilder.into(RequestBuilder.java:780)
	at com.bumptech.glide.RequestBuilder.into(RequestBuilder.java:771)
	at com.microsoft.maui.PlatformInterop.prepare(PlatformInterop.java:229)
	at com.microsoft.maui.PlatformInterop.loadInto(PlatformInterop.java:234)
	at com.microsoft.maui.PlatformInterop.loadImageFromStream(PlatformInterop.java:265)

  --- End of managed Android.Util.AndroidRuntimeException stack trace ---
android.view.ViewRootImpl$CalledFromWrongThreadException: Only the original thread that created a view hierarchy can touch its views.
	at android.view.ViewRootImpl.checkThread(ViewRootImpl.java:9758)
	at android.view.ViewRootImpl.requestLayout(ViewRootImpl.java:1959)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.view.View.requestLayout(View.java:26281)
	at android.widget.ImageView.setImageDrawable(ImageView.java:602)
	at androidx.appcompat.widget.AppCompatImageView.setImageDrawable(AppCompatImageView.java:112)
	at com.microsoft.maui.glide.MauiCustomViewTarget.onResourceCleared(MauiCustomViewTarget.java:30)
	at com.bumptech.glide.request.target.CustomViewTarget.onLoadCleared(CustomViewTarget.java:210)
	at com.bumptech.glide.request.SingleRequest.clear(SingleRequest.java:337)
	at com.bumptech.glide.request.ErrorRequestCoordinator.clear(ErrorRequestCoordinator.java:48)
	at com.bumptech.glide.manager.RequestTracker.clearAndRemove(RequestTracker.java:70)
	at com.bumptech.glide.RequestManager.untrack(RequestManager.java:660)
	at com.bumptech.glide.Glide.removeFromManagers(Glide.java:913)
	at com.bumptech.glide.RequestManager.untrackOrDelegate(RequestManager.java:647)
	at com.bumptech.glide.RequestManager.clear(RequestManager.java:624)
	at com.bumptech.glide.RequestBuilder.into(RequestBuilder.java:811)
	at com.bumptech.glide.RequestBuilder.into(RequestBuilder.java:780)
	at com.bumptech.glide.RequestBuilder.into(RequestBuilder.java:771)
	at com.microsoft.maui.PlatformInterop.prepare(PlatformInterop.java:229)
	at com.microsoft.maui.PlatformInterop.loadInto(PlatformInterop.java:234)
	at com.microsoft.maui.PlatformInterop.loadImageFromStream(PlatformInterop.java:265)

This is inconsistent but happens roughly 1/6 times. Most of the time it works. But once it breaks, the whole app gets glitchy.

I wonder if there is any way to create an Image entirely on the main thread. From what I see there is no single threaded option to circumvent this.

I also tried to use FromStream as follows but it caused the same problem, which makes sense if the issue is occurring at the end of the job not the start (how it is started makes no difference):

                    Assembly assembly = GetType().GetTypeInfo().Assembly;
                    Stream getFromStream() {
                        Stream stream = assembly.GetManifestResourceStream(resourceName);
                        return stream;
                    };
                    imageView.Source = ImageSource.FromStream(getFromStream);

If it matters, this is while attempting to load images listed in my project as "Embedded Resources" (not "MauiImage"). Again, I cannot see the same problem on Windows or iOS with the same code and I can't be any clearer about wanting this to main threaded.

Any thoughts or ideas?

Both issues only occur in Android and both resolve when I comment out the imageView.Source = ...

Repro Project

https://github.com/jonmdev/ImageSource-Bug

• This project loads two random cat photos to screen each time you click on screen.
• Click screen a few times and error will typically be provoked within 1-3 clicks in Android using Pixel 5 API 33 and .NET 7.

Version with bug

7.0.92

Is this a regression from previous behavior?

No, this is something new

Last version that worked well

Unknown/Other

Affected platforms

Android

Affected platform versions

Android API 33, .NET 7.0

[jonmdev]

I was able to reproduce it with about 180 lines of code. Posted repro project here:

https://github.com/jonmdev/ImageSource-Bug

This project just loads two random cat photos to screen each time you click the screen and resizes them. Image loading and resizing is managed with a small ImageLoader Behavior.

Within 1-5 clicks it will spew out the errors posted in the OP about Android.Util.AndroidRuntimeException: Only the original thread that created a view hierarchy can touch its views.

Repro code, for reference, to replace App.xaml.cs:

using System.Diagnostics;
using System.Reflection;
using SkiaSharp;

namespace ImageSource_Bug {
    public partial class App : Application {

        Image image1;
        Image image2;
        public event Action screenSizeChangedEvent = null;
        public double screenHeight = 0;
        public double screenWidth = 0;
        ImageLoader loader1;
        ImageLoader loader2;
        List<string> imageList = new();

        //CODE USAGE INSTRUCTIONS:
        //CLICK SCREEN TO LOAD NEXT TWO PHOTOS, KEEP CLICKING UNTIL ERRORS SHOW UP (ANDROID BUG ONLY, WORKS IN IOS/WINDOWS)
        //TYPICALLY WILL HAPPEN WITHIN 1-5 CLICKS

        public App() {
            InitializeComponent();

            ContentPage mainPage = new();
            MainPage = mainPage;

            //screen size monitor
            mainPage.SizeChanged += delegate {
                if (mainPage.Height > 0 && mainPage.Width > 0) {
                    screenWidth = mainPage.Width;
                    screenHeight = mainPage.Height;
                    screenSizeChangedEvent?.Invoke();
                }
            };

            //layout composition
            AbsoluteLayout abs = new();
            mainPage.Content = abs;
                        
            //images
            image1 = new();
            image2 = new();
            image1.Aspect = Aspect.Fill; 
            image2.Aspect = Aspect.Fill;
            abs.Children.Add(image1);
            abs.Children.Add(image2);

            //image loader
            loader1 = new();
            loader2 = new();
            image1.Behaviors.Add(loader1);
            image2.Behaviors.Add(loader2);

            //image reference
            buildImageList();

            //border to take taps
            Border border = new();
            border.BackgroundColor = Colors.White;
            border.Opacity = 0;
            abs.Children.Add(border);

            //tap function
            TapGestureRecognizer tapRecognizer = new();
            border.GestureRecognizers.Add(tapRecognizer);
            tapRecognizer.Tapped += delegate {
                Debug.WriteLine("TAP RECOGNIZED");
                if (imageList.Count > 0) {
                    loader1.loadImageFromEmbeddedResource(imageList[new Random().Next(0, imageList.Count)]);
                    loader2.loadImageFromEmbeddedResource(imageList[new Random().Next(0, imageList.Count)]);

                    resizeAndPositionImages();
                }
            };

            //screen size monitoring
            screenSizeChangedEvent += resizeAndPositionImages;
            screenSizeChangedEvent += delegate {
                border.HeightRequest = screenHeight;
                border.WidthRequest = screenWidth;
            };

            //DebugTools.debugResources();

            Debug.WriteLine("APP BUILT ON MAIN THREAD: " + MainThread.IsMainThread);
        }
        public void buildImageList() {
            for (int i=0; i <11; i++) {
                imageList.Add("ImageSource_Bug.Resources.Images.cats" + (i+1).ToString() + ".jpg");
            }
        }
        public void resizeAndPositionImages() {
            if (loader1.rawWidth > 0 && loader2.rawWidth > 0) {
                loader1.setDimensionsProportionatelyFromHeight(screenHeight * 0.5);
                loader2.setDimensionsProportionatelyFromHeight(screenHeight * 0.5);
                image2.TranslationY = screenHeight * 0.5;
                image1.TranslationX = (screenWidth - loader1.lastWidthRequest) * 0.5;
                image2.TranslationX = (screenWidth - loader2.lastWidthRequest) * 0.5;
            }
        }
    }

    public class ImageLoader : Behavior<Image> {
        public Image imageView;

        //store image raw width/height on accessing
        public int rawWidth = 0;
        public int rawHeight = 0;
        
        string resourceName = null;

        public double lastHeightRequest = 0; 
        public double lastWidthRequest = 0; 

        protected override void OnAttachedTo(Image ve) {
            imageView = ve;
            if (rawHeight > 0 && resourceName != null) {
                applyPhoto();
            }
            base.OnAttachedTo(ve);
        }
        protected override void OnDetachingFrom(Image ve) {
            imageView = null;
            base.OnDetachingFrom(ve);
        }
        public void loadImageFromEmbeddedResource(string resourceName, bool justReadForSizeManagement = false) { //make it just for size management if nothing to apply - applying photo throgh tinter instead

            Debug.WriteLine("LOAD IMAGE FROM EMBEDDED RESOURCE IS ON MAIN THREAD: " + MainThread.IsMainThread);

            this.resourceName = resourceName;
            Assembly assembly = GetType().GetTypeInfo().Assembly;

            //https://stackoverflow.com/questions/10984336/using-statement-vs-idisposable-dispose
            using (Stream stream = assembly.GetManifestResourceStream(resourceName)) {
                SKBitmap rawBitmap = SKBitmap.Decode(stream); // a bit inefficient as must decode fully bitmap just to get height/width but good enough //DISCARDS THE STREAM
                rawHeight = rawBitmap.Height;
                rawWidth = rawBitmap.Width;

                rawBitmap.Dispose();
                Debug.WriteLine("GOT IMAGE WIDTH " + rawWidth + " HEIGHT " + rawHeight + " " + resourceName + " ON MAIN THREAD: " + MainThread.IsMainThread);

            }
            if (!justReadForSizeManagement) {
                applyPhoto();
            }

        }
        private void applyPhoto() {

            Debug.WriteLine("APPLY PHOTO IS ON MAIN THREAD: " + MainThread.IsMainThread);

            MainThread.BeginInvokeOnMainThread(new Action(() => {
                if (resourceName != null && imageView != null) {

                    Assembly assembly = GetType().GetTypeInfo().Assembly;
                    Stream getFromStream() { return assembly.GetManifestResourceStream(resourceName); };

                    Debug.WriteLine("IMAGE SOURCE FROM RESOURCE IS ON MAIN THREAD: " + MainThread.IsMainThread);
                    //imageView.Source = ImageSource.FromStream(getFromStream);
                    imageView.Source = ImageSource.FromResource(resourceName, assembly); 
                    Debug.WriteLine("IMAGE SOURCE FINISHED ON MAIN THREAD: " + MainThread.IsMainThread);
                }
            }));

        }
        public void setDimensionsProportionatelyFromHeight(double heightToSet) { //return the expected height for reference
            if (rawHeight > 0 && rawWidth > 0) {

                lastHeightRequest = heightToSet;
                lastWidthRequest = heightToSet * rawWidth / rawHeight;

                imageView.WidthRequest = lastWidthRequest;
                imageView.HeightRequest = lastHeightRequest;

                Debug.WriteLine("RAW WIDTH " + rawWidth + " " + rawHeight + " height request " + heightToSet + " Current width " + imageView.Width + " IS ON MAIN THREAD: " + MainThread.IsMainThread);
            }
        }
    }
    public static class DebugTools {
        public static void debugResources() {
            foreach (string currentResource in Assembly.GetExecutingAssembly().GetManifestResourceNames()) {
                Debug.WriteLine(currentResource);
            }
        }
    }

}

[jonmdev]

For whatever it's worth, I wrote my own custom code using the handler and native views to load my images without the async function it was using by default (I really think we ought to have a choice about async vs. sync) and it's completely solved with just this method.

My solution
Image maps to UIImageView in iOS, ImageView in Android, and a Windows class also called Image:
https://github.com/dotnet/maui/blob/ff50be2f541a7eef49243c4d23c4615db20fc581/src/Core/src/Handlers/Image/ImageHandler.cs

UIImageVIew loads UIImage. ImageVIew loads Android Bitmaps, and Windows loads BitmapImage. SKBitmap can be directly converted to Android Bitmap or UIImage by the Skiasharp.Views package. If you have an image loaded to a memory stream, this can be used to make a BitmapImage as well.

Windows was the worst one and took me hours to figure out but I was able to make a non async memory stream and new BitmapImage to assign to the source without any async either. I wanted this anyway

My suggestion
Irrespective of the bug I think you have a few glaring holes in the current image management system.

1. The image system requires async load at all times even though many of us might prefer immediate load (even if it lags for a second avoids visual glitches).
2. If people are to be stuck with async, then there must be a "load completed" event so we can subscribe to it and delay layout changes until that happens.
3. You provide no "Generic Bitmap" type class to wrap BitmapImage, Android Bitmap, or UIImage, so there is no way to directly load into the image rather than the view that holds it or cache/manipulate these accordingly.
4. No data like height or width of the loaded graphic is easily accessible.

I might be missing something. But if it was me, I would make a "Generic bitmap" class with a variety of ways to load one. I would duplicate those methods into the Image class as well so people can bypass the "Generic bitmap" class when wanted and go straight into the Image.

But this is just my opinion as I have been thinking about. It doesn't personally effect me as I understand the system well enough at least at this point to build exactly that for my own needs. If you are interested in improving the system, I think this should be a necessary step at some point.

Decoupling the "Texture" or "Bitmap" from the "View" that holds it is a good step in most systems for design purposes.

[jstedfast]

I believe this is fixed in net8. This looks exactly like an issue that I fixed back in April: #14109

The back-port got stalled for net7.x until very recently: #16640

[jonmdev]

Yes @jstedfast, I just saw that user posting about this one asking for a backport and recognized it immediately as well. I linked his thread above. Thanks for your effort.

Unrelated to the fix, just from having dug into the system a bit, I still think you guys should think about breaking the Image "View" class apart from the Image "Texture/Bitmap" and having a native "Texture/Bitmap" class (UIImage, Android BItmap, BitmapImage) but no big deal - bigger fish to fry.

I can already write my own version for my own needs and I'm doing so. I just think having them separated out would improve the general usability of the system I think in many ways though for others.

You can load and cache the image files, load them async or sync, set multiple Image Views to one image bitmap, etc. It gives a lot of options compared to marrying it to the image view itself and locking people just into async load. This bitmap/texture class could also carry the image height/width inside it for reference.

I also think if you're going to stick with locked in async load there ought to at least be a "LoadCompleted" event or something for this people can subscribe to. Without that it's hard to build anything that responds correctly to the async.

But this is not critical for me. I am writing my own code for all this. I just mention it for MAUI benefit or other users. Thanks again. Feel free to close this one then if it's solved. Thanks.

[samhouts]

Duplicate of #14052