view.takeScreenshot does not work for react-native-skia canvas

[mauricedoepke]

What happened?

When taking a screenshot of a view, react native skia content does not show up.

More data, please!

This issue occures because detox uses a views internal draw function to create the screenshots:

Detox/detox/android/detox/src/main/java/com/wix/detox/espresso/action/ViewScreenshot.kt

Lines 20 to 27 in 620ca86

	class ViewScreenshot() {
	fun takeOf(view: View): ScreenshotResult {
	val bitmap = Bitmap.createBitmap(view.width, view.height, Bitmap.Config.ARGB_8888)
	view.draw(Canvas(bitmap))
	return ScreenshotResult(bitmap)
	}
	}

rn skia uses a TextureView to draw its contents, this however does not support drawing itself to canvas:
Subclasses of TextureView cannot do their own rendering with the [Canvas](https://developer.android.com/reference/android/graphics/Canvas) object.

https://developer.android.com/reference/android/view/TextureView#draw(android.graphics.Canvas)

It seems possible to implement the desired behaviour through the getBitmap method of TextureView though:
https://github.com/facebook/screenshot-tests-for-android/pull/71/files

The skia maintainer confirmed this issue some time ago already:
Shopify/react-native-skia#1880

[mauricedoepke]

I was trying to work on this already. For that I modified my local detox installation within node_modules to make a case distinction for the screenshots:

class ViewScreenshot() {
    fun takeOf(view: View): ScreenshotResult {
        println("-----------------Screenshot-------------------")
        if (view is TextureView) {
            throw Exception("-----------------TextureView-------------------")
            val bitmap = view.getBitmap(view.getWidth(), view.getHeight())
            return ScreenshotResult(bitmap)
        }
        if (view is ViewGroup) {
            throw Exception("-----------------ViewGroup-------------------")
            val bitmap = view.getChildAt(0).getBitmap(view.getWidth(), view.getHeight())
            return ScreenshotResult(bitmap)
            
        }
        throw Exception("-----------------View-------------------")
        val bitmap = Bitmap.createBitmap(view.width, view.height, Bitmap.Config.ARGB_8888)
        view.draw(Canvas(bitmap))

        return ScreenshotResult(bitmap)
    }
}

But for some reason, my code does not get executed when running the detox tests. With detox test also rebuilding with detox build did not help.

Can you give me some hints on how to make detox test actually use the modified code? I am sure it did not, because none of the excpetions were thrown.

[noomorph]

@mauricedoepke doesn't Detox get precompiled already to your node_modules? I mean, changing source code is not enough anyway... 🤷‍♂️

[mauricedoepke]

Thanks @noomorph

With your tip, I found out, that npm build:android inside the detox folder inside my node_modules helped.

I came up with a modification to ViewScreenshot that works in my project so far. It renders the skia canvas and does not interfere with normal views:

class ViewScreenshot() {
    fun drawTextureViews(view: View, canvas: Canvas) {
        if (view is TextureView) {
            val viewBitmap = Bitmap.createBitmap(view.width, view.height, Bitmap.Config.ARGB_8888)
            view.getBitmap(viewBitmap)
            canvas.drawBitmap(viewBitmap, 0f, 0f, null)
        } else if (view is ViewGroup) {
            for (i in 0..(view.getChildCount() - 1)) {
                val childContainerPos = view.getChildDrawingOrder(i)
                val childView = view.getChildAt(childContainerPos)

                val left = childView.left.toFloat()
                val top = childView.top.toFloat()

                canvas.translate(left, top);
                this.drawTextureViews(childView, canvas)
                canvas.translate(-left, -top);
            }
        }
    }

    fun takeOf(view: View): ScreenshotResult {
        val bitmap = Bitmap.createBitmap(view.width, view.height, Bitmap.Config.ARGB_8888)
        val canvas = Canvas(bitmap)

        view.draw(canvas)
        this.drawTextureViews(view, canvas)

        return ScreenshotResult(bitmap)
    }
}

Would you guys be open to a PR with that?

If yes, should the drawTextureViews method stay inside ViewScreenshot or go somewhere else?
Do you have any other hints or tips for me for the pr?
I guess adding a react-native-skia canvas to the test app for an e2e test would be good?

I drew a bit of inspiration from:
https://github.com/facebook/screenshot-tests-for-android/pull/71/files
But I think my code is sufficiently different from it, so that it should not interfere with Detox' MIT license?

[noomorph]

@gosha212 can you take a look?

[gosha212]

@mauricedoepke You are more than welcome! we are always open for new contributions.
Please follow the contribution guide, add e2e test that reproduces your problem and than fix it.

[stale]

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs.
If you believe the issue is still relevant, please test on the latest Detox and report back.

Thank you for your contributions!

For more information on bots in this repository, read this discussion.