Roborazzi 1.37.0 and above brokes almost all screenshots in our app

[tomoya0x00]

We have not yet been able to fully analyze the issue, but #555 has caused almost all screenshots for our app to be just filled with the background color.

Apparently, in the case of our app, even if we don't include Dialog in Previews, the condition of fetchRobolectricWindowRoots ().size > 1 is met and the captureScreenRoboImage is executed.
However, the screenshot we get from it is simply an image filled with the background color.

We're planning to make a sample project that can reproduce this problem, but please give us some time.

If possible, it would be helpful if you could add a flag to always run captureSingleComponent instead of captureScreenRoboImage as described in captureScreenIfMultipleWindows.

[takahirom]

Thank you for reporting this. I'm not sure why you always have two windows. A project reproducing the issue would be much appreciated. 🙏

[takahirom]

Could you check if the Activity is being launched twice? It might be because you're using a Compose rule with ActivityScenario.launch, or something else running at the same time.

[tomoya0x00]

Thanks for the advice.
Our screenshot test calls ComposeContentTestRule#setContent when a Preview contains a Dialog, so I think this is what is triggering the Activity launching.

Here is an excerpt of our screenshot test code.

class ScreenshotTestForPreview(
    private val component: ComposablePreview<AndroidPreviewInfo>,
) {
    @get:Rule
    val composeTestRule = createComposeRule()

    @OptIn(ExperimentalRoborazziApi::class, InternalRoborazziApi::class)
    @Test
    @Category(ScreenshotTestCategory::class)
    fun previewScreenshot() {
        val previewName = component.methodName
        if (previewName.contains("Dialog") || previewName.contains("DropdownMenu")) {
            // Dialogs and DropdownMenu are not supported by captureRoboImage(), so we use captureScreenRoboImage() to capture the entire screen.
            // To jump into this condition, the Dialog @Preview composable must be named with "Dialog",
            // and the DropdownMenu @Preview composable must be named with "DropdownMenu".
            composeTestRule.setContent {
                component.invoke()
            }
            runCatching {
                @OptIn(ExperimentalRoborazziApi::class)
                (captureScreenRoboImage(filePath))
            }.onFailure {
                throw IllegalStateException(
                    "Failed to captureScreenRoboImage[$previewName]",
                    it,
                )
            }
        } else {
            runCatching {
                captureRoboImage(
                    fileWithRecordFilePathStrategy(filePath),
                ) { component.invoke() }
            }.onFailure {
                throw IllegalStateException(
                    "Failed to captureRoboImage[$previewName]",
                    it,
                )
            }
        }
    }
}

When we tried deleting this conditional branch and just calling captureRoboImage with roborazzi 1.39.0 all the time, We could generate normal screenshots.

However, the above changes have caused another problem.
The screenshot taking process, which normally takes 10 minutes, does not finish even after 50 minutes, and some Previews cause OOM.

We will try to create a sample project to reproduce the problem.

[takahirom]

Thank you for providing the updated information. I usually debug this kind of problem by using the debugger in Android Studio when the app seems to freeze or do nothing, to understand what is stopping the tests.
For OOM, I'm not sure, but you might try using VisualVM to get the heap dump. It usually occur Activity leak or something.

[tomoya0x00]

Thx for your kind advice.
We will investigate more details next week.

[tomoya0x00]

I will share what we know at this point. Further research will be conducted next week.

• Running a specific Compose Preview with fun captureRoboImage or composeTestRule.setContent will cause OOM.
• However, running that Compose Preview with composeTestRule.activity.setContent does not cause OOM, and executing fun View.captureRoboImage on the Activity's View will correctly capture a screenshot

I am very surprised that composeTestRule.setContent and composeTestRule.activity.setContent change whether OOM occurs or not.

[takahirom]

It might be a memory leak, but you can try specifying the memory size. I think we need some memory even if we don't have a memory leak.
https://github.com/takahirom/roborazzi?tab=readme-ov-file#q-i-am-seeing-out-of-memory-errors

[tomoya0x00]

Thanks for your kind advice. I tried it, but now I get an AppNotIdleException instead.

androidx.test.espresso.AppNotIdleException: Compose did not get idle after 252223 attempts in 60 SECONDS. Please check your measure/layout lambdas, they may be causing an infinite composition loop. Or set Espresso's master idling policy if you require a longer timeout. The following Idle Conditions failed.

However, we have identified the cause. Certain Lottie usage causes this issue.
If speed=0 is specified to stop the animation, as shown below, there is no problem with the display in composeTestRule.activity.setContent, but in composeTestRule.setContent AppNotIdleException or OOM.

    var showAnimation by remember {
        mutableStateOf(false)
    }
    val composition by rememberLottieComposition(
        RawRes(R.raw.lottie_animation)
    )

    val state = animateLottieCompositionAsState(
        composition = composition,
        speed = if (showAnimation) 1.5f else 0f, // main point
    )

If rewritten as follows, composeTestRule.setContent can be displayed without problems.

    var showAnimation by remember {
        mutableStateOf(false)
    }
    val composition by rememberLottieComposition(
        RawRes(R.raw.ui_common_star_rating_button_animation)
    )
    val state = animateLottieCompositionAsState(
        composition = composition,
        isPlaying = showAnimation, // main point 
        speed = 1.5f, // main point
    )

[tomoya0x00]

Let's summarize.

Roborazzi 1.37.0 and above brokes almost all screenshots in our app

Since this occurs when val composeTestRule = createComposeRule() and fun captureRoboImage are used together, it can be avoided by stopping the use of either.

Although considered a rare case, if AppNotIdleException or OOM occurs, it may be due to the way Lottie is used.
#628 (comment)

[takahirom]

We are currently working on fixing the animation problem by introducing @RoboManualAdvance. However, it will only work with Roborazzi's Compose Preview Support.
#633

[tomoya0x00]

Thanks for the additional information.

This Issue itself has been resolved and will be closed. Thank you for all your kind support.